What To Do With Yahoo!
What to do with Yahoo!
Chances are, if you have a website and were previously paying for an Altavista listing you received something like this:
“AltaVista’s Paid Inclusion Program has changed. A New sign-up is necessary for existing AV express inclusion customers to become Overture Site Match members!
NOTE: YOUR ALTAVISTA MEMBERSHIP DOES NOT TRANSFER OVER TO THE NEW OVERTURE SITE MATCH PROGRAM. YOU MUST SIGN-UP YOUR URLs TO THE NEW OVERTURE SITE MATCH PROGRAM.”
This is an email which was sent to Altavista customers explaining to them that Altavista would no longer be accepting fees for inclusion.
Further to this, you’ve probably also noticed lots of press about Overture’s new Site Match program in which they say that you can pay to be included into the new Yahoo! database. In fact the above notice makes it seem that you must invest in Overture site match to remain in Altavista. You may also have heard that you need to pay for Site Match to get into Yahoo!
Also, just to mix things up a little more, you may have heard about the free Yahoo! submission option. While not a guarantee of if or when your site will be indexed, it is still a way into the index.
So the question I’m sure many people have is this: What the heck is the best way to get into Yahoo! (and by extension, Altavista and Alltheweb) and get found? Is it worth $29-$49 per URL plus 15-30 cents per click to be found in the index? The simple answer is it depends.
Before we get into that, let’s outline the various options currently there to help a website get listed in Yahoo!
The Options
Right now, there are multiple ways to have your pages listed in Yahoo! and most of them cost money.
From the top of a search results page down, let’s look at what the different listings mean.
Along the top and right of the screen are Overture sponsored listings. These are PPC listings in which the highest bids win the top spots. These are the easiest to identify and the simplest to explain.
Once you get into the “Web Results” however, it starts to get a little more convoluted.
Within web results you will find 3 or 4 different types of included results, depending on the competitiveness of the search. In this mass of results you will find Yahoo! directory listings, Yahoo! free listings via the Yahoo! Slurp crawler, Overture Site Match, and Overture Xchange. So what’s the difference?
Well, the Yahoo! directory hasn’t changed since the update last week; it’s still a US$300 submission fee with no guarantee of inclusion. And even if you are included there is no guarantee of rankings. Free listings are just that, listings gathered by either submitting via Yahoo! free submit, or having Yahoo! Slurp find your site another way (i.e. a link from another site).
Overture Site Match is a paid program whereby you pay a URL submission fee to get indexed followed by a cost per click on top of that for every click on your listing. Again, with no guarantee of ranking, only a guarantee of indexing.
Finally, is Overture’s Xchange which is an XML feed similar to the Inktomi XML feed. You have to request a rep from Overture to help you set this up. There is also no guarantee of listings, only a guarantee of indexing.
So, what is the best way to get into Yahoo!?
It depends on the current state of your site in Yahoo! among other things. Is your site currently indexed in the engine? And if so, how many pages are there? So let us start there.
Go to Yahoo! and do a search for your site. You may have to try a couple times to see your site. Try the search with and without the “www” but make sure you have the “.com” or “.ca” (or whatever) extension you have. If your site is indexed in Yahoo! then you should see your listing come up. It may also work with just the main domain, without the “www” and extension.
Now to see how many pages currently indexed for you site you should see a “More pages from this site” link near the bottom right of your listing. Click on this to see how many site pages are indexed. Again, this may take some experimenting with, to see how many pages are indexed. When I initially did a search for “searchengineposition.com”, it only showed one page, however a search for “searchengineposition” then clicking on the “more pages” link returned in excess of 1500. So be patient and experiment.
Chances are good that your site has already been indexed in Yahoo! which leads to the question (which I’m hoping you already know the answer to) “should I pay for Overture’s Site Match program?”
My recommendation is probably not. If you are already in Yahoo! as a result of the free crawl, then why pay for listings? In this case, if most (or all) of your site is indexed then paying for listings won’t influence the rankings.
However, if you find that there are many pages that aren’t indexed, but you feel should be, then perhaps you could consider paying for these pages to be listed. But not until you read on.
What if some (or all) of your site isn’t in Yahoo?
So let us now say that your site isn’t in Yahoo! or pages which you feel are really good aren’t getting indexed. What then? Should you pay for listings in Yahoo! or will it even help?
Again, I’d have to say probably not.
Before rushing out to pay more money, first take a look at some possible reasons on why your site may not be getting picked up by the Yahoo! crawler.
Check your log files to see if the crawler has been visiting your site. If you use a program like Webtrends you can create a custom profile which will analyze your logs specifically for spiders, or if you like, you can narrow the analysis down to just the Yahoo! Slurp crawler. This specific profile might be handy for my next point.
If the spider has visited, and you have the profile specific to this spider, take a look at what pages it has been requesting. Also, look at the paths through the site. Is the spider going deep enough into the site to get to your really important content? Or is it simply requesting one page at a time and not going any deeper than 2 or 3 clicks in? Is it merely requesting the index page over and over, or is it trying to crawl the whole site? Try and see if you can figure out what barriers (if any) the spider is coming up against. Remember that just because other spiders can crawl your site completely doesn’t mean that this one can.
Check your robots.txt file. Sometimes this is a huge barrier if it’s not coded properly. A good indicator of this will be your log analysis report. Is this file the only one being requested, or is the spider requesting a page, then the robots and stopping there? If this is the case, chances are the spider has somehow been excluded from indexing the site.
If you think there is a barrier, work towards fixing it and this may help getting your site re-crawled and re-indexed by Yahoo!
Talk to your hosting provider to see if perhaps they have banned any IP’s from accessing the server. Sometimes, if a hosting provider sees a spike in traffic from a particular IP or range of IP’s they will exclude it from accessing the server for security reasons. Perhaps they inadvertently banned Yahoo!s Slurp from accessing your site.
So what now?
If after your analysis you can’t see any reason why you may not be indexed in Yahoo! then perhaps performing a free submission is all you need to do. Try the free submit and see what happens. With searchengineposition for example, the site is re-cached every couple days, so you should see Slurp visiting your site fairly soon after submission. I know Yahoo! says that they can’t guarantee if or when your site will get crawled, but my gut tells me that it won’t be any longer than a free submit to Google. After all, their crosshairs are set on Google, so they want to at least maintain the same level of service as them.
Summary
I guess what I’m trying to say here is be sure that there isn’t some physical barrier to accessing the site before rushing out to pay for an Overture Inclusion. Because even if you do pay for inclusion, but you still have barriers to indexing, your site still won’t get indexed and you will be out $29-$49 per URL.
So before going out and spending a ton of money and assuming that that will get you in Yahoo! be sure that your site can be indexed. And if it is indexed in Yahoo! then paying won’t help influence your rankings anyways, so save your money and try other proper optimization efforts to positively influence your rankings.
Also remember that in the coming weeks your Yahoo! listing should translate to equivalent Altavista and Alltheweb listings (and to a lesser extent, a way into MSN), so even if you do disappear from Altavista or Alltheweb, be patient as you should come back relatively soon. Remember that these engines need an index to survive, and if the index is a centralized Yahoo! index all the better. In other words, you shouldn’t be in just Altavista and not Yahoo! (or vice versa).
On a more positive note, previously a site owner had to pay for an Inktomi inclusion to get into MSN. Now with the free submit into Yahoo! you should be able to get into MSN for a lot less money. Sure you are still at the mercy of MSN’s ranking algorithms, but there is a good chance that if you are already in Yahoo! that you will be in MSN as well, whether you’ve paid the listing fee or not.
Also, while I haven’t touched on it here, Overture is offering click tracking tools on their paid inclusion listings, including the Site Match. Therefore there may be benefit in paying for clicks if you think that the data collected is worth while. We are still analyzing the click tracking data, so are unable to say at this point how worthwhile it is, but we will keep you posted.
Author Bio:
Rob Sullivan
Production Manager
Searchengineposition.com
Search Engine Positioning Specialists
