Yahoo Sitemap Generator
Posted by Miles Evans
Although I specifically mention Movable Type for making a Yahoo sitemap generator this code will work for any website with a properly formatted Google sitemap...
I got an email from a friend commenting on my previous article on building some free organic rankings. His problem was that he was nailing his keywords, but that it was taking far too long. He also noted that he was getting nowhere getting listed high in the SERP’s at Yahoo. When I ask him if he is generating sitemaps on a regular basis for Google and Yahoo we quickly realize why this is happening. Last month I discussed automating Google Sitemaps for Movable Type and in this article I will describe how I do this for Yahoo as well.
As you may know Yahoo uses a different approach to sitemaps than Google. Instead of XML Yahoo just wants a list of URL’s in a text file. Searching around did not yield too much info but I finally found a solution on Elliot’s blog. His approach was to take the sitemap we generate automatically for Google and convert it to a sitemap for Yahoo with some PHP. Elliot is also kind enough to give away his code under a CCL (creative commons license). Having said that, I had some troubles with Elliot’s code and modified it slightly.
Download the Google to Yahoo sitemap code.
You should have two files: g2y.php and yahoo-sitemap.txt. Open g2y.php and change the top configuration lines to match that of your own website:
$input_file = "http://www.yoursite.com/sitemap.xml";
$output_file = "yahoo-sitemap.txt";
Now upload both files to your server where the Google sitemap.xml file lives and set the permissions on the yahoo-sitemap.txt file to chmod 777, or writable by anyone. Test the script to see if it is working by hitting www.yoursite.com/g2y.php. You see a bunch of URL’s from your website? Good!
Ok so now that you have the sitemap being generated how do you automate it? I stumbled on this for a bit but the best way seems to be good old fashioned cron. Open your crontab and add the following line:
0 * * * * root fetch http://www.yoursite.com/g2y.php
So this will run your Yahoo sitemap generator once/day. Now all that is left to do is head over to Yahoo and submit the yahoo-sitemap.txt file we just created. Unfortunately, unlike Google, you have no way to see if Yahoo is spidering your site or how frequently aside from checking your logs. Yahoo are you listening?
If you are lagging in the Yahoo SERP’s this should really help. If you do not see yourself showing up at all for a long period after implementing this, don’t panic – Yahoo spiders seem to be rather lazy. Eventually they will find you.
In a future article I might look at doing the same for MSN, but nailing the top of MSN is really kind of easy if you follow basic SEO practices. Naturally Miscrosoft spiders expect yet another standard for their sitemaps so this will take some investigation.
If this article helped you give me a digg!
Posted Mar 08, 2006 at 03:56 PM | Permalink | Trackback URL | Del.icio.us | DIGG!
Trackbacks
Listed below are links to weblogs that reference Yahoo Sitemap Generator:
How to Generate a Yahoo Sitemap with Movable Type from berbs.us
Recently, I discovered that Yahoo offers a way of submitting a simple sitemap to their Site Explorer service to let them know about all of the pages on your site. This is very similar in concept to Google's sitemap protocol... [Read More]
Comments
This sounds good for people already using the google sitemap generator.
If you want to know when the file is hit, you could make it a server-side script and just log it when it is read.
I'm not sure if yahoo requires it to be a .txt file, but if so, you can just create a rewrite rule ( if your server supports mod rewrite ) and call the php file that way.
Posted by sde on March 8, 2006 8:04 PM
Yep true. It's funny you mention it because I just wrote a bit of php that checks user agents against a list of known spiders and fires off an email of the visitors. It works good if your into that sort of thing...
I will release it here soon under CCL once I make sure it is working properly ;)
BTW - your site looks fantastic. Love the domain name.
Posted by Miles Evans on March 8, 2006 9:57 PM
I thought Yahoo sitemaps had to be called urllist.txt.
Am I wrong, or does yahoo accept both?
Posted by rick on March 28, 2006 4:22 PM
Nope you can use whatever you want, as you are submitting the blah.txt yourself. I believe urllist.txt confusion arose from Y!'s own example on how to do this.
So again it can be named whatever-you-want.txt
Posted by Miles Evans on April 3, 2006 2:04 AM
Hmmm, I really found this article useful, and guess what this has certainly improved my ranking of one of my site in Yahoo.
Posted by Raj Shah on April 28, 2006 3:19 AM
Hey, shound'n the crontab be 0 1 * * * .... to run just once a day?
best regards
Posted by Sebastian on May 10, 2006 11:55 PM
If you have a Google Sitemap already on your server you can use this Sitemap Converter (http://search-engine-marketing-services.net/Google_to_Yahoo_sitemap_creator.asp) to create a text file for submission to Yahoo. VERY easy to use.
Posted by Mark Barrera on May 12, 2006 7:17 AM
Hi, Thanks for usefull entry.
I got this msg in my email after setting the Cron:
/bin/sh: 0: command not found
Is it ordinary?
Posted by Dr. Ritalin on June 16, 2006 11:15 AM
This is the only program that indexes my 5000 page forum correctly. Thanks
Posted by Dog House + Kennel on July 29, 2006 7:02 AM
This is what I am getting:
Time: Thu, 24 Aug 2006 19:35:09 +0000 ------------------------------------------------------------- ------------------------------------------------------------- Total 0 URLS Time: Thu, 24 Aug 2006 19:35:09 +0000
What did I do wrong.
Posted by Josh on August 24, 2006 9:26 AM
Thanks
Posted by akshat on August 31, 2006 8:06 PM
Great. Thank you for this info.
Posted by Rihanna on September 3, 2006 4:24 AM
Hey, awesome stuff, i am interested in the mod rewrite if that becomes available :)
Posted by Dustin on January 14, 2007 4:33 PM
crontab doesn't know the command "root fetch"
Posted by jakeelee on February 2, 2007 10:55 PM
Yahoo uses a different approach to sitemaps than Google.
Posted by 武汉航空 on January 4, 2008 1:05 PM
Very useful
Posted by Jennifer on March 21, 2008 1:53 AM