Yahoo! Crawling to a Halt on GZ files?
Thursday, 19 July 2007
Well, I really hope the news is true but I’ve reason to believe Yahoo is freezing up on a sitemap compression standard that allows us to reduce the file size of large sitemaps. Months ago, Yahoo, Google, and Microsoft (and more recently Ask) announced that they has agreed to the sitemap protocol outlined at sitemaps.org.
- Published in Search
No Comments
Google Bots and robots.txt
Saturday, 27 January 2007
Just a heads up that there is a an official statement from Google about your robots.txt file with a rarely discussed comment about how to disallow the spider from indexing sub-directories They will have a series of posts about how to manage access to your site and its pages so subscribe or come back here
- Published in Google, Natural Search / SEO, Search