Each of the three major search engines (Google, Yahoo! and Microsoft’s Live Search) have announced joint support of Google’s sitemaps protocol.

2007-04-11: Ask announces support of the sitemap standard. It is not yet clear if they actually use sitemaps. While Google and Yahoo do process sitemaps, Microsoft does not yet use them.

A new site, www.sitemaps.org, has been created to support the sitemaps protocol. While the Yahoo blog indicates Yahoo is apparently already accepting submissions, there is not mention of this on their Site Explorer submission form. Microsoft is committed to supporting sitemaps after finishing internal testing which is currently underway.

[Read more →]

In order to index and display web content in their search results, search engines need to be able to find the content. The first generation of Internet search engines relied on webmasters to submit a site’s primary URL, the site’s “home page”, to the search engine’s crawler database. The crawler would then follow each link it found on the home page. Problems soon emerged – much site content can be inadvertently hidden from crawlers, such as that behind drop-down lists and forms.

Update: Google Sitemaps was renamed Google Webmaster Tools on 5-Aug-2005 to better reflect its more expansive role.

Fast forward to 2005. Search engine crawlers have improved their ability to find sites through from other sites – site submission is no longer relevant. Yet many web sites are still coded in ways which impede automatic search engine discovery of the rich content often available in larger, complex web sites.

[Read more →]