Thursday, December 18, 2008

Registering Web pages in search engines - a better way

Here's my year end nondenominational Christmas, Hanukkah, Kwanza and Eid el Fitr wish list from  Google. It's simple:
1- Don't penalize me for Google bugs. I really cannot control whether someone links to or or or If your spider and software are too dumb to understand they are the same pages, it is not my fault.
2- There is really no difference between a page that is in a subdirectory and one that is not. Really! If you are going to penalize pages that are in subdirectories, or index them later, you ought to be telling people that that's how the spider works. If you like flat directories, we will give you flat directories. Just ask!  
3- Most important - Give us all a quick and easy way to tell you when I have 1 or more pages or to tell you to index the whole site. There are a dozen good technical ways to solve that problem. The XML Site map is one of the bad ones. If you are going to insist on those maps, then provide a free tool that will crawl the site and submit the URLs in any format you like. But in addition to that, please give me an easy way to tell you that there is one new page at the site. I should not have to make a whole site map for a single page.  For bloggers it's easy. There is an RSS syndication file and that file can be sent to an aggregator. You probably use information from aggregators and blogs like Blogline. But what about sites that do not have an RSS feed? Why isn't there a simple interface for submitting a single URL to a queue? It could be used for new pages or changed pages. That could also take a load off Google software, since widespread use of such interfaces reduces the need for frequent spider crawls through thousands of pages to find just one that is really new or changed. It is incomprehensible why registration of new pages has to be such a hassle when there are simple and foolproof technologies available to solve this trivial problem.
Is that too much ask?
Anyone out there who sees this cry of despair - post it to Matt Cutt's blog and maybe Santa will answer our prayers.
Ami Isseroff

No comments: