How Search Engines Make Sense of the Web
Search engines are essentially massive full-text indexes of web pages. The quality of the indexes, and how the engines use the information they contain, is what makes -- or breaks -- the quality of search results.

We're all familiar with back-of-the-book indexes. They're simply alphabetized lists of the important words in the book, and the pages on which they appear.

Search engine indexes are similar, but vastly more complex that back-of-the-book indexes. Although most of us will never want to become experts on web indexing, knowing even a little bit about how they're built and used can vastly improve your searching skills.

A good way to learn about web indexing is to spend some time with a page compiled by The School of Library, Archival and Information Studies at the University of British Columbia. Indexing Resources on the WWW is a lengthy list of links to research, articles and web sites concerned with the process of indexing of all kinds.

SearchDay Readers will want to investigate two sections in particular. The first, Information Retrieval, looks at all types of searching.…
http://www.slais.ubc.ca/resources/indexing/www1.htm
http://www.slais.ubc.ca/resources/indexing/information.htm
http://www.searchenginewatch.com/searchday/article.php/2200541