SEOMining SEOMining


Search Engine Basics   «Prev  Next»
Lesson 5Challenges to finding Information on the Internet
ObjectiveExplain why you need a search strategy.

Challenges to finding Information on the Internet

You have seen some of the vast amount of information that exist on the Web, and you have seen ways that you can find it. It's understandable that you might think that all you have to do is pick a search service, select a category or enter a query, and exactly what you want appears in the results list. Well, sometimes it does, and sometimes it does not.
It is the nature of information on the Internet, and the nature of searching, that:
  1. Your search engine may not have visited the site that holds the information you need, or the site may be marked to hide it from search robots.
  2. Your search results may be "stale"--not current enough for your needs.
  3. The site may have been moved to another URL after it was indexed, or it is just no longer around.
  4. The words that have been indexed are similar to, but not the same as, what you enter in a search query.


(CTR) by SERP Position

Figure 2-5 shows the results from AOL data released in 2006 (http://www.webuildpages.com/jim/click-rate-for-top-10-search-results/). In addition, the first 10 results received 89.71% of all click-through traffic; the next 10 results (normally listed on the second page of results) received 4.37%, the third page 2.42%, and the fifth page 1.07%. All other pages of results received less than 1% of total search traffic clicks. A study on click-through rate by search position done by Cornell University showed similar results, but with an even higher skew toward the first position, with the first result getting 56.36% of the clicks. Why are searchers blind to relevant results farther down the page? Is this due to the "implied endorsement" effect, whereby searchers tend to simply trust the search engine to point them to the right thing? According to the Cornell study, 72% of searchers click on the first link of interest, whereas 25.5% read all listings on the first page and then decide which one to click. Both effects (implied endorsement and rapid cognition) most likely play a role in searcher behavior.
Figure 2-5: Click through rate (CTR) by SERP position
Figure 2-5: Click through rate (CTR) by SERP position