Maybe it's just me, but surfing the Web lately has become an increasingly frustrating experience. Without question, the major search engines have made tremendous strides in expanding the scope of their coverage, increasing the frequency of their updates, and most importantly of all, discerning the relevancy of search results. This is all for the good. Yet at the same time, an equal if not greater amount of intellectual energy and creativity has been expended on getting between users and their search results in the name of commerce. And the cumulative impact of all these tricks and games is beginning to counteract some of the true advances in search.

I first noticed this with paid search links. Every so often, I'd be searching on something incredibly obscure, and my search results would contain a paid ad from eBay claiming the item was for sale on its site. "Not darn likely," I thought, and after some searching on eBay, I determined I was correct. eBay wants me to visit its site so badly it's willing to misrepresent itself to maximize traffic. I am seeing a number of retailers doing this as well. Type in a search for Brand A, and you'll see lots of links to retailers claiming to sell that brand. Click on the link, and you'll find they sell only the competitor to Brand A. Clever online marketing? No, because the retailer has angered me by wasting my time. Good for search engines? No, because these experiences teach me to discount and distrust paid links.

You've probably also noticed the growth in pseudo-directories; sites that you land on by accident (or, increasingly, by misrepresentation) that contain endless lists of paid links, or search features that produce only lists of paid links as results. A close cousin of these sites are the notorious "link farms," sites created to make other sites look more popular and thus raise their ranking in search results. There are also a growing number of links that re-direct you from content you want to see to content a merchandiser wants you to see.

Is this a problem? I think it is, because it reduces the ability of the search engines to deliver a fast and dependable result. Also, keep in mind that I am just discussing navigational sleight of hand. This doesn't even begin to address the growing issue of content accuracy and trustworthiness.

The more I think about all these issues, the more I see a strong future for some form of vertical search. Interestingly, vertical search to date has focused on presenting users with highly focused content, which has value. That's all well and good, but the more I use the general search engines, the more I think that the real value of vertical search will be not only that it provides focused content, but that at the same time it is also providing filtered content, creating treasure by taking out the trash.