We like to think of web search as quite a polished tool. I still find myself amazed at the ease with which difficult questions can be answered just by googling them. Is there really much to go from here?
Brynn Evans has a great post on why social search won’t topple Google anytime soon. In it, she shares some yet to be published results on difficulty in forming the query being a major cause for failed searches. That resonated well with some citations I’m collecting right now for my thesis (on concept-based information retrieval). It also reminded me of a post Marissa Mayer of Google wrote some months ago, titled “The Future of Search“. One of the main items on that future of hers was natural language search, or as she put it:
This notion brings up yet another way that “modes” of search will change – voice and natural language search. You should be able to talk to a search engine in your voice. You should also be able to ask questions verbally or by typing them in as natural language expressions. You shouldn’t have to break everything down into keywords.
Mayer gives some examples to questions that were difficult to query or formulate by keywords. But clearly she has the question in her head, so why not just type it in? after all, Google does attempt to answer questions. Mayer (and Brynn too) mentions the lack of context as one reason. Some questions, if phrased naively, refer to the user’s location, activities or other context. It’s a reasonable, though somewhat exaggerated point. Users aren’t really that naive or lazy, if instead of using search they’d call up a friend, they wouldn’t ask “can you tell me the name of that bird flying out there?”. The same info they would provide verbally, they can also provide to a natural-language search engine, if properly guided.
The more significant reason in my eyes revolves around habits. Search is usually a means, rather than a goal. So we don’t want to think where and how to search, we just want to type something quickly into that good old search box and fire it away. It’s no wonder that the search engine most bent on sending you away asap, has most loyal users coming back for more. That same engine even has a button, that hardly anyone uses, and supposedly costs them over 100M$ a year in revenues, that sends users away even faster. So changing this habit is a tough task for any newcomers.
But these habits go deeper than that. Academic researchers have long studied natural-language search and concept-based search. A combination of effective keyword-based search, together with a more elaborate approach that kicks in when the query is a tough one, could have gained momentum, and some attempts were made for commercial products (most notable Ask, Vivisimo and Powerset). They all failed. Users are so used to the “exact keyword match” paradigm, the total control it provides them with, and its logic (together with its shortcomings) that a switch is nearly impossible, unless Google will drive such a change.
Until that happens, we’ll have to continue limiting innovations to small tweaks over the authorities…
The Alter Egozi 
For a twist on Google searching, try this query modifier at http://apodder.org/ControversyDiscoveryEngine.html. The premise is that simplified searches get trapped in an Organizational Web while the same query embellished with a few modifiers will drive the query into the Analytic Web. The web page references a 5 year old study on “Do Search Engines Suppress Controversy?”
Hi Susan,
I read your study with much curiosity. Obviously you’re aware of the authority-based nature which suppresses not only controversies, but any unpopular aspects. TREC ran some information retrieval challenges on “Topic distillation” or “Aspect retrieval”, with the purpose you’re aiming for – discovering the different aspects of the search query, rather than the authorities.
Of all current search trends, social search might just be the strongest chance to push against authority. The decentralized nature of the individual social graph means that authorities have less influence. Sites such as Digg, by nature single out and emphasize controversial topics. A search based on social news popularity might alleviate the authority-inclination, admittedly not enough.