Danny Sullivan posted an interview of Jimmy Wales, in which they discuss Wikia’s plan to launch a new search engine. Danny asks a lot of great questions and provides some good analysis.
Danny points out in detail how hard it will be for them to succeed in holding off spammers if they begin to get popular.
One point of the discussion that I want to expand upon is the issue of the investment required to succeed at this. Danny touches on this as well, but I believe it deserves some expansion.
Personally, I believe that Google’s most important “core competency” is its ability to set up, manage, and leverage large server farms. I believe that they have displayed a unique capability in this area that can’t at all be trivialized. They do this better than Yahoo and Microsoft, who just aren’t too shabby in this area either.
While I don’t know the costs involved in designing and running a search engine that receives a large percentage of the world’s search traffic, I would imagine that crawling the entire web, indexing it somehow, and handling just 10% of the world’s search traffic would run into the hundreds of millions of dollars (maybe even billions).
I am likewise confident that putting in place the technical staff to make all this happen is also a daunting task. I think that there are lots of things that need to happen before this project can even get off of the ground.
Like Danny though, I am also hopeful about the underlying concept. As I have posted here before, I do think that there is a place for selective human review in making the best possible search engine. However, I don’t think a community element is the way to go here. I believe they will need to be paid employees of the search engine that get targeted on specific tasks that are flagged for them by algorithmic triggers.
