Tim Mayer
The following is the transcript of an interview with Tim Mayer, who is responsible for the product direction of Yahoo! Search Technology.
Tim brings over 10 years of search experience to Yahoo! with previous roles at Overture, FASTSearch & Transfer, Inktomi and Thomson Corp. Tim was previously Vice President of Web Search Products at Overture where he led the product direction of the web search product as well as managing the major web search affiliate partner relationships. Prior to Overture, Tim was Vice President and General Manager of the FASTWeb Search division including the core search technology and the alltheweb.com search destination.
Before FAST, Tim spent two years leading product management and product marketing in the web search division at Inktomi. Before entering the web search area, Mayer worked at Thomson & Thomson as the project manager of the SAEGIS and NameStake search platforms.
Mayer received an MBA from Babson College, Wellesley, Mass., and a BA in English Literature from Hobart College, Geneva, New York.
Interview Transcript
Eric: Can you tell us how this product got started and what is the vision for Search Builder?
Tim: The basic concept was that a lot of people wanted the ability to easily create custom vertical search engines. People have specific interests and want to cover the long tail topics with these search engines. It could be a very specific topic like “surfing in Santa Cruz”, where no company is going to create such a specialized search engine, but an individual, with their own blog or site, may be interested in developing one. A lot of this was about leveraging that person’s expertise around a specific topic to create a superior search experience than you would get using a horizontal search engine.
As for some background on how we got started, we’ve been thinking about this since the Alta Vista days. Since even before Alta Vista was bought by Overture. Lots of different variations of product names, so it’s been a winding road.
Eric: It’s interesting, some business ideas never go away, they just change the name.
Tim: Yes, there are some really passionate people about this stuff. We demoed to Danny Sullivan about 3 years ago, an early prototype of this and he really liked it.
Eric: So you’re basically leveraging the power of human editorial input to provide a better search.
Tim: Yes, we refer to it as human knowledge. To allow other people who don’t have that expertise or knowledge to leverage it to get better search results.
Another aspect of the power of vertical search is that it allows the engine to disambiguate phrases or words that have different meanings in different contexts. You get automatic context around the search. Someone comes to a surfing site and you can assume the mode they’re in is about surfing.
Another thing we looked at is that site owners want to be able to create some differentiation for their site, not only in the content that they produce but also in the search experience that they provide their users. For example, the user might go to a site because they’re getting better results than they’re getting with a horizontal search engine. This can be a traffic driver, and also provide stickiness in that if users know of a site that does provide better answers for a specific topic, they’re more likely to go there.
Eric: Do you think a significant percentage of search traffic is going to go vertical, and go away from horizontal engines?
Tim: The advantages of vertical, aside from context as we talked about, is the use of structured data which helps you narrow your results by something like price, or if you’re searching by cities, say population. Using things like range searches, or requiring particular attributes. For example, saying “I want a blue shirt in size medium”, so don’t include things in my results if the shirt’s not available in blue. So in those two ways, vertical search can provide a better experience.
But we see a lot of people going to that “one box” in the results, where we’re providing a short cut to the answer to their question, which often times comes from a vertical, be it a stock quote, or weather or news. So it’s really integrating these vertical indices when we have pretty high confidence that they’re relevant. That’s really how a lot of vertical search is being leveraged now. With the one box, you’re getting a lot of vertical search answers showing up when appropriate, and users seem very happy with that.
Eric: One of the problems we see is by getting people to build a good vertical search engine. It’s easy to build one that’s worse than regular horizontal search, but it’s hard work to build a good one. Also, when you build a vertical search engine, users may not even be aware that they’re using one. So these may be obstacles to vertical search getting a large volume.
Tim: Yes, being in a specialized enough vertical and creating a good search engine are key issues. If you get into a vertical that’s very specific – like our example of surfing in Santa Cruz – if someone’s used their expertise to construct a vertical search engine with all the best sites about surfing in Santa Cruz, that’s going to provide a much better experience. The creator is vouching for the quality of each site, as well as culling out the noise of sites that aren’t relevant or authoritative enough.
And I’d agree that the user may not recognize they’re using a vertical search engine. Part of that comes down to proper messaging on the site. And enthusiasts are much more likely to see the difference between a vertical search engine and the horizontal engine.
Eric: So what’s Yahoo!’s strategy for driving adoption. Do you have any plans to help make this happen?
Tim: One strategy is to have features that will provide stickiness and differentiation for a site. Another product strategy is to provide the ability for publishers to share in the monetization of the traffic, which will provide them an economic incentive.
Eric: Along those lines, Search Builder includes a visible tag cloud, which should help draw the attention of users to the search engine. What benefits do you think it provides?
Tim: It’s a user interface that goes very well with the social search initiatives as well with Web 2.0. It provides the visitor to the site some ideas about what other people in the community are searching for and what’s “popping”. I think it will also drive more traffic because people will see interesting topics that may be timely or may be popular, causing them to click on it and initiate a search. Often times when users aren’t familiar with a topic they don’t know what terms to search on. So this can give them an idea of things to search for – they may say “that sounds interesting” or “I’d like to learn about that”. This leverages the expertise of the community to help the user learn about a specific topic.
Eric: Various startups have created some highly specialized vertical search engines with features like their own rendering of results and using their own crawls. How do you see that part of the market interacting with vertical search platforms like Search Builder?
Tim: These search engines are typically using taxonomies and structured data to provide higher relevance. These are a couple of things that aren’t typically provided by products in this space. This is a lot of work, and these vertical search engines often have large staffs working on these problems, maintaining the dictionaries and adding new fields to the structured data, etc.
But for the long tail, like our surfing in Santa Cruz example, products like Search Builder allow people to get very specialized without having to invest a lot of effort.
Eric: What are some of the ways that you can leverage community knowledge in search builder?
Tim: We’re always trying to leverage the knowledge of the community of the web at large to improve search for our users. This could be done in numerous ways. One way is to use knowledge and expertise to create niche vertical search engines with a tool like Search Builder. Another way is to use information that people provide by saving documents to del.icio.us, or myweb, or Yahoo! Bookmarks as a way to vouch for particular documents. This metadata will also help us improve our search efforts. We can figure out how to weight that based on peoples’ reputation.
We also have the Answers product where people are providing answers with content that may not appear on the web. Answers is trying to get people to document knowledge which may not be available on the web, to expand the knowledge base of the web. So next time someone has the same question, they can search and find that information.
So those are some of the ways that we’re using social search to improve general web search.
Eric: We saw in the documentation for Search Builder that putting a site into the search engine triggers a basic crawl. Can you tell us more about how that works?
Tim: What it does is to evaluate your site and potentially perform a deeper crawl of your site. For example, a lot of people want to create site search. With that, you want a comprehensive search of the site. Sometimes the site is fairly well indexed. What we’re saying is if you use Search Builder, we’ll potentially include more of your documents into our index.
It’s an incentive for people to use the Search Builder product on their site.
Eric: So another thing we noted is the ability to specify keywords to be included with every search. Can you tell us how you anticipate that feature being used?
Tim: For example, you can “not out” a word to help with disambiguation. For example in our surfing search engine, you might want to “not out” the term “web surfing”. So in this case, it’s about culling out a lot of the noise from your search. On the other hand, you may not want to have to list a lot of domains for your search, and instead specify what’s included by keyword, so it’s potentially a lot less work.
Eric: I believe that you can also implement some keywords that are automatically included in every query. So if you are in the Santa Cruz surfing search engine, you don’t need to type in “Santa Cruz” as part of every search query. Is that right?
Tim: Yes, that’s right.
Eric: Thank you very much for the time Tim.
Tim: You’re welcome. Enjoy Chicago! Make sure you go to the Yahoo! Party tonight.
