Larry Cornett
Dr. Larry Cornett is VP of the Yahoo Search consumer products division. In this capacity, he is primarily responsible for the Yahoo web search experience and heads up a multinational team of product managers, designers, and developers focused on creating a world-class search experience. Prior to this role, Dr. Cornett was a director at eBay, where his team focused on multiple products for tailored shopping experiences, platform, and international sites. Earlier in his career, he was a designer at Apple Computer, where he worked on the Finder, OS, and international software; and IBM, where he worked on database and development tools.
He was also the principal consultant for MindSpan Design, an interaction design agency, where he worked on desktop, web, and mobile solutions for a variety of clients. Dr. Cornett received his Ph.D. from Rice University, where he designed and developed a coaching system for training software users. He holds multiple patents, which include design work on web-based products and hardware solutions.
Interview Transcript
Eric Enge: Can you start by providing us with an overview of SearchMonkey and some detail on what the program is all about?
Larry Cornett: SearchMonkey is part of our Open Strategy at Yahoo Search, which we launched early in the spring this year.
Essentially, what it’s doing is opening up the search results page in two ways. One way is to the publishers and developers so that they can actually influence how the search results are presented on the page. On the other side, it’s opening up the search page so that users can customize their search experience.
On the publisher-developer side, the magic is really in the structured data. It’s about those publishers providing us with access to structured data just like a data RSS feed, or if they have actually used something like Semantic markup on their site we can use that. Then, we have provided a template so the publisher can write a little bit of PHP Code to determine what kind of data they want to show for that URL and how they want that to be structured on our search page within that template format.
It’s truly given them a lot more control to be much more open with the search page. So, if you look at Yahoo Local, Yelp, Citysearch, Zagat, or our Yahoo! Video Player, for example, those are all turned on by default. The user can see those when they run a query and that result shows up in the search results. A user can still turn them off if they choose to, if they don’t want to see it. But, they can also go into the gallery and add any number of other applications that they find useful. So, if they really love WebMD for example, they can turn on that SearchMonkey application and see it for WebMD when they do searches.
Eric Enge: Does the user need to subscribe to the advanced view from that publisher?
Larry Cornett: The users can essentially do it two ways. A publisher or a developer who has access to that site can put a badge on it. Basically, they can promote it, and the user can click it and then be presented with a page to add the application. The other way is that a user can find them in the Yahoo Search Gallery. There have also been a few times where we have given the user the option to see an enhanced listing result directly in the search results. We did this with LinkedIn, for example.
Eric Enge: What kinds of publishers can benefit from the SearchMonkey program?
Larry Cornett: To be honest, I think everyone can benefit from this. The reality is that there are a lot of publishers these days and web pages are no longer just simple text or HTML. Now there is a lot of structured data and a lot of really interesting information on them. This gives them the ability to determine what is most meaningful for this URL and what they would like to have shown up in the search results. So, I think any publisher that has richer data that than could be exposed through a text abstract can benefit from this.
Eric Enge: What should publishers do if they are interested in this? How do they get started?
Larry Cornett: On http://developer.yahoo.com/searchmonkey we have two different sources of information that I think are very helpful. One is for site owners. Here we talk about the benefits for a publisher and how to get started. The developer site is much more developer centric because a developer can write a SearchMonkey Application even for a site he doesn’t own.
So, there is a different perspective on it; hence there are two different sources of information to help people get started.
Eric Enge: Right. So if I back up a bit, you mentioned that people can find applications in the gallery, and they can find the application as promoted by a badge on the publisher site.
Larry Cornett: That’s right.
Eric Enge: Is there anything you can share about the acceptance of the various versions that aren’t on by default?
Larry Cornett: The only thing I can really share is that we are seeing some very interesting engagement, and we’ve seen the CTR increase by at least 15% for some of our applications.
Eric Enge: Are you going to do more tests like the one you did with LinkedIn?
Larry Cornett: I don’t believe we have any that are in that mode right now that I could direct you to. But, we are going to continue to use that as a way to promote and test the high-quality applications that we think would be useful for users.
Eric Enge: So, you’ve mentioned there is a distinction between a level at which a developer can engage and a level at which a less technically astute publisher can engage. Can you expand on that a little bit?
Larry Cornett: I think the big difference is that if you are the publisher of that site you could officially change the full presentation of that resulting abstract. We don’t allow an independent developer to change someone else’s abstract and have that be an official Enhanced Result in our Gallery. Also, any independent developer who is not affiliated with that publisher can always write what we call an Infobar Application.
That essentially augments the result with that line at the bottom and lets people expand and see that additional information. And so, that is one big difference between being the publisher and the independent developer.
Eric Enge: Right. If you are the publisher and you create a neat Infobar type application, it actually might be shown as a supplemental result?
Larry Cornett: That’s right. It’s a line that’s added at the bottom of that result that lets you access the Infobar Applications.
Eric Enge: Interesting. So if we wanted to, we could create an Infobar Application to run off of Yelp or Citysearch?
Larry Cornett: That’s right. We have an example where an independent developer wrote an Infobar Application for Wikipedia results, and you could expand and see the table of contents and click the image at from the top of that page.
Eric Enge: Right. So, I could search on something now and find a result where Wikipedia has an enhanced result, and I’ll see the Infobar Application as well?
Larry Cornett: That’s right, yes.
Eric Enge: Interesting. Okay, so let’s talk a little bit about the formats of data and approaches people can use.
Larry Cornett: Okay, sure. So, there are a couple of different ways. One is if you’ve semantically marked up your page. So, if you are using RDF, RDFa or eRDF for example, we will recognize that when we index those pages. Then that is available in our index for any developer to access.
Eric Enge: So, that’s just semantic markup on the site?
Larry Cornett: That’s right. And then, if a publisher doesn’t want to go through and markup thousands, if not hundreds or of thousands of pages, he can just go back in time and provide you us with a DataRSS feed or create a Custom Data Service that essentially gives us the additional structured information that’s associated with that URL.
Eric Enge: How complicated a task are we talking about if we wanted to provide an RSS feed?
Larry Cornett: It’s essentially like a modified version of XML. We have some examples up on the site and there is a whole bunch of great documentation around this specific topic. It shows what it takes to create one of these feeds.
Eric Enge: Regarding markup, is that more complex than the RSS feed for the average publisher?
Larry Cornett: I think it is similar in terms of tags that identify names and phone numbers. It’s essentially a semantic tag and it’s not that different than what a lot of folks have started to do with XML.
Eric Enge: Ultimately any kind of content that you can put together in a structured form will be something you can implement fairly easily.
Larry Cornett: Right. We do recognize a number of other microformats like hCard, hCalendar, hReview, and so forth. There are a lot of different types of structured data that we are able to recognize.
Eric Enge: When someone has produced a feed or a markup on their pages, it’s just a matter of waiting for the crawler to find it. Alternatively, the DataRSS feed is a way to provide that directly to Yahoo to make you aware that it’s there?
Larry Cornett: Yes. As part of the SearchMonkey development process, we have a pretty simple step-by-step flow for creating one of these. One is to provide a reference for that data as part of the process of building the application.
Eric Enge: SearchMonkey doesn’t influence rankings or what queries you show up for, correct? It just influences what gets displayed?
Larry Cornett: That’s right. It has no influence on the ranking whatsoever. Basically, if it’s been specified that there is a SearchMonkey App and the user has turned it on, then it gets displayed. It doesn’t matter when or where the URL would show up
Eric Enge: What is it that makes for a successful SearchMonkey instrumentation?
Larry Cornett: We’ve had a lot of questions about that because we’ve had a number of developer workshops. What we’ve done is created a best practices document. For example, we have the ability to show a thumbnail image, but you should really only show a thumbnail image if it’s useful and relevant for that URL. If you just show a generic thumbnail image, it’s not necessarily a good thing. Users are looking for information in a result that is relevant for their query and makes sense.
Eric Enge: The big payoff is to increase the click-through rate, right?
Larry Cornett: I think there are a couple of payoffs. One is if you have users who already love your brand, trust your brand, and search for it. When they have turned on an application like this, it’s easier for them to quickly identify your result. And then, the other big part of this is that you are getting more qualified clicks.
People get a much better sense of what your site has to offer when you are showing the structured data. Instead of a user who is basically trying to make a decision based on a simple text summary, they are now getting rich structured data that helps them understand what they are going to see when they click on this.
Eric Enge: Right. You are going from a situation where the result is put together by a dumb software program trying to open a billion web pages and extract the meaningful context and put something up there that a user can use, or you have the site owner specify it.
Larry Cornett: If the site owner is very careful about what he shows, it will be successful. If a site owner has thrown everything but the kitchen sink into this just because they think more is better, then it may not be successful. So, they need to really think about what a searcher looks for and what is the most useful information to include.
Eric Enge: So, let’s take that a little further. If you have a webpage about blue widgets, then you really need to think about what search terms that someone would enter that you would hope would lead them to that page. That’s just general SEO, right? And then, you need to determine what you need to put in that search to return a result which will bring the most qualified people through to your site.
Larry Cornett: That’s correct. Making sure that you have done the right things with your site to have the search engine index and rank you appropriately is very important. But, that’s the search engine’s task.
The other key aspect is the user making a decision, like deciding such as deciding what result is going to be most relevant for what they are looking for.
I would say part of that is the keywords that they are using to search. The other part is if they are searching for you and you offer a certain type of information or service, you have to decide what tasks those users are trying to complete. And so, what is their frame of mind when they are looking for something like you, and how do you best show that you have the right information to support those tasks?
Eric Enge: You made a big announcement in mid-October. You already alluded to it earlier in our discussion, but can you talk about it a little bit more now?
Larry Cornett: Basically what we’ve announced is that we have turned on some additional SearchMonkey enhanced result listings for Citysearch and Zagat by default to complement Yahoo! Local and Yelp. Basically, it is providing these rich enhanced listings for local queries.
Eric Enge: The fact that it’s on by default means that every searcher, whether they have subscribed to the SearchMonkey Application for Citysearch or not, gets it by default. You’ve judged it to be of high enough quality to do that, but they can still disable it if they don’t like it and go back to the old type result.
Larry Cornett: That’s correct. Since these applications are on by default, when a user does a local search and that result shows up in the results, they are going to see the SearchMonkey enhanced result automatically. And, if it’s something that they didn’t prefer for some reason, there is a little ‘x’ in the upper right corner of each SearchMonkey enhanced list so you can turn it off and revert back to the simple, text-based result.
Eric Enge: Right. And, after you see the text-based result, you can go back and turn it back on, right?
Larry Cornett: That’s right.
Eric Enge: When you say local search you mean a search on something like Seattle Pizza?
Larry Cornett: That’s right. It’s any query that has local intent.
Eric Enge: What was it that caused you to pick these particular partners?
Larry Cornett: There is a process that we go through where we are looking at the high-quality providers for this type of information. In some cases, we’ve had developers who have developed some of this proactively themselves. In some cases, we have users who were asking us for this type of information. What we are looking at is a site that provides high-quality data for this type of query, and one that provides us with an application that performs well and has very reliable data as part of this process.
Then, we test it, and we expose it to a small percentage of our users and determine if it is performing well based on the search metrics that we look at and track. If all things are good in terms of the quality, reliability and the metrics that we are seeing, then we’ll make a decision to turn that on by default.
Eric Enge: Right. Is there any relationship between how many users the application and the potential for it being turned on by default? For example, if a smaller publisher, smaller than Yelp or Citysearch, has produced an awesome application, they would be interested in participating in the program. Of course, there is less impact for you in overall search quality than something that shows up for a lot more searches, of course. But, you are not really tying it to any sort of metrics on numbers of users at this point?
Larry Cornett: It’s a level playing field where anyone could participate and create one of these applications. And again, it really does come back to the quality and the reliability of the data, and how it performs with the users.
Eric Enge: You do anticipate doing this more in the future? It seems like it’s an ongoing program by which we can expect more and more things to get added.
Larry Cornett: That’s right. It is the method that we have that we are very interested in trying to understand how to present our users with the most valuable and useful results that we can get them. So, we will continue to use these methods in order to test high-quality search applications.
Eric Enge: Anything else you would like to add?
Larry Cornett: There is one other thing I want to highlight that we released a few weeks ago, which is the ability for experimental applications to be in the gallery. Basically, it’s an opportunity for developers to share their applications with folks who are more comfortable with early adoption. And so, it’s something where a user can go into the preferences in the search gallery, and turn on experimental applications, and it gives users a way to actually play around with applications and see if they are interested.
Eric Enge: Interesting. So, what would an example on an experimental application be?
Larry Cornett: It might be something that hasn’t met a certain bar of quality and reliability to be promoted in our gallery, or certainly not to be made the a defaulted. But, it does give users who are early adopters and are okay with testing stuff a way to go in, turn on their preference, and then see some of the really interesting stuff and play around it.
It’s totally their choice if they want to do that.
Eric Enge: Right. Really what it means is that you’ve become aware of applications which haven’t met the quality guidelines to get in the gallery yet, but this may just be because they are too new.
It’s presumably not because they are crashing left and right, in which case it’s not going to show up in the experimental area. That’s really what you mean by experimental?
Larry Cornett: Yes, that’s correct.
Eric Enge: So, getting in the gallery isn’t guaranteed either then; it’s something that you have to earn your way into.
Larry Cornett: That’s correct. We have thousands and thousands of applications that have been created and in there are currently 97 items in the gallery. There is only a subset that has high quality, added-value, and essentially makes that the bar essentially to make it into the gallery. We want to make sure that we are giving our users quality.
Eric Enge: Thanks Larry!
Larry Cornett: Thanks very much!
