Grant Ryan
Grant is Chief Scientist and a co-Founder of Eurekster, Inc. Grant founded the Internet search company Global Brain.net in 1998 (acquired by NBC, now operating as enterprise search company S.L.I. Systems) and social networking company Real Contacts in 2001.
He is a director of the New Zealand Government’s $150 million Venture Investment Fund (VIF) and has been a director of the Foundation for Research Science and Technology which allocates over $450 million per year. He has a degree in Engineering and a Ph.D. in Ecological Economics.
In this interview, Grant and I talked at length about Eurekster’s social search portal product (Swickis) and about the concept of social search in general. Last week I published a very similar interview with Steven Marder. The interview with Steven was the Bachelor’s degree course, and this interview with Grant represents the Master’s degree course on the topic.
Interview Transcript
Eric Enge: Sounds good. So, Grant one of the things that we talked about last year was that you had been at a social networking company prior to working at Eurekster. I believe it was something that helped people find jobs in New Zealand and the Australian area and was called Real Contacts. Can you talk a little bit more about what Real Contacts does, and how that influenced you with regard to Eurekster?
Grant Ryan: The company started out as a Friendster type of dating service. Then, we moved to doing a job search type thing. That company hasn’t had anyone working in it for the last three years but, it just ticks away in the background, and it’s actually profitable. The really interesting thing with that since we last talked is that Facebook’s taken off because of its open social platform.
Real Contacts had similar functionality before any of these social networking companies had the idea of having an open social network platform. It could actually be some valuable intellectual property sitting there; based on a discussion I had with a few people the other day. It’s quite interesting for me anyways. So, we are just trying to workout what we are going to do with that, it just seems that we were about seven years ahead of their time.
As far as social networking and integration of search, as you are probably aware, we (Eurekster) had a deal with Friendster around social networking search. That was one of our first deals. The first crack at the idea of social search was that a lot of people find information through people they know. We actually did an implementation with Friendster, where everyone had different search results based on their friends and friends of friends, and that was quite a sophisticated application.
But, people have got so many friends that look for different things that it’s kind of all over the place. What happens within groups of users, and this gave us the idea and impetus for the Swicki product, which is now the core focus of Eurekster, where if you have got a topic and a group of people that are interested in that topic, that’s where the group of the people watch for what they could be learning from them. That’s where it really adds a lot of value – at the social level.
Eric Enge: Right. So, you are able to basically leverage that group experience.
Grant Ryan: Yes. Part of the reason is that the Swicki approach leverages a critical mass of people doing a similar thing. The search results start to learn from the users and start to look different. At an individual level, we’ve got friends, and friends of friends, because they were all searching for different things depending on what they are looking for; and because your friends can be so diverse. We found that was more of a struggle to tactfully improve the search results that way.
Eric Enge: Right. That type of signal is potentially a noisy signal, right? Because you have a sort of generic issue that the activity that they do on a social networking site isn’t really focused on search.
Grant Ryan: If you think about the group of people that you would be connected to on a social network and the simple things that they’d be searching on the day-to-day, it’s a small chance that it would really be of interest to you. There was actually too much noise to make it really that valuable.
That’s part of what made the buzzcloud interesting because you could have a buzzcloud that was customized somewhat by your friends and friends of friends. We did see that this, in turn, does propagate through those buzzclouds, but there was quite a lot of noise as well. So, it’s hard to know, how that will all play out, but what we did find is that there were certain groups of users, split up by geographies and age and things like that. We found that there was a good critical mass around those, and that’s why we subsequently had the idea for the Swicki product and went and put that together.
Eric Enge: Let’s talk about the relevant content angle of search engines. This is really interesting to me because when someone comes to your site for example and does a search, does it really cause people to come back to use that search engine or they are going to assume that they can find it somewhere else? I imagine that it’s probably a bigger problem for Google than it is for you for example, not having a general horizontal search, but what’s your thought on that?
Grant Ryan: A very good question. We are finding that most people are not coming to Eurekster.com to do a search. Most people are finding the Swickis via blogs and other sites that they are hosted on. People do go back to those blogs and search regularly because they find it useful. People who click on the search results within that, whatever it happens to be, tend to perform more repeat searches across different sessions, going back to a partner site, which is what we want them to do.
It’s kind of like extending the brand relationship they already have with those users into search. We don’t have to build a brand too much around what Eurekster is. We’ve got a tool that empowers other brands to extend into search.
Eric Enge: The other question is when people come to the site, do they realize that they are using a custom search tool that is hand edited by the owner of the site?
Grant Ryan: I can tell you that our traffic is trending upwards, and the repeat number of searches has actually been going up as well. We are assuming that that is the case that people are clicking on it. The biggest problem I am sure you are aware of all of these little vertical search engines is that people don’t know they exist, and they don’t know where to go to find them.
As a result, we have tried to tie into the product this whole idea of the buzz cloud, so that it draws you in to try it. When we try it, hopefully, the penny drops and they are happy with the results and want to use it again. The ultimate goal for us is that people will look for Swickis and they’ll actively use them. People would go to an auto site to use a Swicki they may have deployed there if they are looking for cars.
Eric Enge: Right. What I am trying to get at here is an aspect of the value proposition. Clearly, there is a place for site search on peoples’ sites or enhanced forms of search on sites, and people will use them; that’s very clear. We have plenty of data on that regarding ourselves, and since that’s the case having a model where you can participate in the revenue share together with a compelling type of search experience is by itself a very interesting value proposition. The other question is does it cause repeat traffic, that otherwise might not have happened?
So, if for a site the return visitors rate is 60% (i.e. 40% are first-time visitors, and the other have been at the site before) will have a Swicki nudge it up to 62%, or 65%, because of the value of the search tool? So, it’s probably a tough question to answer, but does that make sense?
Grant Ryan: This is a very key point, which is whether people understand vertical search engines and their value? Personally, I think most people don’t realize how often they do use vertical search engines. They actually are using more and more of them, if you think about Amazon, it’s a vertical book search engine. Lots of people will use a travel search engine to find travel-related things, and you wouldn’t even think twice about using Google to do a job search; you’d go straight to a job search engine.
People have built businesses around having an engineering search engine, or a health one. I think there is room for more and more of these vertical search engines, and it will basically happen over time. It’s a key trend of all forms of media. We have kind of big generic ones to start off with. Think about the number of TV stations around now or radio stations that we used to have, versus how many we have now. I think the same thing will happen with search, and so you’ll still have the big general search engines, but, there will be more and more of the vertical search engines.
It will take a while before people really understand the true value of them. But, Eurekster is getting close to a million searches a day now. And, this is something that wasn’t around a couple of years ago, and so it’s turning out nicely. Hopefully, we’ll just continue the trend of growth, and people will understand it more and more over time.
Eric Enge: Right. It is a question of time, so I think you are right about that. Let’s talk a little bit about the buzzcloud; start with what it is just so we give people a quick reference point. Then, for follow-up questions does it influence the usage of the Swicki, and how does it get set up?
Grant Ryan: The buzzcloud is a group of popular searches that the community has been using on the search engine. We think of it as an “online water cooler.” It shows what other people are interested in and, it stimulates interest in the people using a search engine. The Internet is a great big place out there, and not only do we want to get informed whatever we are looking for, but it’s also asking to sometimes find what else might be interesting out there that I don’t even know to look for.
What we do is look at the series of queries that people have been typing into the search engine, and then the moderator of the search engine will choose which ones of those go into the buzzcloud. One of the things we’ve been working on recently are ways that we can automate that.
Eric Enge: Right. How quickly does the buzzcloud respond if some search term suddenly has a burst of search traffic on a given day? Will that be enough for it to emerge and show as a larger bolder item in the cloud?
Grant Ryan: Absolutely, yes it will. One of the reasons we have the buzz cloud moderated is to prevent people from spamming it. Now, we’ve got a bunch of smart algorithms now that ensure that the terms people are looking for are actually associated with the topic. Rather than waiting for the moderator to approve them, which is quite a manual process, we now have a way that will become more automatic. So, what you will see is that the buzzcloud has become a lot fresher and more dynamic without the involvement of the moderator, which is pretty exciting.
Eric Enge: Yeah, that is exciting. After all, the moderator might be away for a weekend, and then the buzz cloud isn’t aligned with what’s going on in their space. But, if you do it automatically, of course it’ll resolve that problem.
Grant Ryan: Yes. We are quite excited about it, and we are trying to make the buzzclouds more genuinely up-to-date, and so that there is a lot less effort required by the Swicki owner.
Eric Enge: That makes a lot of sense. Last year you said that in some cases the search is a discovery experience. That is one thing that is attractive about the buzzcloud. When you look at the cloud, you might just be in research mode rather than mission mode. Can you talk about that a bit?
Grant Ryan: Yes. That’s one of the whole things about some of the social media sites like Digg, for example. Digg is successful because people want to see what everyone else is finding interesting and compelling. With these sites you find good things that you never knew existed. We believe the buzz cloud is the key differentiator with what we do, and that’s what drives more people to use a Swicki. We know that if you just put up a search box, people will use it, but with the buzz cloud you can get up to 800% more searches.
The buzz cloud gives people a reason to use a search engine. For example, when you decide to test the search engine, you might not immediately know what search term to use and to test it. Sometimes you draw a blank on that. So, having the buzz cloud draws things in. Then many people then do some additional searches.
Eric Enge: Right. I think one interesting example that fits this notion in a very powerful way is celebrity search. I have been told that celebrity search are maybe as much as 5% of all searches. If the buzz cloud shows that Britney Spears is smaller than usual, and some other person is much larger than usual, you are going to want to find out why.
Eric Enge: When you are doing celebrity searches, most people are looking for the hottest gossip and the latest news. I am sure there are logical equivalents in the medical area; you could probably have a situation where you may have a search engine about diabetes. And, all of a sudden there is this new word you haven’t seen before showing up, and it turns out that it’s a new medicine or a new food that’s been shown to be beneficial to people with diabetes. Those are the kind of scenarios I think that probably are very discovery oriented.
Grant Ryan: Yes. This leads nicely into one of the really interesting new features that we’ve got as part of the Swicki. We know the Web sites that are of particular relevance to that Swicki. What we are doing as part of the new release is looking at Web sites and then taking their RSS feeds, and then skimming them, and then pulling those into the buzz cloud. So, you don’t even have to wait for someone to search for it to get the relevant information fed to the buzz cloud.
If the Swicki is about diabetes, as soon as there is news on diabetes, it will move into the buzzcloud almost instantaneously.
This would make the buzzcloud the best place to find the latest information about whatever is going on. It really is about trying to let people know the very latest of what’s going on, because you can be on a Web site, and they publish a piece of content on another part of the website that you are not on, and you might not even know about it. With the Swicki you can see it as soon as something else is added to the buzz cloud. It’s a way of saying here is the latest and greatest fresh content around this topic.
Eric Enge: Right. So, when I spoke to Steven Marder, one of the things that he mentioned is that you do like your own crawls for freshness of blogs. Is that what you are talking about leveraging here?
Grant Ryan: Yes. The whole thing about Swickis is we want them to be fresher and more focused, and faster than a general search engine, so that you know that it’s all about this topic, and it’s totally fresh, and it’s fast. We are monitoring the RSS feeds for relevance in Swickis, and taking that information as soon as it is new, so it’ll come up in the search results. We are taking snippets from the title of the RSS, and putting them in the buzz cloud so that it really does show up the latest and greatest around any particular topic.
It really enhances that discovery process that you were talking about before, where people want to know the latest, and they want to know it now. And, they want to be the first to tell their friends.
Eric Enge: Great. Let’s move on to the next topic area here, which is some of the social features. This is one of the things that really distinguishes Eurekster is the social functionality, the fact that you can truly start to think about social search. For example, you do some implicit stuff where if somebody is searching for things in a Swicki, let’s use a celebrity Swicki for simplicities sake, and ten people search on Brad Pitt, and most of them click on the third link. That site might get moved up in ranking over time, because that seems to be the most popular thing for Brad Pitt. Is that a fair assessment of that part of the algorithm?
Grant Ryan: Yes, the actual algorithms that we use are quite complicated and we’ve evolved them over the last seven or eight years. But, that was the basic idea behind the first company I had Global Brain. The whole idea there is that every time someone interacts with the search result, they are giving you information about what they are and aren’t finding relevant.
We obviously use that information in a number of ways. We use that to move the results around for a specific search result. We use it to learn what sites have good content for that particular Swicki. As we were talking about before, we can learn which things to add to the buzzcloud.
We take that click stream information as the continuous feedback from users about what they are finding relevant and not relevant. Over time we build up this knowledge base around the topic, around what’s useful from this group’s point of view.
Eric Enge: Right. So, one consideration I would imagine is a listing in the third position is less likely to get clicked on that listing in the first position, simply because of the position it’s in. It seems to me that you’d have to do some adjustment based on placement. It’s just one example, I am sure there are many other things like this.
Grant Ryan: Yes, that’s completely correct. The other thing is that if you didn’t do anything about it the results could get incredibly stale because the ones at the top are at the top and may get more clicks. And so, they stay at the top, and there are a bunch of things we do to ensure that our fresh content has got a chance to be clicked on too, so you’re exactly correct.
Eric Enge: When you make these kinds of adjustments, do you do it on a Swicki-by-Swicki basis? For example, my Swicki gets a particular adjustment because of my audience, or do you aggregate the data at some higher level?
Grant Ryan: No, we do it at a Swicki-by-Swicki basis, and our assumption is that you create a Swicki, that your particular audience has a particular view of the world. What we want to do is record that view of the world and show it to them, and that can be quite different from other Swickis on the same topic.
We have lots of ideas around how we could aggregate that up to different category levels and different things like that, but at the moment we actually think it’s most relevant to do it on the Swicki-by-Swicki basis, and let them evolve as the communities change.
Eric Enge: Right. It would be hard to aggregate, not that you can’t do it, but because you have a different set of sites on each Swicki.
Grant Ryan: Yes, exactly. We want to reward our publishers for the fact that they have gone and created this tool, and they can have this knowledge base of what’s useful for their users. We want them to own it rather than us. It’s a key part of why they would put the time and effort into building a Swicki.
Eric Enge: Right. The other part of your social experience is that users who want to can get more active and they can make contributions. What kind of things can they do?
Grant Ryan: So, there are three levels. One we talked about already, which is the passive learning based on user searches. At the next level, on the search results page are these little buttons where you can actively move things up or down. Finally, at a more detailed level, you can go in and write your own search result. You can just use a back-end blog platform so that you just write as you’d write a blog post, and you can add it, and, that will be added directly into the search results, and then it can live and die on its own merits, people can vote on it.
Another feature that’s coming up shortly is the ability to comment on any particular search result. As with other social media sites and tools, you have the 90-9-1 rule where 90% of people will just use it, maybe 9% of the people will tell other people about it, and only about 1% of people will actually actively contribute.
Eric Enge: Right. Where does a user who is using a Swicki find these opportunities to become active?
Grant Ryan: That is a good question. We have certain courses of action in terms of voting things up or down, and or writing search results, and we have been experimenting with different ways that you can let people know without cluttering up the page. There will be another iteration of that coming out shortly, and we are trying to make it simpler and easier for people.
Eric Enge: Right. Can you talk about reputation management for contributors?
Grant Ryan: We do have a way of measuring reputation at the moment, but we just use it on the back-end of our system to try and track people that are doing things they shouldn’t be doing. We have plans to potentially expose that reputation to users, but that’s not on the immediate horizon.
Eric Enge: The privacy issues with individual search history data, and the implicit social functionality; how do you deal with that?
Grant Ryan: That’s a great question. It’s something we have thought a lot about because there is such a privacy concern. I don’t think most people realize how much data search engines collect about them, and when they do I think they will object a little. What we do is we collect the clicks for each Swicki, but we aggregate each of those groups separately. So, we collect nothing at an individual level, so we don’t know that you’ve contributed to three different Swickis. Basically, while we collect information from users, it’s on a Swicki-by-Swicki basis, not on an individual basis. We don’t have any individual information about what a particular user does.
Eric Enge: Right. Unlike the other search engines, you don’t have to worry about cookie deletion and things like that?
Grant Ryan: That’s correct.
Eric Enge: As I understand it, the way you build your index is you use the indices of Yahoo! and Ask for Web search and blinkx for video search. You also layer some crawling of blogs on top of that for freshness purposes. Do you plan to expand the number of ways that you do your own additional crawling?
Grant Ryan: Our ideal scenario is that we work with partners that have relevant content so we don’t go and reinvent the wheel, and that’s why we are really pleased to be partnering with blinkx, and Yahoo!, and Ask, and they provide fantastic feeds. We just fill out the parts that don’t quite fit with their business model. We actually want to have the latest, for example, RSS around a specific vertical, so it makes sense for us to go and build that ourselves. As required, we will be continuing to do those sorts of things, but we’ve got no plans to go and try and recreate these indices ourselves.
Eric Enge: Yes. But you did feel the need to do some additional crawling in the case of blogs for freshness purposes?
Grant Ryan: Yes. One of the things about the Swicki is that the amount of content that’s really relevant to a given topic is finite. It’s not some big infinite void, so we wanted to make sure that, given what we know that a Swicki is about a particular topic when there is new content around that new content is now interesting. We want to make sure it’s instantly added, so that really does become the precious and most focused source of that topic.
It doesn’t make sense for Yahoo! or Ask to go and do instantaneous searches. It doesn’t fit with their model, so that’s why we’ve built specific technologies around doing that, so that it really builds up the quality of a Swicki.
Eric Enge: Right. What about universal search? I understand that this is something that you’ve been doing, and by universal search I mean image results, and video results, and Web results, and these sorts of things all intermingled in the results a searcher would get. How do you handle that, and what was the most difficult part of doing it well?
Grant Ryan: To be honest, we don’t do it as well as we could; and it’s one of the things we are looking to improve. What we do is we ask people if they want just the Web results or do they want to have pictures and videos. We let the user decide if they want to include it. The current format has the images at the top. The videos can be inline, so they can move around. The next phase of what we want to do is make it more dynamic so that we are learning that if no one is clicking on the pictures, do not show them the pictures, and add that to what we learn from the Swicki behavior. Does that make sense?
Eric Enge: Yes. It does give you some unique opportunities for handling the ranking problem of comparing a video to a Web page. You can let the users sort it out. The Swicki starts with a good initial first guess and then the users sort it out.
Grant Ryan: Absolutely, that’s the philosophy.
Eric Enge: Speaking of video, how long ago did you start doing video search?
Grant Ryan: We announced it in early October. It’s a natural extension. I think the video addition to the buzzcloud looks very interesting to people, but, it does take a while to load. Over time as people are on faster and faster connections, it will get better and better, and I am sure you are on a super fast connection so it probably looks great for you.
Eric Enge: Right. It does.
Grant Ryan: That’s probably one of the reasons that it hasn’t taken off quite as fast as we had hoped, but over time as they speed up it will get much more interesting. It’s a way of trying to do the same thing that we are doing with Web results for video results. There is a whole lot of stuff around a particular topic, and you can see it individually and can click on it and learn.
Also, the number of video results compared to Web results is still a tiny, tiny fraction of what you get for Web results. As the video content grows over time, the value of verticalization in different search engines will become more and more apparent. The true value of it will grow over time, it is my guess.
Eric Enge: Right. What intrigued me about the video buzz cloud is if you talk about Web search, and we talked about it sometimes it’s a discovery experience and sometimes it’s a really a targeted research experience. Many times, of course, people are engaged in a targeted research experience. You just need this website address, or whatever. Whereas in video search, I would imagine that it’s a much higher percentage of the time that the search experience is a discovery experience. A much higher percentage of the time where it’s entertainment-oriented essentially. When I look at the video buzz cloud, it’s extremely attractive and interesting, and you just want to start clicking on things, whether you were planning to watch a video or not.
Grant Ryan: Yes, exactly. It is fascinating how that leads you to look at other more interesting things. That’s why videos are quite compelling because all of the video sites have that discovery element built into them. When you finish your video, YouTube will suggest some other ones look at, and what we wanted to do here was “verticalize” that.
If you are interested in that particular topic, you’ve got a bunch there, and, when you can visually scan a whole bunch of videos you are truly improving the discovery process. So that was exactly the reason for doing it.
Eric Enge: If we take a more generic cut at the problem of doing a video search engine, what are the big challenges with it?
Grant Ryan: The biggest challenge that we have found with this is actually the lack of content. You can do a Swicki about big wave surfing and you can mention all sorts of places, and some of the places that you can expect the videos on a big wave of surfing. There are only a couple of videos, whereas if you did a Web search that would be thousands and thousands of Web pages. So, when you are verticalizing the video results, depending on the level you are verticalizing, there may not be that many relevant videos available.
Over time that’s just going to grow more and more as more people upload videos. That was the one thing that surprised us a little bit. One of the reasons we chose Blinkx is that they had such a good range of videos from so many providers, but even given that, sometimes there aren’t quite as many videos about certain things as you think there might be.
Eric Enge: Right. It’s a little harder to make a video after all than a Web page. Another area that’s really interesting is your advertising model, and it appears to have evolved over the past year. Can you just talk about how the model works, and what some of the unique aspects of it are?
Grant Ryan: When we were talking last year we were about to launch a feature that allowed anyone to advertise on a particular Swicki, around a topic. It was kind of like a paid inclusion way of getting a whole lot of content into a Swicki around the topic. We experimented with that and while it actually works, the main problem was around the critical mass around each Swicki.
In order to make it worth an advertiser’s time to find out about it and advertise on a single Swicki, you need to do a certain quantity of searches to really make it worthwhile. We found that we were a little quick out of the gate with that program. We believe that will become more relevant over time as the volumes that they have in each vertical grows.
For example, if you’ve got a Swicki that’s doing a thousand searches a day, and you are a retailer and you want to reach that audience, and you’ve got products that match one out of ten of their searches, that just won’t result in that many sales.
Eric Enge: We talked quite a bit about social media and search results at the beginning. Are there other things that you see that might happen just at a theoretical level in the future with social media and search engine results?
Grant Ryan: Yes. We are looking at some of the different ways that we can tap into resources like del.icio.us for example. If we know that someone is creating a Swicki around big wave of surfing, we can go to del.icio.us and look up tags and write-ups, and potentially include content from a social site like that as a way of creating a Swicki with good content. There are some interesting ways that we can contact them for some of these completely different services to build on the statistical information that’s out there and we can use this information to improve upon.
I think it is quite remarkable how quickly this whole idea of social media has grown. If you allow people to create anything and add that layer of social media on top of it, it really does become quite a powerful medium. That’s really the core philosophy behind what drives the Swickis, and that’s why we are so excited about the idea. We’ve been looking in the social space for a while, where we’ve learned a lot of things that we think are pretty relevant and pretty interesting.
Eric Enge: Thanks very much Grant.
Grant Ryan: No worries, it was a pleasure. Thanks Eric.
