In October 2015 Google stunned the search world with a rare announcement of a major addition to their search algorithms, called RankBrain. Since that announcement there has been a lot of speculation (some of it, frankly, pretty wild) about what RankBrain does and doesn’t do.
In this episode of Here’s Why, Mark Traphagen asks Eric Enge to share what he’s been able to learn about the update and some of the things we’ve discovered about its possible effect on search results.
Don’t miss a single episode of Here’s Why with Mark & Eric. Click the subscribe button below to be notified via email each time a new video is published.
Resources Mentioned:
- Our study of possible before and after effects of RankBrain
- RankBrain Myth Busting
- Google’s Gary Illyes on Emerging Search Trends (Virtual Keynote)
Transcript:
Mark: So, Eric, what is Rank Brain and what does it do?
Eric: Rank Brain is an algorithm that Google announced back in October 2015. They actually announced that through an article on the Bloomberg site, and it uses machine learning to better understand language. And as I understand it, what it does is it is able to recognize more natural language ways of expressing things than the way we traditionally do search queries. Based on that, since it understands the query better, it’s more likely to present the right search results to the user. What’s interesting about this, that’s true even if the user kind of malforms the query and says something that doesn’t completely make sense.
Mark: Okay. You mentioned machine learning is involved here. What is that and how does that help Rank Brain?
Eric: Great question. First of all, the first thing I want to tell you is that machine learning and artificial intelligence are very different things. We are a long way from artificial intelligence. Where we are right now is machine learning is basically a process by which a computer program can learn how to build an algorithm that can fit a set of data. So let me just explain that a little bit.
Imagine you had a data set which showed the size of a house and the price of the house when it’s sold. So historical sales data. To be fairly simple, you draw kind of a straight line and you kind of figure out the impact of the size of the house on the sales price.
But then when you dig into it a little bit further, because we need to, because that first thing gave you a human could do. But there is other things that affect the sale of the house. The size of the lawn, the number of bedrooms, the number of bathrooms, what town or city it’s in, and then it starts to get very complicated. It’s hard for humans to do that. Machine learning algorithms are able to take a problem like that and relatively easily build an algorithm that fits all the data points and makes sense.a
Mark: Okay, cool. Now you recently did a study looking at Rank Brain and you wanted to find out has it affected the search results. It’s been there, you said, since mid 2015. Tell us how you did that and what you found out?
Eric: Well, at Stone Temple Consulting we maintain a database of results for 1.4 million queries within Google. And we have a snapshot from just before when Rank Brain started rolling out, which is actually in July of 2015, according to Google. So we looked at the results from July and then we looked at that to find queries where it was clear that Google didn’t understand the query well in July. You could see it because the search results matched up very badly with the actual user query. And I’ll give you an example of one in a moment.
We then looked those queries to find ones that Google clearly didn’t understand. And as part of that, we dug a little deeper to make sure…okay, Google didn’t understand it but maybe it’s because there is nothing to understand because sometimes users enter in very strange things. So we also looked to make sure there actually was a good search result for the query that Google could find.
Once we found the set where they didn’t understand the query and there were good search results available, we put those into our study. So I’m just going to give you an example. One query is, “Why are PDFs so weak?” And in July of 2015, the number one result from Google was a page that was a PDF file explaining why the Iraqi resistance to the coalition invasion was so weak.
Clearly, that has nothing to do with the query. So this is an example of one that Google didn’t understand. If you do that search today, the number one result is actually something that talks about the weakness of encryption and security in PDF files, directly related to what the user actually meant.
So, to get some insight on what’s going on Rank Brain is this notion of the word “weak” was not properly interpreted. And I think it also has something to do with the part of the phrase, “why is” and Google wasn’t fully understanding that.
Now RankBrain makes those connections and sees the whole of the user query and interprets the whole rather than what Google would have traditionally done with that query. And just for fun, I’m going to give you another example, which was the one that Gary Illyes shared on the Virtual Keynote that we did together with him. And that one is, “Can you get 100% score on Super Mario without walkthrough?”
Mark: Yes, I can…oh, I’m sorry. You weren’t asking me.
Eric: No, I wasn’t asking you. I’m not actually interested as to whether or not you can do it. Sorry. I pick on Mark all the time, can you tell? But in any case, Google traditionally ignores the word or did ignore the word “without.” So now let’s play back the query with that word missing. “Can you get 100% score on Super Mario walkthrough?” That’s a very different query. And so Google returns pages that would give you a walkthrough to help you do it rather than answering the question, “Can you do it without a walkthrough?”
Mark: And now they include that word “without.” So they understand that changes the whole meaning of the sentence.
Eric: Yeah. So they handle that query much better now, and we found a number of examples like that in our results.
Mark: Okay. I’m going to throw in a teaser at this point. I was going to ask you, Eric, what did we find out? How many of the queries that we looked at were improved? But you know what? I’m going to ask our folks to go check out your entire study because this is really worth reading. And you really got to find out how much did change. What did we find out? You can find out in that study.
Eric: And in addition, you will see a number of examples of different kinds of language that we believe that Google improved on. I gave this notion of “why is” and there are some others like “where is” and things like that. And there is more of that laid out in the detailed study.
Don’t miss a single episode of Here’s Why with Mark & Eric. Click the subscribe button below to be notified via email each time a new video is published.
See all of our Here’s Why Videos | Subscribe to our YouTube Channel
