UPDATE 31 October 2017: Added video of Eric discussing this topic with Google’s Gary Illyes
Click through rate (CTR) on your listing in the SERPs, isn’t used by Google as a direct ranking factor. But, on the other hand, it might cause your rankings to rise for indirect reasons. That good for a confusing start? Excellent!
In this post, I plan to walk you through why it’s not used by Google as a direct ranking signal. However, Google does use it as a way to test the quality of its search results. Understanding this distinction is important if you are serious about digital marketing, and that’s what I’ll explain in today’s post. In addition, I’ll show you how understanding this can help you get better results from your SEO efforts.
But first, let’s give you a direct answer to your question:
Does Google Use CTR as a Direct Ranking Factor
No, it doesn’t, except in limited niche situations. CTR is used primarily in quality control testing of new proposed algorithms, and possibly to audit search quality from time to time. This limited approach is intended to be only one part of how Google evaluates user engagement and satisfaction with the search results, and this data is then used to make decisions about further tuning their core algorithms.
Below, you will see some data and analysis that explains the above statement, as well as examples of tests conducted by Rand Fishkin that provide some examples of tests where CTR has had a temporary impact on rankings. As modified to the prior paragraph, there does appear to be evidence that CTR may be used within Google’s freshness algorithm to discover trending topics.
I believe that this may be what has happened in Rand’s tests, but that this usage is limited to this specific niche situation.
Why CTR Isn’t a Direct Ranking Factor
I’ll give you this one in a sentence: “CTR in the search results isn’t a direct ranking factor because it’s too easy a signal to game, and in many other cases too hard a signal to use effectively.” The following graphic can help illustrate:
If it became clear that CTR was a direct signal, people would implement bots, or pay large numbers of people in third world countries to click on their pages in the search results to drive their rankings up. This would already be happening at a massive scale.
You may have also heard of a more sophisticated concept called pogosticking as well.
Basically, a user clicks on one result in a SERP, spends a short period of time there, and then bounces back to the SERP, and clicks on a different result, and doesn’t return. In theory, this sounds bad for the first page clicked on, as it implies that the user wasn’t satisfied with the first page they clicked on, but did like the second one.
The problem with this logic is that there are some types of user scenarios where that conclusion doesn’t work. For example, if a user is in comparison shopping mode, they will want to see more than one result.
Or, if they are looking for multiple pieces of information off of the one query, got part of what they wanted with the first (they were satisfied), and then went to the second one, didn’t get what they wanted, but got interrupted before continuing to do more research, so they never go back to that SERP.
Another major issue is trying to define what makes a short visit vs. a long visit. There are so many gray areas in how you could try to approach something like this.
In addition, looking at this from a more theoretical perspective, CTR behavior in the SERPs is something you can only measure on content which is ranking in the SERPs. That’s a pretty limited sample set. Contrast that with a signal like links, which is based on crawling, and will allow Google to consider every page they crawl on the web (not just those in the SERPs).
How Google Does Use CTR and Other Quality Measurements
The heart of this is derived from Paul Haahr’s excellent presentation at SMX West, which was titled How Google Works, a Ranking Engineer’s Perspective. Some of the key points of this line of thinking were also confirmed in the conversation that Google’s Andrey Lipattsev had with Rand Fishkin, Ammon Johns, and myself on Anton Shulke’s Google Q&A Show on March 23, 2016.
To summarize, Google uses controlled click-through rate testing to validate the quality of their search results. Please note the emphasis on the word “controlled”. In this case, what that means is that they will selectively test specific portions of their results, or new algorithms and measure the results they get.
Think of it as an audit process, and one that is run under controlled circumstances where they can be sure that the results aren’t being named by someone in the process.
Equally important is how the data is used. In short, it’s used to make tweaks to direct ranking factors (such as links, content, location, measures of relevance, …) that tend to improve user engagement with the SERPs.
This type of signal is just one of many tests Google uses. We also know that Google uses manual testing based on large numbers of people who evaluate search quality as well. Google has a handbook they use to train the raters that you can see here.
These inputs, and likely many other signals are used to implement a process that may look something like this:
Ultimately, this feeds into a larger scale process of testing and refining search quality on a regular basis:
For those of you who are thinking about how to manipulate this, good luck. Google is only doing these things on a very limited basis as part of a quality auditing process. This is not really amenable to large scale CTR fraud.
What About Rand Fishkin’s CTR and Pogosticking Tests?
There have been a number of times at conferences where Rand Fishkin has run tests with the audience where he has asked them to click on a search result en masse to see if that might alter the ranking for that result. This was discussed in the recent Google Q&A Show I referenced earlier. What follows is a cleaned up excerpt from that conversation:
Rand: Seven or eight times in the last two years. I’ve done something where I’ll be standing on a stage, in front of 500 to a couple thousand people, and I’ll ask them, “Hey. Can you all jump on your mobile phones or on your laptops and do a search? I want you to click the seventh result. Then over the next 24 hours, let’s observe what happens to that page’s ranking for that query.”
I’ve had seven or eight of those that have been successful, and I’ve had four or five where the ranking did not change. I’ve run a few of them over Twitter, with a few where the ranking did change pretty quickly and usually sticks around a day or two, and a few where nothing happened. What’s happening there that’s making the ranking for the page that gets clicked on change so rapidly? Then what’s happening when it falls back down again, relatively rapidly, over the next two to three days?
Andrey: It’s hard to judge immediately, without actually looking at the data in front of me. But, in my opinion, my best guess here would be is the general interest that you generate around that subject, and you generate exactly the sort of signals that we are looking out for, mentions and links and social tweets and social mentions, which are basically more links to the page, more mentions of this context. I suppose it throws us off for a while until we’re able to establish that none of that is relevant to the user intent, I suppose.
Eric: It sounds to me that in the case of some of the tests that Rand has done, that there might be something going on that’s temporal in nature, where you’re seeing a burst of activity around something, and you’re responding to it like a trending item.
Andrey: The thing is without looking the way that data is changing inside the system, we can’t really tell.
Eric’s Interpretation: I’ll stand by my guess that I made in the above conversation that some part of the Google algos that are designed to pick up on hot news events is triggering the behavior seen in Rand’s experiments. This would explain the rise in the results and the drop afterward when the click activity tapered off. But, we can’t know 100% for sure.
How Would Google Measure Your Chances of Higher User Engagement?
Before answering that question directly, let’s recap:
- Using CTR as a ranking factor would be highly susceptible to being gamed by spammers. If it were that simple, you would see ads from companies promoting easy rankings gains for you, and you’d be getting daily emails about it, but, you’re not.
- Google has confirmed that they use CTR and other user engagement metrics as part of the process of evaluating search quality, but indicates that they do this only in controlled testing scenarios.
- Rand Fishkin’s tests have shown some aspect of CTR that does appear to work at times. Given the nature of the results, it appears to be from a Google also that is highly responsive, and is therefore likely to be from an algo designed to detect trending activity.
Considering point 1, I’d advise that you don’t invest any energy in trying to use click behavior to manipulate search rankings. Google is obviously highly aware of the potential for this type of activity, and has long ago largely addressed it.
That said, I do think that Rand’s testing has uncovered something important, and that has something to with Google’s trending activity algorithms. However, if you simply sustained the higher click volume, it would no longer be trending, it would be steady state and the benefits would go away rapidly.
Looking at point 2 above, the logical outcome of valuing CTR and user engagement as a search quality check is that Google would look for (less game-able) signals on the web that would be indicators that users would engage more with one particular site than another. What would those be? I thought you’d never ask!
Next, are some sample signals Google could possibly use. Before you read these, I need to emphasize that I am NOT saying they DO use these. I’m just offering them as Now, let’s summarize 3 examples of signals that Google could use:
Example 1: Author Authority
Yes, the Google Authorship program was ended, but that doesn’t mean that Google ended their interest in using author authority as a signal. In our industry, for example, if Danny Sullivan writes an article, you’re very likely to give it more weight than an article from someone you didn’t know.
This will likely increase your chances of clicking on one of his articles in relevant search results, and also more likely to stay on the article page longer and spend more time reading it. Chances are that you are less likely to go back to the search results quickly, and you’re less likely to need to read a related article by someone else.
You could measure authority in several ways, including the average number of links to an author’s post, social activity shortly after it goes live, links to their author/bio pages, search volume on their name, and how often compound searches with their name along with a subject matter title (e.g. “Danny Sullivan Panda”) take place.
I’d bet that this is something that Google actually does use in some fashion, though it may be specific to personalized results. You could imagine that they tested this signal, saw that it generated improved overall user engagement in their SERPs, with no unacceptable downsides, and then rolled it out. I also consider it likely that they have tested hundreds of versions of this in a systematic fashion to tune to the best possible way to leverage a signal like this.
Example 2: Brand Authority
This is something that Google can also measure, and it’s clear that people will spend more time engaging with a brand they know and trust. This is perfectly analogous to the situation with author authority. One metric you could look at would be the brand search volume. This would be a weak signal though, as it would be easily gamed by simply doing large numbers of searches on your brand.
However, the counter to this attempt to game the system is an easy one. Does the brand’s home page get tons of links, including many from authoritative, highly trusted, sites. How many mentions of the brands are there on the web? How often does the brand name show up in the anchor text of links? How does the brands metrics compare to that of competitors in the same market space?
Note that I deliberately chose a somewhat deceiving example to illustrate that this type of analysis isn’t simple. Far more people search on “Coca Cola” then they do on “The Coca Cola Company”, and sorting out these types of situations would be one of the many issues that Google would have to address to make this particular signal work.
As with author authority, it would not surprise me at all if Google uses some type of signal like this. The concept makes complete sense. The only question is whether or not they’ve found a good combination of signals similar to the ones that I suggested above that makes it a clean signal.
Example 3: Leverage Search History to Evaluate Content Quality
Of course, Google tried to evaluate this at many levels, but let me focus on how this can be used to measure user engagement. What if you could measure the likelihood that a user will find what they want on your page after entering a particular search query? For reference, I first started writing about this publicly in August 2015.
How could Google measure something like this? Simple, Google has data on all the searches users perform. They can see when you enter a query, such as “oil filters”, for example. They can then see the follow on query. Perhaps this is for “oil”, “oil filter wrench”, ” or “windshield wipers”. Google could then decide that one company’s oil filter page is better than another if they address more of these types of follow-on needs.
what does a user search on next? and then show some choices
Google could even manually select specific signals for their algo based on controlled tests across eCommerce sites, and observing that users respond better to sites with shopping carts (helps lower the chances of an affiliate site ranking) or privacy policies that are readily visible on the page.
What’s the Win In it For Me?
These are all just ideas based on free brainstorming on my part, but you can see how they might find various types of signals that are leading indicators of sites that might offer better engagement, without actually trying to use something like CTR directly. As you can imagine, Google has tons of brilliant people that brainstorm these ideas for a living. Then they run these ideas through a systematic testing process to find out what works and what doesn’t. These tests are used to evolve the ideas into something that works on a massive scale (the whole web). It also involves many levels of internal review to make sure it’s truly a good set of signals.
Google also systematically works to make sure that each new signal is subject to confirming signals. That’s why they would never use brand search volume as a signal by itself. As a result, many of these signal ideas are probably combined with other checks involving tried and true signals like links.
For that reason, you need to care deeply about user engagement on your site. Here is a checklist of eight things to think about as you do that:
- Are you working on building your brand?
- Do you have a recognizable expert working with your business? This has a similar effect to having a recognizable brand.
- Review analytics data on major pages to see bounce rate, exit rate, and other engagement metrics. Google may not use them, but that doesn’t mean you can’t.
- Check your site search to see what things people aren’t finding. People are leaving clues about what they want for you there.
- Review the sites of your competition to see what appears to be working for them.
- Perform advanced content comparison analysis to see what deficiencies you may find in your content.
- Make your site a sweet experience for Smartphone users.
- Do everything you can to make your site as fast as possible.
As you do each of these things, consider what their impact is on what percentage of users are likely to be satisfied with what they see on your site. Rand Fishkin has a great phrase for this, “task completion”. Do you help the visitors to your pages complete the task they had in mind when they got there? You can see more about how Rand get to this conclusion here
BONUS: Google’s Gary Illyes Discusses CTR & Ranking with Eric Enge
So here’s where I am with all this:
- Google cares a lot about overall user engagement with the results they show in the SERPs.
- Google does not use CTR as a direct core ranking signal.
- As uncovered by Rand, there does seem to be something related to freshness that can be driven by CTR.
- Google does use CTR in testing of search quality, and in testing of proposed new algos.
- This will lead Google to try and define easily measurable factors that correlate strongly with overall user engagement
- As a publisher, the difference to you shouldn’t really be that significant.
- Go back and read point 1 in this list. If user engagement matters a ton to Google, then it should matter to you as well.