UPDATE: In May 2016 Google rolled out a second mobile-friendly ranking update. See “What was the impact of Google’s second mobile update?” for our updated results.
In this study, we look at the impact of the so-called “Mobilegeddon” update. This update was viewed as being a potentially very large update. At SMX Munich, Google’s Zineb Ait Bahajji was quoted as saying that its impact would be larger than Panda or Penguin. In addition, at SMX Advanced, yesterday, June 2nd, Google’s Gary Illyes reiterated that the Mobile Update indeed had a bigger impact than Penguin and Panda combined.
As featured in The Wall Street Journal
However, media reports since the update have suggested that it was a lot tamer than expected Some have even called it a non-event. So what’s the real story?
Luckily, at Perficient Digital we set up a test to measure the update’s impact. We pulled ranking data on the top 10 results for 15,235 search queries the week of April 17th (before the algo rollout), and again the week of May 18th. We pulled ranking information, and also identified whether or not the URLs in the results were designated as Mobile Friendly by Google or not.
Note that in the meantime, Google also did their Quality Update, so this will have some bearing on the data as well. However, the numbers below still tell a story that suggests that the impact of the April 21 mobile algo update by Google was bigger than what most people currently believe.
What Happened to the Top 10 URLs as of April 17th?
This is a unique view of the data that we took that tells the story. Basically, we took the URLs that ranked in the top 10 for our test queries, and we saw where they ended up in our May 18th view of the data. In other words, if a URL was in position 4 on April 17th, but showed up in position 54 on May 18th, we tracked that in detail.
Here is what we found:
The original set of Non-Mobile Friendly URLs got nailed. Nearly half of them dropped in rankings, and 2.3 times as many dropped as went up. In contrast, the Mobile Friendly pages fared much better, with an overall increase in rankings.
You might ask: Wait, why did the average rankings for the Mobile Friendly pages not go up more? Great question. There are two reasons why this actually makes a great deal of sense:
- URLs that were in the #1 spot (15,235 of them) had no opportunity to gain in rankings. They could only go down, and 70% of these were Mobile Friendly.
- The quality update from Google caused some shifts in rankings too, and some of the demoted domains from that update were Mobile Friendly.
In conclusion, this view of the data shows that Google did provide a material preference to the original set of Mobile Friendly URLs.
Percent of Mobile Friendly URLs in the Top 10 Results
In addition to seeing what happened to our original top 10 URLs, we also pulled data on the makeup of the current top 10 as of May 18th. The idea was to see if the percentage of the results that are Mobile Friendly has increased. Here is what we found:
- On April 17th, we had 56,164 Non-Mobile Friendly URLs (36.9%), and as of our May 18th measurement, that dropped to 54,162 URLs (35.6%).
- On April 17th, we had 96,186 Mobile Friendly URLs (63.1%), and by May 18th that had increased to 98,188 URLs (64.6%).
This method of testing the results of the April 21 mobile algo release appears to provide a conflicting view to our other slice at the data. The net shift here is only a 1.3% increase of Mobile Friendly URLs in the top 10 results. This view suggests that Mobilegeddon didn’t come close to the impact of Panda or Penguin.
Conclusions
These two data views appear to contradict one another, but given the long slow rollout of the algo, and the release of the Quality Update in the middle, there are many additional factors in play. From the initial URL set, Non-Mobile Friendly URLs saw a significant negative impact, but in many cases, they ended up being replaced by new Non-Mobile Friendly URLs.
Here is a breakdown of what went on:
These other changes are likely the result of 3 factors:
- The Search Quality update
- Other algo tweaks along the way
- General churn that takes place in Google’s search results
Summary
In summary, I’d suggest that the impact of this release was indeed significantly bigger than originally met the eye. The trade press did not see it as large because of the slow rollout, and the intervening Search Quality Update.
Update: New research released by Adobe shows that they saw a drop of 12% in traffic to sites that were non-Mobile Friendly. An article released by Alistair Barr of the Wall Street Journal featured both our study (this post) and the Adobe data. You can see that article: Google’s Mobilegeddon was a big deal after all here.
In addition, this is likely just the start of what Google plans to do with this algorithm. It is typical for Google to test some things and see how they work. Once they have tuned it, and gain confidence on how the algo works on the entire web as a data set, they can turn up the volume and make the impact significantly higher.
It’s my expectation that they will do that. In the long run, don’t be surprised if the impact of this algorithm becomes even greater, and that people will stop debating whether or not it was greater than Panda or Penguin.
UPDATE: In May 2016 Google rolled out a second mobile-friendly ranking update. See “What was the impact of Google’s second mobile update?” for our updated results.
Appreciate the hard numbers that contradict the perception the update’s impact wasn’t as deep as it truly was. But it left me thinking a DEEPER dive into the Top 20-50 might have been far, far more revealing.
Google’s Top-10 is dominated by utterly entrenched Authority players with an average domain age that’s now typically 10 to 20 years old. Other established ranking factors likely protect them from mobile un-friendly flags to a greater degree. Indeed there appears to be only 1 or 2 ‘slots’ on Page 1 that Google leaves open for young and emerging sites to rank in otherwise long-entrenched niches.
In short: Studying the Gods on Mount Olympus might not be the best way to analyze the true impact on the Mortal World further below… 🙂
Thanks for the thoughts Glenn. Could indeed have been interesting to dive deeper, but we thought studying the first page (where most users spend all their time) would offer the most value.
This was a well-written article with careful analysis. I’m left feeling like the impact of this update was big but highly variable, most likely due to the quality update. It would be great to separate those two updates in the results, but one can only do so much with the hand they’ve been dealt.
I would say that I’m left wanting more from the case study urls that were tracked pre- and post- mobile update. If so many were in the #1 position as mentioned, then it’s really hard to say what any impact there was. I’m curious, maybe in aggregate what were the number of urls that remained in the top 10. I’d also like to see exactly what rank they started at (individually) and what each url’s rank was after the update. I think that last view would be the most informative as to the impact of the mobile update.
Thanks for sharing your findings Eric. It is important for brands to see data that illustrates that Google updates are real – with real consequences.
Those that feel that they can ignore what the search engines view as a ‘quality website’ (a topic I emphasize in my book) do so at their own peril.
Did you find differences in local specific searches vs general search terms? I’d be curious to know how it impacted that.
Hi Jason – Our test focused on more general queries, so I don’t have a read on that I am afraid!
Agreed Rebecca!
First, thank you for doing this analysis. Having a third party measure impact like this is invaluable. But, this image doesn’t do a great job conveying that impact.
https://www.PerficientDigital.com/wp-content/uploads/2015/06/mobile-friendly-urls-changeover.jpg
Any way we could make that easier to digest? Thanks.
Great article Eric! I have been trying hard to mobile-friendly’ize my sites, really hard when have 100’s. A lot of my older blogs have custom theme’s there are the ones I’m having a hard time trying to find coders/designers to help make them mobile-friendly.
Anyone have ideas on where to find a good person to look over a persons sites and do whatever work necessary to make them MF?
Thanks ya’ll 🙂
Superb data Eric. I’ve been curious about this for a while, I purposely left sites as not mobile-friendly to see what happens to them but the change was so minimal that I was shocked. I knew there must be more to this and this post certainly cleared that up. Would be epic to see data of at least 5 pages down! 😀
I do agree that it’s just being setup to get even bigger as time goes by.
Sharing this out. Thanks for this!
The ranking on Google are important but it seems that everybody on those blogs miss the fact that even you are ranked in the top on Google on a search from a mobile device, if the customer can´t read your page you will loose the customer.
At least Google have pushed a lot of sites to be mobilefriendly.
This gave me an idea as to why my ranking started dropping at the end of May, beginning of June. The site is competing for UK, mobile-friendly URLs and I haven’t done anything on it for a while. The mobilegeddon passed by without any changes in either traffic or rankings, it was weird. For two weeks now the rankings are recovering and I’m not sure what it was that hit the site before so h.a.r.d. was it the Search Quality update or the mobilegeddon.
Great breakdown, Eric. We are still seeing older Authority sites that are not mobile-friendly on the first page, but below the first page, it looks like there has been fluctuation.
Part of this may be the quality update coinciding with Mobilegeddon.
I also believe this is not the last fluctuation in the mobile-friendly algorithm.
I really enjoyed seeing these numbers and reading the analysis. My sites are not mobile friendly yet and I have seen a drop (not a huge one) in mobile traffic. So, now that I know that Google isn’t bluffin’ we are starting the mobile-friendly project here. We are going to include some other design improvements and better mobile monetization. Thanks for the data.
Glad to hear you’re on your way, Snuffy!
Hi Eric, sorry if I missed this info in the article, but are these changes in rank related to searches performed on mobile only, desktop or both? Thanks for the insight!
Eight weeks after the monumental Mobilegeddon, there are still 20% of sites on my 3 top keywords that are ranked in the top 5 pages that are NOT mobile friendly. Meanwhile, I’ve had a responsive site for the past 18 months and it dropped from page 5 to page 12. So based on my experience, I would have to say that the Google Mobilegeddon was nothing more than a giant hoax.
Hi Sam – searches performed from Smartphones only.
The reason most websites are in the top positions is that they are actively managed by web visibility professionals. Consequently, with all the hoopla over mobile-friendly sites, the very same sites are also the ones most likely to have been modified to be mobile friendly.
They remain at the top because they are managed by pros, not because of the mobile algo.
Jason – I have been watching 2 non-mobile friendly sites. One is a landscaper in a small market, and the other is a cosmetic surgeon in a large market. Neither of these URLs have dropped in SERPs when doing a local search for their services. Admittedly, this is neither detailed nor scientific, but it appears local hasn’t been hit hard, yet.
I have not seen a drop in many sites. All of the sites that I have are responsive, but I was hoping that it would give them an advantage.
Interesting figures. Do you know what % of results were URLs that remained in the top 10 but changed from non- to mobile friendly? With the publicity ahead of the update I can imagine some sites will have upgraded to being responsive at the last minute? Knowing the degree to which they then saw a rankings boost would be insightful.
Hi Dave – unfortunately, we don’t have that data readily available, though I can get at it with some analysis. I will see what I can do to get that done.
Thanks for the report Eric, however I don’t see where you mentioned if your testing was done on mobile devices or desktop. From what I’ve heard by Google experts, Mobilegeddon only affects mobile search, and also sites that were already ranking high (even if not mobile friendly) may not be affected. I have only seen minimal changes in sites that I have redesigned for mobile.
Hi Lori – all testing was done on Smartphones.
With regards to ‘mobilegeddon’ and the rights and wrongs of Google pushing webmasters to make major changes to their sites: Google’s been calling the shots for years and if you want to be found in their results you’ve got to play by their rules. Plus they did give warning this time (for a change). Recently I’ve tried to visit a few sites to be warned by Google that they aren’t secure; given that Google announced that https would give you a bump in the rankings; I’m predicting that securing sites will soon be added to webmaster’s ‘to do’ lists
Hi Nick – I agree, I think that HTTPS will be made a bigger factor sometime soon. Note that I am working on an interview with Gary Illyes about their plans for HTTPS, so I hope to have something to say about that soon!
Thanks for the great post Eric. I work for a predictive analytics company here in Salt Lake City called diib (www.diib.com). We watched Google’s mobile update very closely–not as closely as you guys did :).
One of the things I wondered about was having a strong ranking in Google’s mobile searches and its impract on certain metrics, like mobile bounce rate and social channels. I’m willing to guess that those sites that moved up in Google’s mobile searches might also be doing better with their traffic from Facebook, Twitter, and Pinterest. However, with mobile bounce rate, I am going to guess that going up on Google’s mobile searches might lead to a slight increase in bounce rate, as most people just check the first page for contact information or address on a mobile website.
Eric,
Great work – but please, can you detail how you did your ranking study?
For instance, what ranking tools you are using?
I’m viewing position reports by device through the GWMT Search Analytics interface across numerous sites and am seeing no appreciable difference before and after for a variety of sites.
also congrats – great to see you linked from and cited in the WSJ!
We used our own internal rank tracking. We downloaded the entire set of SERPs, and have them on our servers here, and then ran the ranking tool on those. The reason that’s important is that it allowed us to hand verify thousands of the results to make sure we were 100% satisfied with the accuracy.
This is great Eric, I am able to use this in a presentation I am developing for a conference I am presenting at next week to the tourism industry! Thanks again for your great evidence- based insight. You guys are certainly my go to peeps.
I can not find very many websites ranking for anything of substance that are not mobile friendly these days. It seems people have smartened up!