Now that Google has full, real-time access to all of Twitter, they must be indexing every tweet in Google search, right? Wrong!
In this video see the results of the Perficient Digital study that looked on Google for 133,000 tweets from over 900 Twitter users. We gathered data on the level of indexation of tweets from those accounts in February 2015 (before the Google-Twitter deal) and again in June 2015 (after Google began to implement the new tweet access).
Mark and Eric explain why, even though Google now has full “firehose” access to Twitter, it still doesn’t even come close to indexing every tweet in its search engine.
The Digital Essentials, Part 3
Developing a robust digital strategy is both a challenge and an opportunity. Part 3 of the Digital Essentials guide series explores five of the essential technology-driven experiences customers expect, which you may be missing or not fully utilizing.
Don’t miss a single episode of Here’s Why. Click the subscribe button below to be notified via email each time a new video is published.
- How Does Google Index Tweets Today? How the New Twitter Deal Will Impact SEO
- Tweets take flight in the Google app
- Twitter Usage Statistics
- All of our “Here’s Why” Videos
- Subscribe to our YouTube Channel
- Subscribe via Google+
Mark: Google now has access to every tweet in real time, so that means every tweet can be found in Google’s search results now, right? Wrong! In fact, a study we did at Perficient Digital shows that while after Google started implementing their new firehose deal with Twitter they also began indexing many more tweets, they aren’t even coming close to getting all of them. In this episode of Here’s Why, I’ll have Perficient Digital’s Eric Enge share our study results, and ask him why Google still isn’t indexing all of Twitter.
Mark: Eric, tell us about the study you conducted to see how much of Twitter is indexed by Google.
Eric: Sure thing, Mark. First, to capture how much of a change Google’s implementation of its firehose access to tweets in real time brought, we’d need good before and after snapshots. Fortunately, we had captured a really good picture of Google’s indexation of Twitter in February of 2015. At that time we pulled data on over 133,000 tweets and then looked for those tweets on Google. We found that only just over 0.6% of tweets showed up in Google’s search results within the first 7 days, and 7.2% within 7 weeks.
That means almost 93% of the tweets in our test set never showed up in search! In addition, most of these took more than a week to get indexed. Then in late May Google announced that they were beginning to make use of their new deal with Twitter and start showing more tweets in search results. So in June, we took a second snapshot. For a good comparison, we looked for the same data set of tweets in our June scan. This time we found that 3.4% of the tweets were indexed within 7 days, a whopping 466% increase over February. But of course, 3.4% is a tiny minority of all the tweets.
Mark: So now that Google has access to all of Twitter in real time, why aren’t they indexing 100% of the tweets?
Eric: There are at least a couple of probable reasons. First, keep in mind that we’re still in the very early days of this implementation. Google is probably testing a lot of things to see how to bring tweets into search in a way that’s most useful to searchers. I expect that they will continue experimenting and that we will see a lot of changes in the time ahead. Here at Perficient Digital, we’ll rerun this study periodically to track and report how Google’s Twitter indexation changes over time.
Mark: If you want to be first to know about those updates, and all of our social media, SEO, and content marketing studies, sign up for our newsletter at the link on your screen. Eric, you said you had a couple of possible reasons Google still isn’t indexing all of Twitter. What’s the other one?
Eric: What we’re seeing, Mark, is actually a good reminder that as huge as Google is, they don’t actually have unlimited resources. Even though their computing and storage capacities are probably the largest ever known, they still have limitations. So they have to make decisions about how and for what to allocate those resources. And then think about Twitter. Every second Twitter users create 6000 tweets. That’s 350,000 tweets per minute or 500 million tweets per day.
At that rate, even Google’s computing resources would get quickly overrun. So Google has to be selective, using algorithms that prioritize which tweets and Twitter users should get the most attention. In fact, in our study data, we found a strong correlation between the social authority of a Twitter profile and how likely its tweets were to show up in Google search. Higher authority profiles get indexed more, although I’m sure there is more to Google’s criteria than just profile authority.
Mark: Thanks Eric. We hope you’ve enjoyed learning some of the “why” behind this quickly evolving search and social media story. You can see our other Here’s Why videos on our Here’s Why page at the link on your screen. On that page you’ll also find a link to our YouTube channel where you can subscribe to this video series so you’ll never miss another episode of Here’s Why. Thanks for watching!