DME

Google Home vs. Amazon Alexa Showdown: Which One is Smarter?

Webp.net Resizeimage 2019 06 04t090558.398

The arrival of the Amazon Echo in the U.S. on June 23, 2015, was notable because it was the first mass-market consumer-level device responding to voice control with general purpose functions, such as playing music. More recently on November 4, 2016, Google released a competing device known as Google Home.
As seen in International Business Times
At Perficient Digital, we’ve been tracking the capabilities of voice-activated knowledge devices to respond to questions since we released the Knowledge Box Showdown in October of 2014. In today’s article, we’re going to publish the data on similar tests, with Amazon Echo and Google Home in head-to-head competition.
What We Tested: It’s very important to note that our test is NOT a test of all the capabilities of both devices. Our test focused on the ability to provide direct answers to specific user questions. In short, we’re only testing the ability of each device to provide factual answers to questions. The devices may use one of two sources for those answers:

  1. A knowledge database, such as Google’s Knowledge Graph
  2. Answers extracted from third-party websites (what Google calls “Featured Snippets”)

What We Didn’t Test: The devices have several other areas of functionality that we did NOT test. These include:

  1. Home Entertainment
  2. Personal Assistant
  3. Smart Home Controller

If you prefer to consume this post via video, you can check out my YouTube video here!

Covid 19
COVID-19: Digital Insights For Enterprise Action

Access Perficient’s latest insights into how you can leverage digital technologies to not only respond to the pandemic, but drive your operations forward and deliver experiences your customers need.

Get Informed

You can also see a much broader comparison of all the capabilities of Google Home and Amazon Echo here.

Structure of the Test

We collected a set of 5,000 different questions that we wanted to ask each device. We then asked the devices the question, and noted many different possible categories of answer, including:

  1. If the device answered verbally
  2. Whether an answer was received from a database (like the Knowledge Graph)
  3. If an answer was sourced from a third-party source (“According to Wikipedia …”)
  4. How often the device did not understand the query
  5. When the device tried to respond to the query, but simply got it wrong

Did you know that you can “Ask Perficient Digital” SEO & digital marketing questions on Google Assistant and Amazon Alexa devices? Here’s how to do it!

Test Results

For the first test result, let’s compare how often the two devices attempted to answer the questions asked:
 

[Tweet “Google Home attempts an answer to over 3X more questions than Amazon Alexa. See study at”]
So Google Home tries to answer the most questions, but for all of the questions each responded to, how often are they correct?

As it turns out, there are a few different ways to be “wrong”:

  1. The query might have multiple possible answers, such as “how fast does a jaguar go”
  2. The device may have responded with a joke
  3. Or, it may simply get the answer flat out wrong

Let’s first take a look at that last category, the for all of the questions each device responded to, what percentage of the time were they wrong?

[Tweet “Amazon Echo answers questions incorrectly over 2x more often than Google Home. See study at”]
The remaining queries were made up of situations where more than one answer was possible, or where the question was partially answered. For example, if someone asked “how old is George Washington,” and the device responded with the year he was born, we would count that as partially answered.

Summary

All in all, for this particular test, Google Home won handily. As noted above, there are several other important categories for which the devices can be analyzed and compared that weren’t included in this test.
However, with the backing of Google’s search volume, Google Home has the clear edge in the knowledge department. In particular, 2,228 of Google’s answers came in the form or “Featured Snippets,” which means that the results were sourced from a third-party website. 2,122 of the correct answers for Google Home were from this source (not all of the 2,228 featured snippets were 100% correct!). In contrast, for the Amazon Echo, only 556 of their correct answers came from third-party websites, 527 of which were sourced from Wikipedia.

About the Author

Eric Enge leads the Digital Marketing practice for Perficient. He designs studies and produces industry-related research to help prove, debunk, or evolve assumptions about digital marketing practices and their value. Eric is a writer, blogger, researcher, teacher, and keynote speaker and panelist at major industry conferences. Partnering with several other experts, Eric served as the lead author of The Art of SEO.

More from this Author

Thoughts on “Google Home vs. Amazon Alexa Showdown: Which One is Smarter?”

  1. Google helped me to answer the spam protection question on this page. So, if you can red this – it was right 🙂 I am extremely interested in Digital Assistants since years – but the problem in europe is, that our versions are strongly behind and you never know if it does not work or if there is no feature. Thanks for testing this – it will be helpful for my session at #SEOcampixx Berlin. Maybe one day we will do a session in the US if there is a feature like ski-lift and ski-area Assistant by Google, Cortana, Siri … or other that we can show in the us.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to the Weekly Blog Digest:

Sign Up