Which voice search app – Google, Siri, or Windows Cortana – is best at directly answering questions?
We wanted to find out, so we prepared a set of over 3000 questions that could potentially generate a simple answer and asked them to all three apps. For complete test methodology and results, see our article The Great Knowledge Box Showdown.
Now here are the chief results of our test in one sharable infographic. To embed this infographic on your own site, scroll to the bottom and copy the provided embed code.
Click the image to view full size
)
Mark,
I goes to show the first to market advantage is somewhat overblown. Sure, Siri was the first, but she obviously leave s something to be desired.
Steve
Mark/Eric — Just to be clear, you are saying with this study that Google answered 58% of the 3000 questions directly and of those, 88% were answered completely? So the overall expectation would be that Google Now could answer your question well and completely 51% of the time?
Hi Jeff – yes, that is correct. However, be aware that we picked questions that they engines might be able to answer.
Yeah, I get that Eric. In the Butler Study of Jan 2013, Google Now came out at about the same number for coverage and accuracy [50%], though of course the corpus of questions was different.
I initially thought based on your study that Google had made significant progress past the 50% barrier. And given the massive growth in their knowledge graph, I expected movement.
But that didn’t happen… I think we can attribute that lack of progress to how hard semantic understanding really is.
Out of curiosity, what was the fail rate of their research vs. their language comprehension? How often did they fail to understand the question being asked and how often did they fail to find the correct answer?
Hi LJ – this is not something we measured. However, the failure to recognize the spoken commands was relatively low.
Nice work guys – both on the Study and the Infographic itself. my company is doing similar type of study , but knowing the results of the work you guys have done helps us out a lot. So thank again, You Guys Rock!!
Thanx for a great article discussing voice search issues. appreciated and shared further.
I assume Google answers more complicated questions better than both Siri and Cortana? I mean as you have shown with the first and second example – astronomy question – the three tied while only Google was able to answer the second one correctly.
Thanks for sharing and Discussing the voice app. I look forward to read more and sharing in social media site’s.