The latest version of Perficient Digital’s now annual Digital Personal Assistants accuracy study is now out. The study compared responses to about 5,000 queries on seven different devices, which includes Amazon’s Alexa, Microsoft’s Cortana, Google Assistant and Siri.
Google Assistant performed the best at the highest level, but Cortana tried answering the most questions. An attempt on Alexa’s part was made in an attempt to answer question. Overall, the study found that accuracy seemed to have declined across all devices.
The following chart compares devices by numbers of answers attempted. This means that they didn’t respond with “I can’t help with that yet” or “I don’t know that.” Cortana and Alexa, over the last three years, have grown the most in answer attempts, and Cortana has been edging Google for most questions answered, even if they weren’t always correct.
Based on the results, Google (on smartphone) is the most accurate assistant with Alexa coming in second. As mentioned above, accuracy appears to have declined across the devices and most for Cortana, which could be related to its attempts to answer more questions. Siri even suffered quite a meaningful decline in accuracy.
Perficient Digital even explored the use of featured snippets by the assistants. It defined snippets as “answers provided by a digital personal assistant or a search engine that have been sourced from a third party” (with attribution).
Google was able to serve up the most snippets, with Google Home beating out the Assistant on smartphone. The smartphone version of Google Assistant saw a significant decline use of snippets. It was the only platform to see such a decline.
In a lighter note, Alexa and Siri tied for the most jokes offered in response to queries. The were both deemed to be “the funniest” assistants.