Perficient Digital publishes the latest version of its annual study on the accuracy of digital personal assistants . He compared responses to about 5,000 queries on seven devices, including Amazon's Alexa (Echo and Echo Show), Microsoft's Cortana, Google Assistant (Home, smartphones) and Siri.
More answers, less precision. At the highest level, Google Assistant was the best, but Cortana was trying to answer most questions. Alexa has also shown improvement in response attempts. But accuracy has declined on all devices, according to the study.
The table below compares devices based on the number of responses attempted. In other words, they did not answer with the words "I do not know" or "I can not stop it at the moment". Three years of data show Cortana and Alexa Most of the response attempts have been successful, with Cortana reading Google for most questions (but not always correctly).
Alexa is the second most accurate assistant after Google. The most accurate wizard is Google (on a smartphone). Alexa comes in second. However, the accuracy seems to have decreased in all areas, especially in Cortana, which could be related to his attempt to answer more questions. Siri also suffered a significant drop in accuracy.
Perficient Digital explored the use of excerpts described by the assistants. It defines code snippets as "answers provided by a personal digital assistant or a search engine from a third party" (with attribution).
Decline in the use of code snippets by Google. Google has provided the most excerpts, with Google Home flying the wizard on smartphones. However, Google Assistant on smartphone has also recorded a significant drop in the use of code snippets. It was the only platform to see such a decline.
Finally, Alexa and Siri were matched for the largest number of jokes proposed in response to questions. As a result, they were deemed to be "the funniest assistants".
Why we should care. Perficient Digital concludes in its analysis of the data that "progress has slowed to a certain extent". The company explains: "We no longer see any major progress on the part of the players," adding, "The next significant step forward will probably require a new approach. "Google may have found this approach in BERT which helps the search engine better understand the meaning of more conversational queries.