Amazon ran a commercial on this year's Super Bowl that pretended its digital assistant Alexa had temporarily lost her voice. It featured celebrities like Rebel Wilson, Cardi B and even the company's chief executive, Jeff Bezos.While the ad riffed on what Alexa can say to users, the more intriguing question may be what she and other digital assistants can hear - especially as more people bring smart speakers into their homes.
Amazon and Google, the leading sellers of such devices, say the assistants record and process audio only after users trigger them by pushing a button or uttering a phrase like "Hey, Alexa" or "OK, Google."
But each company has filed patent applications, many of them still under consideration, that outline an array of possibilities for how devices like these could monitor more of what users say and do. That information could then be used to identify a person's desires or interests, which could be mined for ads and product recommendations.
In one set of patent applications, Amazon describes how a "voice sniffer algorithm" could be used on an array of devices, like tablets and e-book readers, to analyse audio almost in real time when it hears words like "love," bought" or "dislike."
Some patent applications from Google, which also owns the smart home product maker Nest Labs, describe how audio and visual signals could be used in the context of elaborate smart home setups.
One application noted that voices could be used to determine a speaker's mood using the "volume of the user's voice, detected breathing rate, crying and so forth," and medical condition "based on detected coughing, sneezing and so forth." The same application outlines how a device could "recognize a T-shirt on a floor of the user's closet" bearing Will Smith's face and combine that with a browser history that shows searches for Smith "to provide a movie recommendation".
In a statement, Amazon said the company took "privacy seriously" and did "not use customers' voice recordings for targeted advertising." Google said it did not "use raw audio to extrapolate moods, medical conditions or demographic information." The company added, "All devices that come with the Google Assistant are designed with user privacy in mind."
But many consumers are also becoming increasingly nervous that tech companies are eavesdropping on them. The recent revelations that data firm Cambridge Analytica harvested the information of 50 million Facebook users has only added to the public's wariness over the collection and use of personal information.