One of my friends was absolutely stoked when she got her Alexa for Christmas last year. She took pride of place on the shelf beside her wall-mounted TV, where she could maximize her listening capability. Over the next few months, Alexa began to sink into the background of her home, almost as another family member. Whenever she needed help remembering the final ingredient of a recipe, or who was in the Six Million Dollar Man, Alexa came to the rescue.
But over a few visits, I started noting a trend. Alexa could almost predict what my friend was going to look for. Later, I learned this was simply a function of an algorithm a lot like predictive text and not at all as nefarious as I originally suspected, but the seeds were sewn.
“You know that thing’s listening to you, right?” I told her one night.
“Alexa? Of course she’s listening.”
But she didn’t understand. I barely understood myself.
The recent revelations from Edward Snowden’s PRISM leaks that the NSA was spying on its own people, accompanied with Facebook’s (FB.Q) implications in the Cambridge Analytica scandal, had left a bad taste in our collective mouths regarding the supposed beneficence of Silicon Valley.
In 2015, Tim Cook, Apple’s (AAPL.Q) CEO, called privacy a “fundamental human right,” while his machines were busy processing more than a billion requests a week. The technology had advanced so far by that time that users no longer needed to push a button to activate the voice assistant. It could be assumed that she was always listening, and she was.
“Having microphones that listen all the time is concerning. We’ve found that users of these devices close their eyes and trust that companies are not going to do anything bad with their recorded data. There’s this creeping erosion of privacy that just keeps going and going. People don’t know how to protect themselves,” said Florian Schaub, a University of Michigan professor who studies human behaviour around voice-command software.
You have to dig into the legalese to find the proof, but Apple said that their voice data might be recorded and analyzed to improve Siri, but nowhere in there did it say that other human beings might be listening. Yet these smart-devices depend on thousands of low-paid humans to listen to every racist and homophobic rant, every bout of flatulence and every time you have sex, so tech companies can upgrade their platform.
Your deepest darkest secrets, whispered into Siri’s hungry ears as a 3 a.m. drunken confessional, have become one of their most valuable datasets.
And because there’s so much competition in this market the companies involved have only gotten more aggressive about their data collection habits. Apple treated Siri’s development like a verbal search engine, preparing endless user queries and ramping up its dependence on audio analysis to feed the assistant’s lexicon. They employed temps to for clips of various languages, dialects and cultural idiosyncrasies.
It makes me wonder whether a not a Siri operating in Cape Breton uses ‘aboot’ when she really means ‘about.’
But I digress.
My friend is right about some things. Siri only responds when her name or a codeword is given. Hers does, at least. But there have been cases where Siri (or Alexa or the loathsome Cortana) has awakened by accident. A chance swirling of molecules from a raised arm or a zipper pulled a quarter of a microsecond too fast, and her lights flash and she scans the room and starts listening.
By 2019, after Apple brought Siri to products such as its wireless headphones and HomePod speaker, it was processing 15 billion voice commands a month; 0.2% of 15 billion is still 30 million potential flukes a month, or 360 million a year.
It gets worse…
The people categorizing your 3 a.m. confessions work long hours under terrible tech-sweat shop conditions, listening to potentially traumatizing audio, while the corporations steal their wages and then cut them off at the knees.
Former contractors from a GlobeTech office in Cork, Ireland, make references to George Owell’s 1984. They sit in silence at MacBooks wearing headphones, slaving away transcribing 1,300 clips a day, each of which could be a single sentence or an entire conversation. The quota, they say, has actually come down from 2,500 clips, to improve accuracy rates. The contractor clicks o n a voice recording and the computer populates a text box with words it thought Siri heard, and the worker is then prompted to either approve or correct the translation.
But it gets even worse than that.
Not all Siris and Alexas and Cortanas are built equally. Some may roll off the line with loose chips, or microphones set to always-on, or wonky drivers, making the machine do unforeseen things.
For example:
In Portland, Oregon, a woman’s Echo sent recordings of private conversations to one of her husband’s employees. Amazon (AMZN.Q) said that the Echo must have misheard the wake word, followed by a misheard request to send a message, oh and then misheard a name in its contacts list and then again, another misheard confirmation to send the message—all during a conversation about hardwood floors.
Or maybe how an Amazon customer in Germany received 1,700 audio files from some random person’s Echo, giving enough information to name and locate the user and his girlfriend. Amazon called this one human error, but companies will say anything to avoid accountability.
Oh, it gets so much worse…
Want to see something scary?
There’s a few of these videos on YouTube. Go look at them.
It’s surprisingly easy to hack one of these devices. Checkmarx, a cybersecurity firm, attached malicious code to a seemingly harmless app, and used it to gain access. The firm took advantage of Alexa’s functions to prevent the system from shutting down at its programmed time after a response had been given. Alexa has a flag that allows a session to remain open if an app requires a second response. It’s how you can use Alexa to set an alarm. You say 7:30 and it will take a moment and ask you whether you meant morning or night, and then shut down after the alarm is set. The hack allowed the flag to stay open and keep recording after the expected response.
So it might not be only Amazon, Google (GOOG.Q), or Microsoft (MSFT.Q) that’s listening in on your conversations.
I told my wife I’d dropkick it off the back balcony if she got me one of those things for Christmas.
Lucky for me, though, she’s even more paranoid about these things than I am. Smart girl.
—Joseph Morton