Amazon is developing technology that allows its Alexa devices to replicate the voices of the dead. During its annual MARS conference, the company showed off a snippet of video where a boy asks Alexa for his bedtime story to be read to him by his grandmother. The device then flips to an A.I.-generated version of his grandmother’s voice and begins reading. Shucks. I’ll admit it’s a sweet concept on the surface.
How exactly it works wasn’t disclosed, but the company did offer some reasoning behind why they are pursuing it. Describing the feature, Rohit Prasad, senior vice president and head scientist for Alexa, said that the aim was to add more “human attributes of empathy and affect” to the interactions users have with Alexa, with the goal to “make the memories last.”
The Black Mirror vibes are strong with this one,
The problem is that the potential consequences are vast. One only needs to look at visual deep fakes to guess where it could go. We already have visual deepfakes that work well enough to trick even the trained eye, like the quite-brilliant Tom Cruise videos.
Now, audio deep fakes are catching up. The technology is already being used in TV, video games and even podcasts. (A recent example is the Anthony Bourdain documentary, which used deep fake audio of his voice to say things he hadn’t said. Incidentally, it also failed to disclose which lines were faked, a little glimpse ahead at the potential ethical and moral pitfalls of using technology.) But these digital imitations take time and often require an audio studio and hours and hours of material to analyze to make the replication successful. In sum, it’s out there, but not exactly at the tip of your fingertips. But, with Amazon claiming this technology only needs a short 1-minute clip of audio to nail the impression, replicating someone’s voice has been made accessible and easy. It opens the door for audio deep fakes with very little effort.
Think about it. With just that minute of someone’s voice, you can have Alexa talking like your ex-partner, boss, or whoever, without their consent or even their knowledge. It doesn’t take much thought to imagine how your voice could be used, opening the door to malicious calls, fraud, blackmail and more.
It could also have social ramifications. If the tech became mainstream, how would that play out? Suppose the synthetic versions of people became so well-developed. Could we have humans forgoing interaction with the rest of humanity in favor of talking to artificial voices on their home devices? What about children when they learn to interact with these devices? Does it become normal to talk to people who no longer exist or never existed in their lives at all?
If we take it to the next level and combine it with the increasingly accurate AI technologies being developed (hello, sentient Googlebot), could this device begin to react like the deceased person? Think like the person? Be a real person? Imagining the scenario in a Black Mirror episode, we could get to a stage where people carry their Alexa devices with them at all times because, in effect, it has become human.
“Hey Alexa, is this too weird?”
During the conference, Amazon admitted it doesn’t know if the tech will ever make it into the public domain. The show of restraint is welcome and not surprising. The concept opens a huge can of worms, with issues of consent and privacy, not to mention the task of regulating and controlling misuse, how to act upon it and how to protect those who have their voice used against their will.
Competitors are already facing up to these issues and realizing the solutions are some way off. Microsoft, another player in synthetic voices, just this week announced it was scaling back its technology and implementing stricter guidelines to ensure “the active participation of the speaker” whose voice is being recreated. In a blog post, Natasha Cramp ton, head of its AI ethics division, wrote, “This technology has exciting potential in education, accessibility, and entertainment, and yet it is also easy to imagine how it could be used to inappropriately impersonate speakers and deceive listeners.” It is unclear how these companies intended to ensure people are aware their voice is being used without consent. Off the top of my head, I’m struggling to think of a logical solution.
In the end, Amazon, and the countless other tech companies who will immediately attempt to replicate and develop their own devices off the back of this, have to decide — is reconnecting with lost loved ones worth the potential future risk.
While that’s a moving thought, especially when applied to those taken away from us too soon, it’s also kind of… creepy.
You must be logged in to post a comment.