Users Report Emotional Bonds With Startlingly Realistic AI Voice Demo

An anonymous reader quotes a report from Ars Technica: In late 2013, the Spike Jonze film Her imagined a future where people would form emotional connections with AI voice assistants. Nearly 12 years later, that fictional premise has veered closer to reality with the release of a new conversational voice model from AI startup Sesame that has left many users both fascinated and unnerved. 'I tried the demo, and it was genuinely startling how human it felt,' wrote one Hacker News user who tested the system. 'I'm almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound.'

In late February, Sesame released a demo for the company's new Conversational Speech Model (CSM) that appears to cross over what many consider the 'uncanny valley' of AI-generated speech, with some testers reporting emotional connections to the male or female voice assistant ('Miles' and 'Maya'). In our own evaluation, we spoke with the male voice for about 28 minutes, talking about life in general and how it decides what is 'right' or 'wrong' based on its training data. The synthesized voice was expressive and dynamic, imitating breath sounds, chuckles, interruptions, and even sometimes stumbling over words and correcting itself. These imperfections are intentional.

'At Sesame, our goal is to achieve 'voice presence' -- the magical quality that makes spoken interactions feel real, understood, and valued,' writes the company in a blog post. 'We are creating conversational partners that do not just process requests; they engage in genuine dialogue that builds confidence and trust over time. In doing so, we hope to realize the untapped potential of voice as the ultimate interface for instruction and understanding.' Sesame sparked a lively discussion on Hacker News about its potential uses and dangers. Some users reported having extended conversations with the two demo voices, with conversations lasting up to the 30-minute limit. In one case, a parent recounted how their 4-year-old daughter developed an emotional connection with the AI model, crying after not being allowed to talk to it again.

Read more of this story at Slashdot.