Jack Dorsey’s Block Unveils Dessa, an AI Model Trained Exclusively on Human Speech

Image: Jack Dorsey’s Block made an AI model that talks to you like an awkward new friend · Source

Block, Jack Dorsey’s company, revealed Dessa, an AI voice model developed solely from human conversations, aiming to enhance natural speech interaction and emotional understanding in AI.

Block, the company founded by Jack Dorsey, has introduced a pioneering AI model named Dessa that is trained entirely from human speech data without relying on text. This speech-to-speech model is designed to engage in conversations more naturally, reflecting the nuances and timing inherent in human interaction. According to The Verge, Dessa’s development team emphasizes that the model’s learning process mirrors the way babies acquire language — by listening and speaking rather than reading or writing.

Dessa can operate locally on devices with the computing power of a MacBook Pro M4 Max or better, which underlines Block’s focus on accessible, high-quality AI voice interaction. The company showcased several conversations that demonstrate Dessa’s capabilities, including a quirky dialogue where it initially mistakes a person’s identity, and a guessing game played over voice. Additional exchanges reveal Dessa’s unexpected and emotionally resonant responses, such as expressing gratitude after being told about a surprise trip to Paris.

One notable feature is Dessa's ability to detect emotional cues in speech, addressing a critical safety concern. The developers pointed out that current voice AIs often flatten speech to text, neglecting the emotional context, which can be detrimental, for instance, when a user is in distress. By preserving emotional inflections, Dessa represents a step toward safer and more empathetic AI interactions.

The motivation behind Dessa extends beyond curiosity and entertainment. The Block team aims to deepen understanding between humans and AI while improving safety. To this end, Block is granting early access to a select group of AI safety researchers, indicating their commitment to collaborative development and responsible innovation.

While Dessa is not yet widely available, its unique approach to voice AI, built from scratch using only human conversations, positions it as an intriguing advancement in the field. It challenges conventional voice AI paradigms that rely heavily on text-based training and could influence future developments in natural and emotionally aware artificial intelligence.