Block’s New Voice AI Model Sounds Uncannily Human
Summarized by AI from reporting by TechCrunch AI, published under our editorial policy.
Block has introduced a new voice AI model called Dessa that runs locally and mimics human speech with emotions. The model is trained on human speech data and can operate on consumer-grade computers, though it’s not yet publicly available.
Key takeaways
- Block’s new voice AI model, Dessa, sounds uncannily human with expressions like laughs and pauses.
- Dessa is a full-duplex model that can talk and listen simultaneously, trained on human speech data.
- The model can run locally on consumer-grade computers like a MacBook M4 Max or better.
- Block is seeking safety researchers to test Dessa before making it publicly available.
- Dessa’s emotional expressions are built into the model, unlike many other voice AI models.
Block, the financial technology company founded by Jack Dorsey, has unveiled a new voice AI model named Dessa from its research lab. This model runs locally and sounds remarkably human, complete with expressions like laughs, pauses, and cries. Unlike many other voice models, Dessa is trained entirely on human speech data and a single Nvidia GPU rack. This development marks a significant step forward in the realm of voice AI, offering a more natural and emotionally expressive interaction experience.
What Makes Dessa Unique
Dessa is a full-duplex model, meaning it can talk and listen simultaneously, similar to OpenAI’s GPT-Live model. Most speech models rely on a large language model for knowledge retrieval, but Block claims Dessa’s knowledge comes from conversational data it has been trained on. The company also asserts that Dessa’s emotional expressions are built into the model itself, rather than relying on voice tags to drive emotion, as seen in models like ElevenLabs v4 or Fish Audio Drama 3. This approach aims to create a more seamless and human-like interaction.
Block criticizes current voice AI for being unnatural and annoying, stating that these models convert voice into text, losing valuable emotional information. The company believes this leads to a worse user experience and potential dangers. To demonstrate Dessa’s capabilities, Block’s researchers shared multiple examples where the model exhibited different voices, cadences, and emotions. For instance, in one conversation, the bot sounded tired after mentioning it was tired from work. In another example, the bot was talking about making someone comfortable while sounding emotional and almost breaking down. These examples highlight the model’s ability to convey a wide range of emotions and tones.
How Dessa Compares to Existing Models
Dessa can run locally on consumer-grade computers like a MacBook M4 Max or better. This local processing capability sets it apart from many cloud-based voice AI models, offering potential benefits in terms of privacy and speed. However, there is no public access to Dessa at the moment. Block has called for safety researchers to test the model, likely to identify scenarios where a voice agent could deceive users. This focus on safety is crucial, as voice AI models can potentially be used to manipulate or trick people.
Block’s foray into advanced voice AI could have significant implications for both businesses and consumers. The company is already using voice AI in its Square payment solutions to handle ordering. With a locally running voice model like Dessa, Block can power a wider range of business and consumer experiences at a lower cost. This could lead to more natural and efficient interactions with AI, enhancing user experiences in various applications. For example, businesses could use Dessa to provide more personalized and empathetic customer service, while consumers could enjoy more engaging and natural interactions with their devices.
Why This Matters for Everyday Users
The introduction of Dessa represents a significant advancement in the field of voice AI. By running locally on consumer-grade hardware, Dessa offers a more privacy-focused alternative to cloud-based voice AI models. This could be particularly appealing to users concerned about data privacy and security. Additionally, the model’s ability to convey emotions and tones more naturally could make interactions with AI feel more human and less robotic. This could enhance user experiences in a variety of applications, from virtual assistants to customer service bots.
What You Can Do Today
While Dessa is not yet publicly available, you can explore other advanced voice AI models to get a sense of the current capabilities. For example, you can try OpenAI’s GPT-Live or other voice AI services to see how they compare to traditional text-based interactions. Keep an eye on Block’s updates for any announcements about Dessa’s availability and testing opportunities. Additionally, you can consider the implications of voice AI in your own life and how it might enhance or change your interactions with technology.
Frequently asked
- Is Dessa available to the public?
- No, Dessa is not yet publicly available. Block has called for safety researchers to test the model before making it accessible.
- How does Dessa differ from other voice AI models?
- Dessa is trained on human speech data and can run locally on consumer-grade computers. It also incorporates emotional expressions directly into the model, unlike many other voice AI models that rely on voice tags.
- What devices can run Dessa?
- Dessa can run locally on consumer-grade computers like a MacBook M4 Max or better.
- Why is Block focusing on voice AI?
- Block believes that current voice AI models are unnatural and annoying, losing valuable emotional information. They aim to improve user experiences with more natural and efficient interactions.