<h1>Nari Labs Releases Dia: A 1.6B Parameter Open-Source TTS Model for Real-Time Voice Cloning and Expressive Speech Synthesis</h1>
<p>Nari Labs has unveiled Dia, a groundbreaking open-source text-to-speech (TTS) model equipped with 1.6 billion parameters, designed for real-time voice cloning and expressive speech synthesis on consumer devices. Under the Apache 2.0 license, Dia presents a formidable alternative to proprietary TTS systems, emphasizing accessibility and scalability for both commercial and academic applications.</p>
<h2>Key Features of Dia</h2>
<ul>
<li><strong>Real-Time Voice Cloning</strong>: Dia supports zero-shot voice cloning, enabling users to replicate voices with minimal input data.</li>
<li><strong>Expressive Speech Synthesis</strong>: It is capable of generating ultra-realistic dialogue in a single pass, conditioning on audio inputs to control emotion and tone.</li>
<li><strong>Non-Verbal Sound Generation</strong>: The model can produce non-verbal sounds, enhancing the realism of the synthesized speech.</li>
<li><strong>Efficient Operations</strong>: Designed to operate efficiently on consumer devices, Dia also runs on GPUs, with planned support for CPUs in the future.</li>
<li><strong>Modular Design</strong>: This allows for ease of integration and customization, making the model adaptable to various use cases.</li>
</ul>
<h2>Availability and Accessibility</h2>
<p>Dia is available for download on platforms like Hugging Face and GitHub, offering pretrained model checkpoints and inference code. Detailed installation instructions accompany the model to ensure smooth setup. A demo comparing Dia with ElevenLabs Studio and Sesame CSM-1B showcases its capabilities and performance.</p>
<h2>Background and Development</h2>
<p>Developed by two undergraduates, Toby and Jay, Dia was inspired by Notebook LM podcasts. Despite the lack of funding, they achieved notable results, drawing influence from Google’s Soundstorm and Parakeet models. The training of Dia leveraged Google’s TPU chips, underscoring the model’s advanced technical foundation.</p>
<h2>Ethical Considerations and Legal Guidelines</h2>
<p>Nari Labs has provided ethical and legal guidelines to govern the use of Dia, addressing concerns about potential misuse and the ethical implications of utilizing possibly copyrighted data. These guidelines are intended to ensure responsible usage of the technology.</p>
<h2>Future Directions</h2>
<p>Looking ahead, Nari Labs plans to expand Dia’s capabilities, including the development of a social platform and support for additional languages. These enhancements aim to broaden the scope and usability of the model, making it more accessible to a diverse range of users.</p>
<p>Dia’s release marks a significant milestone in the TTS landscape, challenging established models from ElevenLabs, OpenAI, and others, and advancing the field of real-time voice cloning and expressive speech synthesis.</p>


Collections

Nari Labs has introduced Dia, a 1.6 billion parameter open-source text-to-speech model designed for real-time voice cloning and expressive speech synthesis. Featuring zero-shot voice cloning, non-verbal sound generation, and efficient operations on consumer devices, Dia is versatile and accessible for both commercial and academic use. Developed by undergraduates inspired by Google's models, it is available on platforms like Hugging Face and GitHub, with guidelines for ethical and responsible usage.

Nari Labs' Dia: An Advanced TTS Model Capable of Ultra-Realistic Dialogue Generation