AI Startup MyShell Releases OpenVoice Algorithm for Precise Voice Cloning
In Brief
Canadian AI startup MyShell announced it has open-sourced its OpenVoice algorithm for instant voice cloning.
Respeecher, Voicemod and ElevenLabs – all three startups have one thing in common – they all provide algorithms and AI software for making voice clones. Now, a new player, Canadian AI startup MyShell announced it has open-sourced its OpenVoice algorithm for instant voice cloning.
MyShell shared the update on social media platform X and said, “Clone voices with unparalleled precision, with granular control of tone, from emotion to accent, rhythm, pauses and intonation, using just a small audio clip.”
Under the collaboration, researchers from MIT, MyShell.ai and Tsinghua University unveiled OpenVoice, which can replicate a speaker’s voice and the generate speech in multiple languages, using only a brief audio snippet from the original source. It also captures the unique tone and color of the speaker’s voice.
According to the company, the algorithm adds crucial stylistic elements such as emotion, accent, rhythm, pauses and intonation. These elements are crucial for making speech sound real and creating interesting conversations. It helps avoid the boring sound you often get with regular text-to-speech.
How the Voice-Cloning AI Model Works
In a research paper, OpenVoice shared the methodology behind its voice-cloning AI. OpenVoice is composed of two distinct AI models: a text-to-speech (TTS) model and a “tone converter.”
The model can manage style parameters and languages, and has undergone “training using 30,000 sentences” from English (in both American and British accents), Chinese and Japanese speakers. The training involved labeling the samples based on the expressed emotions, and the model learned intonation, rhythm, and pauses from these audio clips.
On the other hand, the tone converter model was trained on a vast dataset of over 300,000 audio samples from more than 20,000 different speakers. In both cases, human speech audio was converted into phonemes – specific sounds that differentiate words – and represented using vector embeddings.
The TTS model, using a “base speaker,” combines with the tone derived from a user’s recorded audio in the training process. Together, these two models can replicate the user’s voice and modify the tone color – the emotional expression conveyed in the spoken text.
The startup was founded in 2023. Last year, MyShell raised $5.6 million in seed funding, led by INCE Capital, and saw the participation of prominent investors such as Folius Ventures, Hashkey Capital, SevenX Ventures, TSVC and OP Crypto, among others.
According to the company, the funding will help in the advancement of proprietary AI models, the creation of a Creator Studio tailored for AI-native apps, and the establishment of a vibrant creator ecosystem within the realm of blockchain technology.
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.
More articles
Kumar is an experienced Tech Journalist with a specialization in the dynamic intersections of AI/ML, marketing technology, and emerging fields such as crypto, blockchain, and NFTs. With over 3 years of experience in the industry, Kumar has established a proven track record in crafting compelling narratives, conducting insightful interviews, and delivering comprehensive insights. Kumar's expertise lies in producing high-impact content, including articles, reports, and research publications for prominent industry platforms. With a unique skill set that combines technical knowledge and storytelling, Kumar excels at communicating complex technological concepts to diverse audiences in a clear and engaging manner.
