News Report
October 03, 2022

StabilityAI announced AI Music Generator Harmonai based on Dance Diffusion Model

In Brief

Harmonai brings open-source audio models to help shape the future of music

Dance Diffusion aims to make music production more approachable and enjoyable for everyone

With Harmonai, a cleverly called new method for, you guessed it, creating an audio out of nothing, StabilityAI’s researchers have made a big advancement in the field of AI audio-generation. The outcomes are astounding and diverse, and they are all just a little bit unsettling.

StabilityAI announced AI Music Generator Harmonai based on Dance Diffusion Model
Image generated by Stable Diffusion

What is Harmonai?

Harmonai is a community-driven organization that publishes open-source generative audio tools to increase everyone’s access to and enjoyment from music composition. To put it simply, it generates never-before-heard sounds in a process called diffusion, in which random white noise is refined into sounds based on a pre-trained model that contains millions of parameters that were calculated during its training process to encapsulate the characteristics of the source material it was trained on.

The goal of the deep learning research lab Harmonai is to produce open-source generative audio models, support researchers and developers through compute funding and a community and offer new creative tools to artists and audio professionals. The goal of Harmonai is to make music production more approachable and enjoyable for everyone by developing open-source audio synthesis and manipulation tools.

What’s Dance Diffusion?

Imagine Stable Diffusion only producing sound instead of images. Dance Diffusion is a collection of publicly available generative music diffusion models. The initial models generate 1-3 seconds of audio and can also be used to interpolate and style transfer lengthier audio recordings.

Is it therefore similar to text-to-image AI for audio?

In spirit, yet with some significant variances. There isn’t a text entry form where you can type “snare drum” and press GO, thus this isn’t a text-to-audio or text-to-speech generator. Never say never, of course—this project is still in its early stages—but for now, it’s up to you, the user, to supply a pool of audio files (be they snare drums or whatever it is you wish to generate variations of) to train the model on. Alternatively, you can use one of the 6 pre-trained models already present in the Dance Diffusion notebook to get you started—and there’s plenty of fun to be had playing with them alone!

Yep. Harmonai is Dadabots + more amazing people + 4000 GPUs. In their YouTube channel, Dadabots’ artists already use AI generated music:

Freshly generated AI audio samples by Harmonai

Follow the community discord channel to listen to more audio files.

Interested in reading more? Here is some additional news to check out:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

More articles
Damir Yalalov
Damir Yalalov

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

The Calm Before The Solana Storm: What Charts, Whales, And On-Chain Signals Are Saying Now

Solana has demonstrated strong performance, driven by increasing adoption, institutional interest, and key partnerships, while facing potential ...

Know More

Crypto In April 2025: Key Trends, Shifts, And What Comes Next

In April 2025, the crypto space focused on strengthening core infrastructure, with Ethereum preparing for the Pectra ...

Know More
Read More
Read more
BNB Chain Rolls Out Model Context Protocol To Enable AI Integration Across Its Ecosystem
News Report Technology
BNB Chain Rolls Out Model Context Protocol To Enable AI Integration Across Its Ecosystem
May 6, 2025
DeFi Without Borders: How Interoperability Is Shaping Next-Gen Finance
Hack Seasons News Report Technology
DeFi Without Borders: How Interoperability Is Shaping Next-Gen Finance
May 6, 2025
Crypto’s Crossroads: What’s Next for The Market? The Hottest Insights From Industry Opinion Leaders
News Report Technology
Crypto’s Crossroads: What’s Next for The Market? The Hottest Insights From Industry Opinion Leaders
May 5, 2025
Hack Seasons Conference Concludes In Dubai: A Landmark Event Uniting Web3 Pioneers To Explore The Future Of DeFi, AI, And Blockchain Innovation
Hack Seasons Business Lifestyle Markets News Report Technology
Hack Seasons Conference Concludes In Dubai: A Landmark Event Uniting Web3 Pioneers To Explore The Future Of DeFi, AI, And Blockchain Innovation
May 5, 2025