News Report Technology
December 16, 2022

Stable Diffusion can create new music by generating spectrograms based on text

In Brief

Riffusion is a real-time music generation app that uses stable diffusion

Since the early days of AI, scientists have been trying to employ it to generate new and interesting music. The team behind the Riffusion project has found a very original use of AI for image generation in music production. They trained the open Stable Diffusion model on spectrogram images depicting the frequency and amplitude of a sound wave over time, as well as a text description. As a result, the AI can generate new spectrograms based on your text requests, and when you play them, music is played.

Stable Diffusion can create new music by generating spectrograms based on text
AI can generate new music by modifying spectrograms in response to your requests.

Similar to image modification in Stable Diffusion, the method can be used to change existing sound compositions and sample music synthesis. You can also combine different styles, make a smooth transition from one style to another, or modify an existing sound to solve problems like increasing the volume of individual instruments, changing the rhythm, and replacing instruments.

The Stable Diffusion algorithm is already showing a lot of promise for music generation. And, since it is open source and licensed under the MIT license, anyone can use it to create their own music. On the project website, you can listen to samples of generated music.

Listen to these freshly generated music examples by Stable Diffusion:

Read more about music and AI:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

More articles
Damir Yalalov
Damir Yalalov

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

Hot Stories
Join Our Newsletter.
Latest News

The Calm Before The Solana Storm: What Charts, Whales, And On-Chain Signals Are Saying Now

Solana has demonstrated strong performance, driven by increasing adoption, institutional interest, and key partnerships, while facing potential ...

Know More

Crypto In April 2025: Key Trends, Shifts, And What Comes Next

In April 2025, the crypto space focused on strengthening core infrastructure, with Ethereum preparing for the Pectra ...

Know More
Read More
Read more
Web3 On-Chain Data Insights: In April, Solana Tops Activity, Ethereum Sees Capital Inflows, Bitcoin Shows Structural Rebound
News Report Technology
Web3 On-Chain Data Insights: In April, Solana Tops Activity, Ethereum Sees Capital Inflows, Bitcoin Shows Structural Rebound
May 9, 2025
Exploring AI Revolution In Web3: Decentralized AI, Data Ownership, And The Road Ahead
Hack Seasons News Report Technology
Exploring AI Revolution In Web3: Decentralized AI, Data Ownership, And The Road Ahead
May 9, 2025
The Future Of AI Agents: Innovation, Challenges, And Opportunities
Hack Seasons News Report Technology
The Future Of AI Agents: Innovation, Challenges, And Opportunities
May 9, 2025
Adidas, Binance, And Coinbase: Major Crypto Partnerships Of May 2025
News Report Technology
Adidas, Binance, And Coinbase: Major Crypto Partnerships Of May 2025
May 9, 2025