News Report Technology
November 23, 2022

Music-to-Dance: EDGE AI generates an infinite number of dance ideas for TikTok based on audio input

In Brief

EDGE: The AI Generated Choreography Software That’s Changing the Dance World

Stanford University has announced a powerful music-to-dance AI that generates dance from audio input called EDGE. EDGE is a potent method for customizable dance creation that can produce physically believable, realistic dances while adhering to any supplied music. With the help of Jukebox, a potent music feature extractor, and transformer-based diffusion model, EDGE offers powerful editing capabilities that are ideal for dance, such as joint-wise conditioning, motion in-betweening, and dance continuation. It discovers that human raters greatly favor dances generated by EDGE when compared to recent approaches like Bailando and FACT.

Music-to-Dance: EDGE AI generates an infinite number of dance ideas for TikTok based on audio input

From Music to Choreography: How EDGE Generates Its Dances

Even for music samples found in the wild, EDGE can produce high-quality dances using music embeddings from the robust Jukebox model. A frozen Jukebox model is used by EDGE to incorporate input music. A series of five-second dance clips are used to teach a conditional diffusion model how to map the music embedded into the clips. Before joining batches of multiple clips together to create an arbitrary-length full video, temporal constraints are applied to ensure temporal consistency.

From Music to Choreography: How EDGE Generates Its Dances
EDGE is trained on 5-second dance clips, but by placing temporal constraints on batches of sequences, it is capable of creating dances of any length. In the illustration, EDGE forces each sequence’s first half to match the second half of the one before it.

Freshly generated music-to-dance outputs

Arbitrary temporal and spatial limitations are supported by EDGE. There are numerous end-user applications that can be supported by this, including:

  • By requiring temporal continuity across batches of numerous sequences, arbitrary-length dances can be produced.
  • Dances that include joint restrictions, such as lower body motion causing upper body generation, or the opposite.
  • Dances with beginning and ending moves that are predetermined.
  • Dances that begin with a predetermined motion are known as dance continuations.
Complex, deliberate, sliding foot-ground contact is prevalent in dance. New Contact Consistency Loss, which significantly enhances physical realism while maintaining sliding, enables EDGE to learn when feet should and shouldn’t slide.

Researchers found that human observers overwhelmingly preferred dances generated by the EDGE algorithm to those generated by other algorithms. The study’s authors say that the EDGE algorithm represents a major advance in the field of computer-generated dance. They believe that the algorithm could be used to create dances for real-world and virtual events, such as social and TikTok trends, weddings, parties, or corporate functions.

Read more about AI:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

More articles
Damir Yalalov
Damir Yalalov

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

From Ripple to The Big Green DAO: How Cryptocurrency Projects Contribute to Charity

Let's explore initiatives harnessing the potential of digital currencies for charitable causes.

Know More

AlphaFold 3, Med-Gemini, and others: The Way AI Transforms Healthcare in 2024

AI manifests in various ways in healthcare, from uncovering new genetic correlations to empowering robotic surgical systems ...

Know More
Join Our Innovative Tech Community
Read More
Read more
Crypto Exchange Jupiter To Introduce Jupiter Swap V3, Dynamic Slippage, Among Other Key Updates In Coming Weeks
Markets News Report Technology
Crypto Exchange Jupiter To Introduce Jupiter Swap V3, Dynamic Slippage, Among Other Key Updates In Coming Weeks
June 14, 2024
Polygon Debuts Its Governance Hub, Providing Unified Transparent Interface For Community Governance 
News Report Technology
Polygon Debuts Its Governance Hub, Providing Unified Transparent Interface For Community Governance 
June 14, 2024
Binance Registers Over 30M New Users In 2024, Customer Assets Surpass $100B Milestone
Markets News Report Technology
Binance Registers Over 30M New Users In 2024, Customer Assets Surpass $100B Milestone
June 14, 2024
Amazon Supports Generative AI Startups With $230M Investment, Allocates $80M To Its Second AWS Generative AI Accelerator Program
Business News Report Technology
Amazon Supports Generative AI Startups With $230M Investment, Allocates $80M To Its Second AWS Generative AI Accelerator Program
June 14, 2024