Music-to-Dance: EDGE AI generates an infinite number of dance ideas for TikTok based on audio input

News Report Technology

In Brief

EDGE: The AI Generated Choreography Software That’s Changing the Dance World


The Trust Project is a worldwide group of news organizations working to establish transparency standards.

Stanford University has announced a powerful music-to-dance AI that generates dance from audio input called EDGE. EDGE is a potent method for customizable dance creation that can produce physically believable, realistic dances while adhering to any supplied music. With the help of Jukebox, a potent music feature extractor, and transformer-based diffusion model, EDGE offers powerful editing capabilities that are ideal for dance, such as joint-wise conditioning, motion in-betweening, and dance continuation. It discovers that human raters greatly favor dances generated by EDGE when compared to recent approaches like Bailando and FACT.

Music-to-Dance: EDGE AI generates an infinite number of dance ideas for TikTok based on audio input

From Music to Choreography: How EDGE Generates Its Dances

Even for music samples found in the wild, EDGE can produce high-quality dances using music embeddings from the robust Jukebox model. A frozen Jukebox model is used by EDGE to incorporate input music. A series of five-second dance clips are used to teach a conditional diffusion model how to map the music embedded into the clips. Before joining batches of multiple clips together to create an arbitrary-length full video, temporal constraints are applied to ensure temporal consistency.

From Music to Choreography: How EDGE Generates Its Dances
EDGE is trained on 5-second dance clips, but by placing temporal constraints on batches of sequences, it is capable of creating dances of any length. In the illustration, EDGE forces each sequence’s first half to match the second half of the one before it.

Freshly generated music-to-dance outputs

Arbitrary temporal and spatial limitations are supported by EDGE. There are numerous end-user applications that can be supported by this, including:

  • By requiring temporal continuity across batches of numerous sequences, arbitrary-length dances can be produced.
  • Dances that include joint restrictions, such as lower body motion causing upper body generation, or the opposite.
  • Dances with beginning and ending moves that are predetermined.
  • Dances that begin with a predetermined motion are known as dance continuations.
Complex, deliberate, sliding foot-ground contact is prevalent in dance. New Contact Consistency Loss, which significantly enhances physical realism while maintaining sliding, enables EDGE to learn when feet should and shouldn’t slide.

Researchers found that human observers overwhelmingly preferred dances generated by the EDGE algorithm to those generated by other algorithms. The study’s authors say that the EDGE algorithm represents a major advance in the field of computer-generated dance. They believe that the algorithm could be used to create dances for real-world and virtual events, such as social and TikTok trends, weddings, parties, or corporate functions.

Read more about AI:

Disclaimer

Any data, text, or other content on this page is provided as general market information and not as investment advice. Past performance is not necessarily an indicator of future results.

Damir Yalalov

Damir is the Editor/SEO/Product Lead at mpost.io. He is most interested in SecureTech, Blockchain, and FinTech startups. Damir earned a bachelor's degree in physics.

Follow Author

More Articles