News Report Technology
February 08, 2023

Gen-1: AI Generates New Videos From Existing Ones by Combining Prompts and Images

RunWayML, an artificial intelligence startup, has announced a new product called Gen-1, a neural network that can generate new videos from existing ones by combining prompts and images. For years, neural networks have been limited to the task of style transfer, which is the process of taking an image and applying the style of another image to it. This is how we get those trippy deep-learning style transfer videos where, for example, a landscape is rendered in the style of Van Gogh’s Starry Night.

Gen-1: AI generates new videos from existing ones by combining prompts and images
@Midjourney / That Weird Ghost

Now, with Gen-1, neural networks can do more than just style transfer. It can generate new videos from scratch, using prompts and images as input. This opens up a whole new range of possibilities for AI-created videos. Right now, the videos generated by Gen-1 are short and simple. But as the technology develops, we can expect to see more complex and realistic videos being generated by AI.

Recommended post: Men’s reproductive health is improved by watching adult videos in virtual reality

What is Gen-1?

Strong picture creation and editing tools are unlocked by text-guided generative diffusion models. While these have been applied to the creation of videos, the present methods for editing the content of already-existing material while maintaining its structure demand expensive retraining for each input or depend on the risky propagation of picture alterations between frames.

Developers describe a model for structure- and content-guided video diffusion, which alters films based on either written or visual descriptions of the desired result. Due to insufficient decoupling, conflicts between user-provided content modifications and structure representations arise. They demonstrate that the structure and content integrity may be controlled by training on monocular depth estimations with various levels of detail.

Synthesize new videos in a realistic and consistent manner by applying the composition and style of an image or text prompt to the structure of your source video. It’s like filming something new without actually filming anything.

 Synthesizing a new video using this method enables filmmakers to quickly produce content in a cost-effective manner
Synthesizing a new video using this method enables filmmakers to quickly produce content in a cost-effective manner

Transferring the style of an image or prompt to every frame of a video can be a great way to unify your project, give it an overarching visual theme, and create consistency.

Through the use of software and creative design, mockups can be transformed into aesthetically pleasing and interactive renders that bring the user’s vision to life.

 Mockups allow designers to visualize their ideas and create tangible results that can be presented for review
Mockups allow designers to visualize their ideas and create tangible results that can be presented for review

With video editing features, it’s easy to isolate parts of a video and enhance them with text prompts.

Applying an input image or prompt to untextured renders can greatly improve their realism, bringing the 3D models to life.

Runway Research is dedicated to building multimodal AI systems that enable new forms of creativity. Gen-1 represents yet another of our pivotal steps forward in this mission. If you’re a creative looking to experiment with the future of storytelling, request access below.

Gen-1 can also be used to create entirely new videos from scratch. It can be used for a number of purposes, including creating new versions of existing films or creating new films altogether. This news is sure to excite filmmakers and other creatives who are always looking for new ways to push the boundaries of their art. With Gen-1, they now have a new powerful tool at their disposal.

The ability to generate new videos from existing ones has a number of potential applications. For example, it could be used to create new versions of existing video content, so it could be used for creating alternative versions of scenes or shots that may be too expensive or logistically difficult to film.

You can apply to use the model while it is in beta testing here.

Read more about AI:

Disclaimer

In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.

About The Author

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

More articles
Damir Yalalov
Damir Yalalov

Damir is the team leader, product manager, and editor at Metaverse Post, covering topics such as AI/ML, AGI, LLMs, Metaverse, and Web3-related fields. His articles attract a massive audience of over a million users every month. He appears to be an expert with 10 years of experience in SEO and digital marketing. Damir has been mentioned in Mashable, Wired, Cointelegraph, The New Yorker, Inside.com, Entrepreneur, BeInCrypto, and other publications. He travels between the UAE, Turkey, Russia, and the CIS as a digital nomad. Damir earned a bachelor's degree in physics, which he believes has given him the critical thinking skills needed to be successful in the ever-changing landscape of the internet. 

Hot Stories
Join Our Newsletter.
Latest News

Supply and Demand Zones

Cryptocurrency, like any other currency, is a financial instrument based on the fundamental economic principles of supply ...

Know More

Top 10 Crypto Wallets in 2024

With the current fast-growing crypto market, the significance of reliable and secure wallet solutions cannot be emphasized ...

Know More
Join Our Innovative Tech Community
Read More
Read more
Binance Unveils Launchpool Project Ethena for BNB, FDUSD and TUSD Staking
News Report Technology
Binance Unveils Launchpool Project Ethena for BNB, FDUSD and TUSD Staking
March 29, 2024
Parcl Unveils Tokenomics Ahead of PRCL Launch, Allocates 75M Tokens to Point Holders
News Report Technology
Parcl Unveils Tokenomics Ahead of PRCL Launch, Allocates 75M Tokens to Point Holders
March 28, 2024
Circle to Introduce Native USDC On zkSync Network for Enhanced Liquidity
News Report Technology
Circle to Introduce Native USDC On zkSync Network for Enhanced Liquidity
March 28, 2024
GoPlus Report Unveils How Utilization of API Security Data by Blockchain May Communities May Aid in Addressing Web3 Threats
News Report Technology
GoPlus Report Unveils How Utilization of API Security Data by Blockchain May Communities May Aid in Addressing Web3 Threats
March 28, 2024
What You
Need to Know

Subscribe To Our Newsletter.
Daily search marketing tidbits for savvy pros.