Meet video inpainting: text-driven editing with Stable Diffusion and Neural Atlases

In Brief

AI leads to new video editing tools for video editors – here’s how

The Trust Project is a worldwide group of news organizations working to establish transparency standards.

Artificial intelligence is changing the face of video editing. New tools powered by AI are making it possible for video editors to work faster and smarter. One of the most promising applications of AI in video editing is automatic video editing. This is where the AI system analyzes the video footage and makes edits accordingly. This can be a huge time-saver for video editors, who often have to go through hours of footage to find the best shots.

Meet video inpainting: text-driven editing via Neural Atlases and Stable Diffusion

Symbiosis of two AI models, Neural Atlases and Stable Diffusion, led to the creation of new software for video editors that lets you edit and change objects in the video with text prompts. It’s remarkable how new tools — like the “make pretty” button — that were previously only imaginable are now created from various AI suits.

It provides a technique for adding text to naturally occurring photos and videos to alter their appearance. The aim is to add new visuals to the scene (like smoke or fire) or alter the appearance of existing things (like object texture) in a semantically meaningful way, given an input image or video and a target text cue. Such an approach uses an externally pre-trained CLIP model to calculate the loss while training the generator using an internal collection of training samples produced from a single input (a picture or video and a target text prompt). The main concept is to produce a changeable layer (color and transparency) on top of the original input rather than directly generating an edited outcome.

This is undoubtedly the last day for video composers; have a look at the description. Here, they not only create fresh content using masks but also skillfully and transparently compose it.

Neither a pre-trained generator nor user-supplied editing masks are necessary for this method. As a result, it can make targeted semantic modifications to high-resolution videos and natural photographs of a variety of objects and scenarios.

More video examples are provided below:

With this new tool, video editors have countless creative options for making unique videos. All of these new AI-powered tools are making video editing easier and more efficient. As AI continues to develop, we can expect to see even more new and exciting video editing tools in the future.

Read more:


Any data, text, or other content on this page is provided as general market information and not as investment advice. Past performance is not necessarily an indicator of future results.

Damir Yalalov

Damir is the Editor/SEO/Product Lead at He is most interested in SecureTech, Blockchain, and FinTech startups. Damir earned a bachelor's degree in physics.

Follow Author

More Articles