Adobe’s New Tool Transforms Text into Editable Videos

MetaversePlanet September 12, 2024

0 1 minute read

Adobe has unveiled a groundbreaking tool that allows users to create videos from text descriptions, offering the ability to modify camera angles, movements, shooting distances, and more through simple commands.

Earlier this year, Adobe introduced the Firefly video model, designed to enhance video and audio editing with AI support within Creative Cloud applications. Now, Adobe has released a video showcasing the next generation of AI-powered video tools built on Firefly, which can generate video clips from still images.

This teaser highlighted Firefly’s capability to produce videos from text inputs, a feature previously announced but not yet demonstrated. With this tool, users can create video clips based on textual descriptions and fine-tune them using a variety of “camera controls,” simulating angles, movements, and distances. Additionally, it can generate videos from reference images.

Here’s the introductory video

When watching the video, it is notable that the quality of the generated videos closely resembles OpenAI’s Sora model. Adobe has previously explored integrating this model with Premiere Pro for third-party use. However, at present, Adobe’s tool is only capable of producing 5-second clips.

One area where Firefly may have an edge over Sora is emphasized in the promo video: Adobe claims that Firefly is “commercially safe” to use. The company reportedly trained the model using images that are either openly licensed, in the public domain, or sourced from Adobe Stock, effectively avoiding copyright concerns.

Adobe has announced that the new Firefly video model will be integrated into Creative Cloud, Experience Cloud, and Adobe Express applications. Additionally, the Generative Extend feature, which allows the expansion of visible areas in videos—similar to the Generative Expand tool used for backgrounds in Photoshop—will also be introduced.