Google DeepMind’s new artificial intelligence tool will create music for both AI-generated and traditional videos.
Google’s DeepMind artificial intelligence lab is preparing to solve the problem of creating background music and even dialogue, one of the biggest shortcomings of video-creating AI, which has been popular recently. Sharing its advances in what it calls V2A (audio beyond video) technology, the lab can also pair this tool with video creation tools such as Google Veo and OpenAI Sora.
According to a blog post shared by the DeepMind team, the system can understand raw pixels and combine this information with text prompts. Sound effects are also created through this match. This tool can also be used for silent movies or other videos that don’t have sound.
The DeepMind team relies on their tools
In fact, DeepMind’s technology isn’t the first AI to be used to create sound, and it won’t be the last. ElevenLabs has also released such a tool before. However, the DeepMind team says their tool is “different from existing ultra-video audio solutions in that it can understand pixels, and adding text prompts is optional.”
To develop this technology, DeepMind trained researchers with AI-generated data, including videos, audios, detailed audio descriptions, and transcripts. The researchers also note that V2A technology addresses existing problems, such as a decrease in the audio quality of the output when there are distortions in the source video. The DeepMind team also pledged to “put the technology through rigorous security assessments and testing” before making it public.
You may also like this content
- Meta Building World’s Fastest AI Supercomputer for Metaverse
- Artificial Intelligence Will Make Decisions Instead Of People
- Bill Gates: Artificial Intelligence Over Web3 and Metaverse
-
-
-
-
-
-
-
-
-
-
