Artificial Intelligence | February 9, 2024
Google has introduced what could be one of the most advanced video generators to date: it’s called Lumiere and allows you to create content from a text description or an image. But unlike existing artificial intelligence models, the company emphasizes that theirs creates “realistic, diverse, and coherent” movements. And in just seconds, of course.
Google had previously introduced other video generation tools, but more limited than Lumiere. The new technology uses a model called Space-Time-U-Net (STUNet), which determines where objects or elements are in a video, how they move, and change simultaneously. In addition to generating videos and animated images, it can copy artistic styles and even allows editing the created clips.
“We demonstrate cutting-edge generation results and show the applicability of our approach for a wide range of implementations,” say the developers in the research report. The model was trained with over 30 million videos, along with their subtitles. Californians, like other developers, did not disclose the source of this data.
Lumiere is capable of generating 80 frames at 16 frames per second, Google said. This allows videos of up to five seconds in duration, with a resolution of 1024 x 1024 pixels. In addition, according to a study conducted with users, its results were preferred over existing artificial intelligence video models.
Lumiere is not yet available for testing, and it is not known when or if Google will ever release it to the public. But the company was kind enough to leave us some samples in a promotional video. In some cases, if you pay attention, you can see some clues on the faces of animals that reveal the use of artificial intelligence. But there are other scenes, like a turtle swimming, that are impressively realistic.
Google also demonstrates Lumiere’s ability to create delirious scenes, like a panda driving a car or a cat playing the piano. It also boasts its tool by giving movement to famous paintings like the Mona Lisa.
Like most options so far, Google Lumiere is not designed to generate videos that involve multiple shots or transitions between scenes. “Generating such content remains an open challenge for future research,” the authors acknowledge.
Google’s developers included a warning about how Lumiere could fuel the spread of deepfakes. “There is a risk of misuse by creating false or harmful content with our technology,” they say in the report. “We believe it is crucial to develop and apply tools to detect biases and malicious use cases to ensure a safe and fair experience.”
The creation of forged content sparked a major controversy last week when fake and sexually explicit images of Taylor Swift were circulated. Even the White House commented on it. “It’s alarming,” said Jean-Pierre, a spokesperson for President Joe Biden’s office. The government “is committed to ensuring that we reduce the risk of AI-generated fake images through executive actions. The work to find real solutions will continue,” he wrote on X (formerly Twitter).
