AI has impacted every area of human knowledge. One promising area experiencing the impact of generative AI is filmmaking. Streaming giants like Netflix, Amazon Prime, Disney+ Hotstar, and YouTube are pioneers in extending the technology across their operations.
They are not only focusing on reducing production costs but also on bringing a personalised experience to shows and films, thereby enhancing their quality in unprecedented ways. As of now, the focus on generative AI in filmmaking primarily revolves around text-to-video generation tools.
The development of AI text-to-video generation tools has sparked tough competition among tech giants. OpenAI’s Sora and DALL-E, Microsoft’s VASA-1, Adobe’s Firefly, and Google’s Veo, as well as independent platforms such as Stable Video, Midjourney, Runway, and Pika Labs, are all vying for the top spot in this rapidly evolving field.
The Rise of Runway
AI video startup Runway has made its mark in the industry as one of the most popular and state-of-the-art tools for AI video generation.
Its recent launch of advanced camera control for Gen-3 Alpha Turbo on November 1 further solidified its position in the industry. This pushed their presence in the text-to-video sector to a whole new level.
Advanced Camera Control is now available for Gen-3 Alpha Turbo. Choose both the direction and intensity of how you move through your scenes for even more intention in every shot.
(1/8) pic.twitter.com/jRE6pC9ULn— Runway (@runwayml) November 1, 2024
That’s not all. In September, Runway made history by collaborating with Lionsgate, the maker of John Wick. This deal set a new benchmark in the entertainment industry, pushing it far ahead of OpenAI’s Sora. Under this deal, Lionsgate could use Runway’s AI tools exclusively to create content.
Interestingly, this deal highlighted the use of Runway in the Oscar-winning movie ‘Everything Everywhere All at Once’ for special effects, which saved a lot of time, redacted costs, and minimised manual effort.
Runway also created updates that completely changed production artists’ reliance on traditional motion capture systems. In October, AIM reported that Runway announced Act-One, a new state-of-the-art tool for generating expressive character performances, inside Gen-3 Alpha.
But, Others are Not Backing Down
Another tool rising to fame is Midjourney. In September this year, it made a mark by releasing its Niji feature. Many users then took to the internet and shared their personalised artworks on the platform, expressing appreciation for the feature.
A user on X also recently made a bold claim that 99% of AI videos made with tools like Luma, Pika, and Runway are generated from images originally created in Midjourney. In the same month, AIM also speculated why artists prefer Midjourney over FLUX.
But is Midjourney just an image-to-video generation tool, or is it a jack of all trades and master of none? AIM speculated this by inquiring into why Midjourney is entering the hardware business.
Midjourney to Sora
The launch of OpenAI’s Sora has been widely compared to a ‘ChatGPT moment in video generation’. The model was a new, intelligent text-to-video tool that could create videos of up to 60 seconds featuring detailed scenes, complex camera motion, and multiple characters displaying vibrant emotions.
India also entered this domain this year and developed tools and solutions for enterprises to create AI videos. Phenomenal AI, an Indian startup, launched the country’s first text-to-video AI platform in July.
As earlier reported by AIM, Bengaluru-based AI startup Unscript is another platform that allows users to create professional-looking videos. Unscript stands to compete with OpenAI’s Sora as a major competitor. Another Indian AI video creation platform Rephrase.ai was recently acquired by Adobe.
Not Just One Alternative to Sora but More!
Amid rising AI video tools like Sora, China has emerged as a major competitor, surpassing the capabilities of several existing platforms.
Chinese TikTok competitor company Kuaishou’s Kling, a powerful AI video tool launched this year, is popularly regarded as an alternative to Sora.
This model creates large-scale realistic motions that simulate physical world characteristics. It also delves into producing 3D face and body reconstruction backed by the company’s proprietary technology. A Spanish AI content creator, MayorkingAI, recently mentioned on X that Kling’s dynamism and VFX “are wild.”
Another X user Déborah,’ an AI video generation enthusiast, expressed her confusion about what’s real and what’s not when creating media using Kling.
What is real, and what is not?@Kling_ai animation pic.twitter.com/OLRnV6KspB
— Déborah (@dvorahfr) November 7, 2024
Kling has also made it possible for users to train their own AI video characters. “While the US is debating heavily on AI ethics and incorporating a responsible AI, China seems unperturbed and is probably responding to these AI ethicists with Kling,” AIM earlier contemplated, soon after Kling’s release.
MiniMax, another text-to-video generator launched by a Chinese startup, has recently been recognised by AIM for some of its best AI-generated videos. Speaking on the rise of such Chinese tools, a Reddit user said that China is more advanced in AI than many of us have been led to believe, based on an article on the AI research boom.
Regardless, even though systems like Sora and Kling have showcased impressive capabilities, they remain accessible only to select users.
Similarly, Luma’s Dream Machine boasts remarkable features but is limited to a restricted audience. Regardless, Dream Machine 1.5 has created some of the most famous AI videos on the internet.
A new competitor to Kling and Sora has arrived. Pollo AI is a platform that strives to democratise AI video generation. It was developed by HIX.AI, a Singapore-based all-in-one AI solution provider.
Rishabh, the co-founder of AI Insights, recently took to X to show videos generated by Pollo AI. He said, “Sora and Kling AI are in MASSIVE trouble.”
Sora and Kling AI are in MASSIVE trouble.
Pollo AI just dropped, and it's mind-blowing.
Here are 10 wild examples: pic.twitter.com/RZXiRal6CW— Rishabh (@Rixhabh__) November 5, 2024
In the future, text-to-video generation tools will also have visible impacts on the creation of video games. In 2025, text-to-video games will likely erase traditional gaming forever. With the blend of AI in such domains, it’s only a matter of time before we can play our movies like games and watch our games like movies, limited only by imagination.
The post Lights, Camera, AI! The Future of Filmmaking is on the Runway appeared first on Analytics India Magazine.
