Creating video games from simple text prompts has now become a reality. Recently, Decart AI launched Oasis, the world’s first real-time, generative AI-based playable world model. This fully interactive game generates each frame through a Transformer model that responds instantly to keyboard and mouse inputs, simulating physics, game mechanics, and graphics.
The company has also open-sourced the model’s weights and revealed that Oasis v1 surpassed 1 million unique users within three days and seven hours of its launch. The Israel-based AI company also introduced a new feature called ‘Custom World’, which allows users to add their own starting image to Oasis and have the AI transform it into a unique, playable world.
1/ We are excited to introduce Oasis, the world's first real-time AI world model, developed in collaboration with @Etched. Imagine a video game entirely generated by AI, or a video you can interact with—constantly rendered at 20 fps, in real-time, with zero latency pic.twitter.com/WAJFRyfTzS
— Decart (@DecartAI) October 31, 2024
The model has learned actions like moving, jumping, picking up items, and breaking blocks by observing gameplay. Gamers can use text or voice commands to guide their play. One interesting thing about the game is that it can run on Etched’s AI chip Sohu, which is built to handle massive video models, running in 4K for 100B+ parameter models.
“This is the most absurdly impressive use of AI by far. You play Minecraft in real time WITHOUT programming or a game engine! Every frame is approximated via the AI – it works in your phone browser free. I haven’t been this excited about something in ages,” posted a user on X.
Similarly, the Hong Kong University of Science and Technology, the University of Science and Technology of China, and the Chinese University of Hong Kong have jointly launched GameGen-X, a diffusion Transformer model purpose-built for generating and interactively managing open-world game videos.
By simulating a variety of game engine features, this model delivers high-quality, open-domain generation with elements like creative characters, dynamic environments, intricate actions, and varied events.
What we’ve seen so far is just the beginning. Google DeepMind recently introduced GameNGen, the world’s first game engine driven entirely by neural models capable of simulating complex video games like DOOM in real-time without traditional coding.
It uses a neural network to generate game frames at over 20 frames per second, producing visuals that are nearly indistinguishable from the original game.
Similarly, Tencent introduced GameGen-O, an AI model capable of generating open-world video games from simple text prompts. Trained on 4,000 hours of video game footage, it can autonomously create characters, environments, actions, and events.
Threat to Gaming Companies?
With generative AI now easily capable of creating 3D worlds, gaming engines like Unity, Unreal, Roblox, and Godot could soon face some serious competition. This has pushed gaming companies and media houses to take generative AI seriously.
Netflix recently appointed Mike Verdu as the VP of GenAI for games. “I am working on driving a once-in-a-generation inflexion point for game development and player experiences using generative AI. This transformational technology will accelerate the velocity of development and unlock truly novel game experiences that will surprise, delight, and inspire players,” said Verdu in a LinkedIn post.
A month ago, Electronic Arts (EA) chief strategy officer Mihir Vaidya demoed the ‘text-to-game’ concept. He said that two friends can create an open sandbox in a matter of seconds without coding. “They then quickly shifted to remixing existing elements from their favourite EA Games asset libraries and community elements like characters, weapons, gameplay systems, and logic,” he said.
Jon Lai, partner at Andreessen Horowitz, believes that interactive video games is an exciting space as it blends the best of films and games. He cited an example, explaining how generative AI companies could team up with studios like Disney to turn their movies into games. “The next Pixar will be an AI company built on interactive video – a blend of film & games where frames are generated entirely by a neural net in real-time, with no code, assets, or game engines,” he said.
Besides generative AI games, recently, several video models have also made a splash. These include Runway, Sora (OpenAI), Lumiere (Google), Veo (Google), Pika 1.0, Meta’s Emu Video, Adobe Firefly Video Model, VideoPoet (Google), and MovieGen (Meta).
Got Enough Hardware?
Generative AI games, however, demand substantial computing power and hardware resources. According to Lai, with the ongoing hardware and model improvements, one can expect fully generative interactive video to reach commercial viability in roughlytwo years.
For instance, OpenAI has not released Sora to the public yet since they have allocated their compute resources towards training the next frontier model. In a recent AMA on Reddit, CEO Sam Altman said that AGI is achievable with current hardware, but the company is facing tough decisions regarding compute allocation and has chosen to prioritise training o1.
Perhaps, after the launch of Sora and o1, OpenAI just might announce a text-to-video game model next year. Interestingly, the company is also planning to release AI agents next year, for which it has developed an innovative training approach called Video PreTraining (VPT) to enable agents to play Minecraft.
Last year, the Altman-led company acquired Global Illumination, the organisation behind an open-source sandbox MMORPG built for the web, allowing users to build, forage, and play mini-games directly from the browser, much like Microsoft’s Minecraft.
Meanwhile, NVIDIA, originally known for its 3D graphics in video games, is now returning to its roots. The GPU-king recently announced NVIDIA ACE, a suite for creating digital humans with generative AI, now including the on-device Nemotron-4 4B Instruct model powered by RTX AI. This model improves role-play, retrieval-augmented generation, and function-calling, allowing game characters to better understand and respond to player instructions.
‘Mecha BREAK’ from Amazing Seasun Games is the first game to use ACE technology.
At the Consumer Electronics Show in Las Vegas earlier this year, NVIDIA unveiled its latest GeForce RTX 40 SUPER Series, featuring the RTX 4080 SUPER, 4070 Ti SUPER, and 4070 SUPER models. These GPUs are built specifically to handle “supercharged generative AI workloads” on PCs and laptops.
With AI now coding the future of gaming, it’s only a matter of time before we play in worlds limited only by imagination.
The post 2025 Will See Text-to-Video Games Erase Traditional Gaming Forever appeared first on Analytics India Magazine.
