While OpenAI basks in the glory of the o1 launch, Google is quietly working on a model that could surpass it. Since last December, when Google launched Gemini, the two companies have been locked in a competitive battle to outperform each other.
AI insider Jimmy Apples recently posted a video on X, suggesting that Google is soon releasing something better, featuring the message, ‘Patience, they are coming.’ Even Logan Kilpatrick, Google’s lead product manager for AI Studio and the Gemini API, shared a post which read ‘Gemini Mode,’ hinting at new Gemini releases.
Kilpatrick told AIM that Google plans to release Gemini 2, which will feature better reasoning quality and a longer context window—potentially up to billions or trillions of tokens. As per Kilpatrick, the model will be fully multimodal, with the capability to understand large videos as well.
OpenAI thinks it has o1
In a recent interview OpenAI chief Sam Altman, accused other research organisations of copying OpenAI’s methods. “We try to have a very focused research program. I think one of the mistakes that other research programs make is they don’t have enough conviction and concentration. It’s very easy to copy something once it works,” he said.
However, this might not entirely be true.
After Transformer, which was first published by Google in the ‘Attention Is All You Need’ paper back in 2017, OpenAI incorporated one of Google DeepMind’s most popular reinforcement learning school of thoughts, pushing Google to accelerate its timeline for releasing their advanced models.
Recently, Apples shared a document on X, dated last year, revealing that Google is planning to integrate the ‘PLANNING’ piece in the LLM. Moreover in an old Wired article, Google’s Demis Hassabis also said that his team will combine the technologies used in AlphaGo, aiming to give the system new capabilities such as planning and ability to solve new problems.
Interestingly, when OpenAI o1 was launched, Subbarao Kambhampati, professor at Arizona State University, speculated that o1 incorporates RL over Private CoT methodologies.
Chain of Thought or ‘CoT’ refers to generating step-by-step reasoning or thought processes, which can enhance the model’s ability to tackle complex tasks.
He drew an analogy to Google DeepMind’s AlphaGo, a program developed for playing the game of Go, suggesting that o1 could operate in a similar way by defining ‘moves’ for problem-solving.
Moreover, Kambhampati suggested that the RL task involves generating and selecting a CoT based on the original prompt, evaluating success or failure by whether the output aligns with expected answers from the training data. In AlphaGo, success was determined by game outcomes. For o1, success could be measured by whether the model’s expanded prompts lead to correct answers based on the training data.
His claims might be valid, as OpenAI stated in its blog post that, “Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses. It learns to recognize and correct its mistakes and to break down tricky steps into simpler ones.”
Google earlier this year published a paper titled, ‘Chain of Thought Empowers Transformers to Solve Inherently Serial Problems’ which says that by increasing the length of CoT can drastically make transformers more expressive. The researchers define a new class of problems that can be solved by transformer models with specific limitations, like how many steps they can take and the size of the data they can handle.
Not only that, Google recently published another paper titled ‘Training Language Models to Self-Correct via Reinforcement Learning’. Google Deepmind has developed a multi-turn online reinforcement learning approach to improve the capabilities of an LLM to self-correct. SFT is shown to be ineffective at learning self-correction and suffers from a distribution mismatch between training data and model responses.
On the other hand, many are claiming that OpenAI’s o1 could be considered the first successful commercial launch of a System 2 LLM.
A System 2 LLM is a type of language model intended to mimic more deliberate and analytical thinking, paralleling Daniel Kahneman’s concept of “System 2” thinking. In this framework, System 1 is characterised by fast, automatic, and intuitive responses, while System 2 involves slower, more methodical reasoning that necessitates conscious effort.
“For those raving about GPT-4 o1, Google has been working on extending the chain of thought (i.e., System 2) since Gemini was released. That is why one should pay attention to specialised open-source Gemma implementations. System 2 thinkers are specialists, not generalists,” posted Carlos E. Perez, co-founder of Intuit Machine, referring to Google Gemini’s ‘Uncertainty Routed Chain of Thought.’
Google DeepMind is the Brain Behind all the AI Innovations
Kilpatrick told AIM that the Google Gemini team and Google DeepMind work very closely together. He further revealed that the Google DeepMind team ultimately wants to ensure that the technology reaches both developers and the wider world. “They care a lot about making sure that the product teams building on top of the models are kept in the loop with what’s coming.”
Google DeepMind’s recent models, AlphaProof and AlphaGeometry 2, won a silver medal at this year’s International Mathematical Olympiad (IMO). Meanwhile, OpenAI o1 scored 83% in a qualifying exam for the International Mathematics Olympiad, compared to GPT-4o’s 13%
Alpha Geometry2, a neuro-symbolic hybrid system built on Gemini, is trained from scratch using an order of magnitude more synthetic data than its predecessor. With further improvements to Google DeepMind’s RL techniques and their integration with Chain of Thought in Gemini, Google could create a model that outperforms OpenAI’s o1.
Not to forget, Google recently announced that YouTube is set to roll out advanced generative AI tools for creators in the coming months, enabling them to generate video content using the AI models Veo and Imagen 3 through a feature called Dream Screen, leaving Sora behind.
The post Why Google Will Make a Better Model Than OpenAI’s o1 appeared first on AIM.
