Generative AI, particularly exemplified by chatbots like OpenAI’s ChatGPT, Anthropic’s Claude 3.5 Sonnet, Google’s Gemini, have captured widespread attention and fascination.
However, the notion that generative AI might soon wane in popularity is gaining traction. In a recent podcast, tech critic Ed Zitron claimed that language models will soon run out of data to train on, and therefore stop getting better. “LLMs just cannot create ‘new’ knowledge,” he added.
End Of The Story For LLMs?
On similar lines, a study by Google DeepMind proved that LLMs lack genuine understanding, and as a result, cannot self-correct or adjust their responses on command.
Andrej Karpathy is also going through similar tussle, and calls this “Jagged Intelligence”, which he describes as the inconsistent performance of SOTA LLMs that excel at complex tasks, yet fail at simple ones, such as incorrectly determining that 9.11 is larger than 9.9.
This has led to a fear that there is very limited application of current AI systems in the real world. In an exclusive interaction with AIM, Meta AI chief Yann LeCun said that systems in their current form are just for entertainment, and do not lead to anything useful. “I don’t think these systems in their current state can be fixed or called intelligent in ways that we want and expect them to be,” said LeCun.
Meanwhile, Atlantic CEO Nicholas Thompson also gave an interesting example for this. “If you ask an AI how do you get a man and a cabbage across a river you will potentially get an insane (and hilarious) answer that very clearly shows the weakness of building word-prediction engines that act like they are thinking,” he said.
A research paper by researchers including the creator of GAN, Ian Goodfellow, titled, ‘Explaining and Harnessing Adversarial Examples’ demonstrated that even state-of-the-art deep learning networks are often unable to learn how to recognise images in a manner convincing enough to be generalised for different tasks.
Even machines that are adept at playing games, which use deep reinforcement learning, aren’t known to follow generally-applicable principles that can then help them play many other games.
So, what’s the solution?
AI analyst, and critic, Gary Marcus argued that the solution is neurosymbolic AI.As per Marcus, neurosymbolic AI combines neural networks, which are adept at quick pattern recognition, with symbolic systems that utilise formal logic and structured reasoning.
By integrating these two methodologies, neurosymbolic AI aims to leverage the strengths of both approaches.
Recent advancements by Google DeepMind illustrate the potential of neurosymbolic systems. GDM’s AlphaProof and AlphaGeometry represent sophisticated hybrids of neural and symbolic techniques.
AlphaGeometry, for example, integrates a neural language model with a symbolic deduction engine to solve complex geometry problems. Similarly, AlphaProof combines neural and symbolic elements to achieve impressive results in mathematical theorem proving.
Neurosymbolic AI Nut
The conversation around neurosymbolic AI isn’t new. For decades, making a machine fully capable of learning by observing its environment has been the biggest dream for many researchers.
I’m glad we all agree that neurosymbolic AI is the way to go. Who proposed it first really doesn’t matter.
— Pedro Domingos (@pmddomingos) September 27, 2022
Even deep learning, the branch of ML that is fashionable now, can be traced back to 1943. Likewise, a section of scientists had long anticipated the potential in adopting neurosymbolic AI systems so machines can reach human-levels of comprehension.
Work In Progress
IBM is using neurosymbolic AI to explore how to make AI systems learn like humans. A notable collaboration on neurosymbolic AI involves IBM, MIT, Harvard, and Stanford, supported by DARPA funding. Their project aims to reverse-engineer cognitive capabilities observed in infants, such as object permanence and basic reasoning.
In short, the research involves using probabilistic programming and game engine design principles to develop AI systems that can simulate and understand real-world dynamics, much like how infants learn from their environment.
Glimpse of Future
As per David Cox, Director of MIT-IBM Watson AI lab, Symbolic AI systems rely on pre-coded concepts and rules, which can make them brittle and unable to handle unexpected or novel situations.
Additionally, these systems also struggle with problems that require learning from data, as they are not as adept at extracting patterns and generalising from examples. “It’s time to reinvent artificial intelligence”, Cox said. And, according to him, neurosymbolic AI is the answer.
Neurosymbolic models are emerging as an effort towards AGI by both exploring an alternative to just increasing datasets’ and models’ sizes and combining learning over the data distribution, reasoning on prior and learned knowledge, and by symbiotically using them.
Demis Hassabis, CEO of DeepMind, has also suggested that the ultimate goal of AI is to create AGI systems that can learn and reason like humans. In an interview, Hassabis stated that “the path to AGI will likely involve a combination of deep learning with other techniques like reasoning and planning.”
LeCun has a similar vision for autonomous machine intelligence. In his paper, LeCun proposed several solutions and architectures that can be combined and implemented to build self-supervised autonomous machines.
The paper distills much of my thinking of the last 5 or 10 years about promising directions in AI.
It is basically what I'm planning to work on, and what I'm hoping to inspire others to work on, over the next decade.
2/N pic.twitter.com/UeCO66GRQd— Yann LeCun (@ylecun) June 27, 2022
The approach has even found its way into very recent models like CICERO, an agent announced by Meta AI. CICERO was the first AI to reach human-level performance at Diplomacy, a strategy-based board game.
Marcus, admitted that, while it wasn’t clear how generalisable Cicero was, “some aspects of Cicero use a neurosymbolic approach to AI, such as the association of messages in language with symbolic representation of actions, the built-in (innate) understanding of dialogue structure, the nature of lying as a phenomenon that modifies.”
In a paper published in December last year titled, ‘A Semantic Framework for Neural-Symbolic Computing,’ authors Simon Odense and Artur d’Avila Garcez refer to how integrating a semantic framework can help neurosymbolic AI further.
All of this is to just say that neurosymbolic AI is worth more than one shot to smoothen the road to AGI.
The post Can Neurosymbolic AI Save LLM Bubble from Exploding? appeared first on AIM.
