The present-day AI system is smitten with hallucinations – for some, it is a feature (and not a bug), and for others, it is a nuisance (or frustration). The latter keeps Amitava Das, a professor at the University of Southern Carolina, worried, leading to the creation of what he calls ‘Civilising AI or Constitutional AI’
“AI [LLMs] are very smart and at the same time, they are also stupid. So this smartness and stupidity, we don’t know how to handle this. On the other hand, AI-generated content becomes so eloquent that it is impossible to detect it. No Turing test is able to do it,” said Das, in an interview with AIM, sharing the origin of Civilised AI, a framework to proactively measure the hallucination of a model, alongside mitigating and managing risks associated with it.
Hailing a PhD in Natural Language Processing (NLP) from Jadavpur University in Kolkata, Das has been working in the field for more than 20 years, which he says was all because of his passion for working in AI ever since the beginning. Having two academic postdocs from Europe and the USA, he is currently a professor at the Artificial Intelligence Institute at the University of Southern Carolina (AIISC).
Das started the AI research lab at Wipro and is still an advisor there. Having worked across the globe including Japan, Norway, and along with IITs and universities across the globe, he has also created a joint PhD program for industry practitioners to earn doctorate degrees. Currently, he is focused on reducing hallucinations within language models.
There are several research papers and models coming up to solve the AI detection problem. GPTZero, DetectGPT, and RoBERTa baseline are all trying to do the same, but a lot still needs to be done. The most recent paper, Ghostbuster, also works on solving the same problem.
That is why Das and his team coined the term ‘Civilised AI’ to explain how to handle the problem of hallucinations, and at the same time also make it easier to detect if a text is AI generated, and tackle the misinformation and the possible harm it poses to the society.
Single metric to civilise AI
Using the example of Google’s Bard and how the company lost around $100 million because of its hallucinations, Das emphasised the need to control what AI is writing. “On one hand, we need AI to be creative when it comes to fields like journalism as it helps in writing. On the flip side, if I am a lawyer and doing a legal case, hallucination cannot be allowed at all,” he explained.
Das’s paper, titled The Troubling Emergence of Hallucination in Large Language Models – An Extensive Definition, Quantification, and Prescriptive Remediations – addresses this problem of balancing hallucinations with eloquence within AI models, along with ‘Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think — Introducing AI Detectability Index’, the paper that won the Outstanding Paper award at EMNLP 2023.
“We call this dichotomy a factual mirage and silver lining, how much of each is allowed,” explained Das, and that is what led his research team to introduce new metrics called AI detectability index, AI adversarial attack vulnerability Index, and AI hallucination index.
“AI is just a balance between these three,” he added that there is still not a clear-cut solution for this, but the team is trying to derive a formula for this, which is called the Civilising Index, an amalgamation of all of this.
He said it is a difficult scenario to tackle giving the example of Einstein’s Mass-Energy Equivalence Relation, which led to the creation of the atomic bomb, along with also giving examples of other AI-related research that needs to be regulated if it involves risk to human lives such as bioinformatics.
“It is the people who created the atomic bomb and the people who created atomic energy. So there are people who should make AI models better, and then there is my team who will make better detection techniques,” he added.
“Regulation has to come to use case specific, not in a broad way,” he asserted. Each field such as medicine, military, legal, or any other field needs to have separate licences on how they can use AI to tackle the hallucinations, leading to emerging risks within the systems.
The need for Indic LLMs
Apart from this, Das is also passionate about how language models can be made for Indic languages and has also released a code-mixing model called CONFLATOR for the same purpose. He started working in this field around eight years back and is regarded as one of the pioneers of the field by Microsoft Research.
He explained that the problem that comes with Indic languages and datasets is that it is very hard to understand. “Hindi is getting mixed into English or English is getting mixed into Hindi,” Das explained about how it is very difficult to understand the switching point in languages as most of the data is from the Internet.
“That is why we introduced Conflator,” he added, which is still a complicated field that deals with introducing electromagnetic waves into a neural network for code-mixing for positional encoding formulation.
When it comes to Indic language models “we do not have a lot of monolingual data in India. This discussion has been going on between NITI Aayog and IIIE for several years, before the launch of ChatGPT and others,” Das was part of the discussions with the Indian government for making Indic language models possible.
That is why he believes that Indian models are still far behind English models in terms of power and detectability. Das told AIM that he is closely working with the Indian government and IIIT Hyderabad and is soon going to publish a paper on the topic.
The post This USC Professor from Kolkata is on a Journey to Civilise AI appeared first on Analytics India Magazine.
