TATA AIG, one of the country’s leading insurance providers, is building a large language model (LLM)- powered chatbot for customers and plans to release it soon via WhatsApp.
However, given that hallucinations with LLMs remain an unsolved problem, TATA AIG is taking a cautious approach. For now, the insurer has launched an internal AI chatbot, which its customer service agents are currently using.
According to Krishnan Badrinath, SVP of information technology at TATA AIG, they are among the first insurers in the general insurance space in the country to launch a Generative AI use case.
“Currently, we’ve achieved an accuracy rate of over 80%, and our goal is to further enhance this accuracy before delivering it to our customers through WhatsApp,” he said.
Multiple LLMs-both powers TATA AIG’s AI chatbot closed and open-source models. “Cost efficiency is crucial, so for simpler tasks, we opt for open-source LLMs. But for complex tasks requiring high accuracy and processing power, we utilise the GPT models by OpenAI or Google’s Gemini.”
Further, Badrinath revealed that the company has established an enterprise layer to select the appropriate service strategically. “For instance, out of 20 processes, 15 may use open LLMs, while the remaining 5 utilise GPT models or Gemini for more intricate tasks,” he added.
With thousands of open-source and proprietary models evolving rapidly, the TATA AIG team consistently evaluates these options. It selects models based on thorough assessments to determine the best fit for their specific use cases.
“We maintain a continuous feedback loop, assessing the accuracy of each model based on the specific dataset and other scenarios. This mechanism allows us to adjust and switch between models as needed, including Panther, Llama, and others currently in use,” Ritesh Pandey, head of the Innovations Lab at TATA AIG, told AIM.
Challenges Galore
Pandey further recognised the obstacles of utilising LLMs in the insurance sector and emphasised the challenge of fully deploying an autonomous solution without human intervention.
He believes the regulatory nature of the industry instils scepticism and caution in embracing full autopilot modes. Instead, many companies aim for a copilot approach where algorithms handle the bulk of tasks, with human oversight for validation and feedback.
Badrinath noted that LLM hallucination is one of the key reasons for TATA AIG’s initial internal testing of LLMs. “We’ve begun the process from within, ensuring our agents provide feedback on the responses generated by the chatbot,” he added. Moreover, a dedicated team at TATA AIG reviews each response to gauge its confidence level.
Nonetheless, the company has taken a proactive approach towards AI. Tata AIG embarked on its AI journey well before the Generative AI surge. The insurer has been using AI to analyse the sentiment of received emails. It has also developed an AI-based self-inspection module for customers to assess their performance independently.
“We have trained 150 vision models which identify different vehicle parts and help customers carry out the self-assessment. We have also employed AI during live video streaming to extract pertinent information and evaluate risks associated with the business or property being inspected,” said Badrinath.
Reducing Hallucinations
While generative AI does bring value to the larger banking, financial services and insurance (BFSI) industry, hallucinations in LLMs remain an unsolved problem.
A reputable insurer, like TATA AIG, would not want a chatbot to disseminate nonsensical, inaccurate, or financial information as it could harm customer trust, regulatory compliance, and financial stability.
TATA AIG said it is actively working to minimise hallucinations and enhance confidence in its internal chatbot. “By specifying parameters like the temperature, which controls the model’s creativity, hallucinations can be minimised. In our case, accuracy is paramount, even if the model occasionally cannot provide an answer, but it should not provide incorrect answers.
“Therefore, maintaining a low temperature ensures that the model stays within the prescribed search space, limiting hallucinations and aligning with our use case requirements,” Pandey said.
Recently, ServiceNow reduced hallucination in structured outputs through Retrieval-Augmented Generation (RAG), enhancing LLM performance and enabling out-of-domain generalisation while minimising resource usage.
Another technique is Chain-of-Verification (CoVe) by Meta AI. This method reduces hallucination in LLMs by breaking down fact-checking into manageable steps, enhancing response accuracy and aligning with human-driven fact-checking processes.
Then, there is prompt engineering. Marc Andreessen also recently said with the right prompting, you can unlock the latent super genius in AI models. “Prompting crafts in many different domains such that you’re kind of unlocking the latent super genius,” he added.
What’s next?
TATA AIG is not alone in the generative AI race. Other insurers in India are also exploring generative AI use cases. For example, Bajaj Allianz General Insurance launched a new AI chatbot called Insurance Samjho, which helps customers better understand their policies. Similarly, Aditya Birla Sun Life Insurance partnered with Artivatic.ai last year to use LLMS for underwriting insurance.
As we advance, TATA AIG looks to leverage LLMs to solve more complex business problems, from analytics to servicing, fraud prevention and fraud detection.
Badrinath said generative AI could be used for forecasting as well. Given that TATA AIG has products in multiple verticals, such as motor, health, travel, personal accident, personal lines, extended warranty, etc. LLMs can forecast business numbers for the next six months.
“Businesses today are sitting on a plethora of data, but currently, this forecasting is manual and channel-specific. With LLMs, a simple query in English, such as ‘What are my motor business projections for the next six months?’, businesses can get the data instantly,” Badrinath said.
Moreover, he believes generative AI will be crucial in the complex process of new business ratings within the general insurance industry.
“Unlike life insurance, which typically has a limited range of products, we deal with many categories. This complexity extends to the rating systems, where rule engines are heavily relied upon. We anticipate that Generative AI will eventually replace these rule engines, streamlining and enhancing the rating process,” concluded Badrinath.
The post TATA AIG is Building an LLM-Powered WhatsApp Chatbot for Customers appeared first on Analytics India Magazine.
