The National Testing Agency (NTA) recently came under fire following allegations of malpractice in conducting the 2024 National Eligibility cum Entrance Test (NEET).
Several students scoring perfect marks in the test raised alarms, leading to allegations of cheating and paper leaks. While the NTA bears the brunt of the scandal, it also raises concerns about the reliability of the test itself.
The fact that these irregularities were caught after the blowout and not during the exam itself raises concerns about how effective NTA’s proctoring technology is. This is particularly timely as well, since this year, the testing agency made a big deal about using AI for surveillance.
“Generative AI can play a huge role in assessment content authoring. For example, our Ai-levate product includes an AI-powered service called item rewriter, which can create five different versions of the same question while maintaining its core essence and difficulty level,” Adarsh Sudhindra, the VP for growth and strategy, Excelsoft, told AIM.
This, he explained, helped ensure that questions weren’t repeated and made it difficult for potential anyone to memorise a specific question, in the case of leaks.
“A proper use of technology at various stages in the assessment life cycle can significantly improve the overall security of testing. There are points where human error can occur; if we can automate all those points and plug those leaks and bring in safe, secure practices of using technology, this would not happen,” said Sudhindra.
Where Does GenAI Fit into Proctoring and Assessment?
Mysuru-based edtech company Excelsoft has been providing AI-based proctoring services for about eight years. Ai-levate is just one of several products that the company offers, with plans to integrate GenAI within their e-learning offerings as well.
With 10 to 20% of the company’s top line being AI-driven products, its proctoring solution easyProctor is used in several countries, including the US and the UK, to proctor large-scale examinations.
Apart from a number of features, easyProctor is primarily an AI-powered solution which makes use of AI algorithms and real-time monitoring to help proctor different examinations and tests both offline and online.
Now, while easyProctor itself doesn’t use generative AI for proctoring, Sudhindra said that it had a huge scope for assessments, with services like the item rewriter.
It provided the benefit of ensuring that examinations with massive question banks, like NEET, would not have questions that had potential similarities or conflicts with each other. This, in particular, is quite common, with grace marks awarded every year due to poorly framed questions.
For proctoring purposes, the company also has its own set of models, fine-tuned specifically for the purpose of proctoring, switching over from using Amazon Bedrock and SageMaker.
“Since 2021, we’ve been developing, training, and fine-tuning our models specifically for the proctoring use case. We used to use Amazon’s services a couple of years ago, but the cost and trainability were a challenge.
“So we have developed our own models and they are on par with Amazon’s models because we are able to train and fine-tune it for the proctoring use case. Whereas some of these open models out there are not built for proctoring,” he said.
This means multi-modal capabilities fine-tuned specifically for the purpose of proctoring, including multi-camera monitoring, environmental scanning, facial recognition, screen and audio monitoring, keystroke analysis, and malpractice detection, all powered by their AI models.
Their success has shown that AI does have a place in the proctoring field, especially since, according to Sudhindra, ExcelSoft’s easyProctor is currently being employed by 12 major universities in India.
Despite competing with major companies like ProctorU, Honorlock, and Eklavvya in India, the company has managed to bag several major government partnerships with European countries and state governments in the US to proctor their driver’s licence and civil service exams respectively.
Why Doesn’t the NTA Employ Similar Methods?
Well, they sort of do. The exact details on the NTA’s usage of AI in the NEET examinations is unclear.
However, what we do know is that they made use of AI-powered CCTV surveillance, analytics tools, face recognition and post-examination analysis, with the command centre also using AI to assess issues of malpractice. In fact, the NTA also released details of malpractice issues being called out this year, thanks to the use of these AI-powered measures.
But, as Sudhindra said, there are still points of exposure that can be rectified when it comes to the entire process of proctoring and assessment. “Everywhere there is human exposure, wherever you can bring in technology and automate this, you are increasing the overall security of the assessment process. So, like I said, the construction of the exams can be all digitised,” he said.
Additionally, he suggested the use of a single central cloud server with limited access, increasing accountability and reducing the possibility of leaks. He also spoke about the costs of these measures.
While the amount of AI integration would determine the cost, overall, it was less expensive to employ these methods than to hire more proctors to plug these gaps.
“The ratio of the proctor to the candidate also matters. It’s 1:8, which is the current industry standard. With augmentation, it can go to 1:16. With a little more AI, it can go to 1:32, thereby bringing the cost down and decreasing the bandwidth also,” he pointed out.
Potentially, with more time and as the NTA gets used to using AI-powered solutions, the accountability of major exams like NEET can improve, ensuring that students are given a fair chance when it comes to pursuing their career goals.
