Every year, floods displace millions of people in Assam, forcing them to seek shelter in relief camps. In fact, devastating floods were some of the watershed events in the state’s history.
Each year, the Indian government allocates hundreds of crores towards relief efforts during Assam floods. But, a major challenge the government faces is ensuring that these funds are effectively utilised and that the right resources reach the appropriate locations promptly.
This issue persists due to the fragmented nature of government data, which is often stored in isolated silos. In the case of Assam government, the data related to disaster management is stored across 18 different departments and other central agencies.
But a research lab working at the intersection of data, technology, design and social science has come to their aid. Called Civic Data Labs (CDL), the startup is working closely with the Assam government to unify its data and analyse it in a much better way.
( Credits: Reuters)
Setting Data Standards
“The Government of Assam is pushing in a lot of money to build disaster risk reduction, but they’re still not confident that whether money is going in the right geography at the right time, and whether that’s translating into increased risk reduction,” Gaurav Godhwani, co-founder of Civic Data Labs told AIM.
The finance ministry in Assam and the Disaster Management Authority knocked on the startup’s door, and they decided to answer. Godhwani revealed that the startup was given access to all the datasets except one under a non-disclosure agreement. When it came to the dataset related to public procurement, the startup had to manage the data and standardise it.
“We worked with the department to see what procurements were happening related to floods– When did the last embankment repair happen after damage happened? When was the restoration of that particular geography conducted? Pretty soon, we realised that the Assam government did not have a clear standard to manage the data,” he said.
Hence, CDL introduced an international data standard known as the Open Contracting Data Standard, which is adopted in over 50 nations. As a result, Assam became the first state in India to adopt this international standard.
Moreover, Godhwani and his team created a platform where all the datasets come together. They also created a data exchange layer that enables all the publishers, including the 18 departments and other central agencies, to manage disaster-related data.
“Most of them have their own databases, their own APIs, not talking to each other. The challenge was to bring all of those together in one place. So we streamline the datasets, standardise them based on common geographical indicators, like state, district, and revenue circles and give an ID to all of those data sets so that they are all uniquely identified,” Godhwani explained.
(Stakeholder Consultation with ASDMA, Source: Civic Data Labs)
Early success
Once the datasets were streamlined and standardised, CDL used simple machine learning techniques to derive valuable insights from the data.
“We presented our initial findings, demonstrating hazard analysis and disaster modelling using GIS techniques and simple machine learning techniques. We highlighted districts and revenue circles at higher risk, emphasising areas needing greater attention,” Godhwani said.
In response, the Assam Government requested the integration of disaggregated data for periodic analysis spanning a five-year period.
CDL developed a platform which allows comprehensive assessment of past trends and facilitates planning for the upcoming monsoon cycle and subsequent restoration efforts.
Godhwani revealed that the Assam Government has already started results by leveraging the data model CDL built. The authorities used the model to determine the allocations and verified them with assistance from on-ground staff.
“They found that the algorithm effectively identified areas that had previously been overlooked. As a result, revenue circles that historically received insufficient funds have now begun to receive more support based on our model,” Godhwani pointed out.
Secondly, the department’s capabilities have significantly expanded. Previously, generating analytics reports during disasters was a time-consuming process. They lacked knowledge of suitable software and effective data visualisation techniques.
“Now, they are actively pursuing courses on geospatial data management and AI on Coursera. We have observed a renewed enthusiasm for data science and what it can do.”
( Capacity building workshop with Assam state and district officers conducted by CDL, Source: Civic Data Labs)
Machine Learning Techniques Come to Aid
Godwani said this was done using three three machine learning algorithms. The initial algorithm employed was Random Forest, aimed at understanding the potential for inundation. This was achieved by analysing satellite imagery sourced from multiple sources to observe the extent of inundation during the monsoon season.
Secondly, the Data Envelopment Model was used to extract variables from textual information, like maternal and child health indicators, public procurement etc.
“The algorithm helps in understanding how many times procurement has happened before the monsoon season, during the monsoon season and after the monsoon season. The algorithm creates a model on top of all of these, giving them scores based on the past data, and when the past data is missing, it creates a baseline that whenever the new data is coming, we can measure whether we are up that baseline or below the baseline.”
Lastly, CDL then utilises the Topsis scoring method to consolidate all data into a risk index, enabling measurement on a scale of one to five, where one indicates lower risk and five indicates higher risk. This scoring methodology computes all variables in one central location and provides timely updates to the department on which areas to prioritise each month. This ensures that updates can be made on a monthly basis.
“Earlier, it used to take two years or so to compute just one timestamp for the department. So that’s the kind of automation which we’re seeing with these machine learning algorithms,” Godhwani pointed out.
Leveraging generative AI
While machine learning algorithm has sufficed so far, Godhwani believes Large Language Models (LLMs) can be leveraged to scale the platform.
Currently, all the data generated goes to a public platform, but Godhwani believes that if the same data has to be disseminated to the public, something like a rule-based engine would not suffice.
Information could be disseminated to the public through a chatbot or through social media applications which the public is already using.
“Information such as, for instance, if you’re going in this particular area, please be careful there is a likely inundation, which may happen in the next 24 hours, 48 hours because of heavy rains. That’s the kind of opportunity we have with large language models,” Godhwani said.
CDL has already conducted a pilot and shown the results to the Assam government. But in terms of deployment, it’s still a few months away.
“The government agencies need to become more comfortable in piloting this at scale. The other thing is ensuring the ethics and the modelling are right. It can’t be hallucinating, as most of these models today still hallucinate. If they are, the risks are extremely high.”
About Civic Data Labs
Godhwani founded Civic Data Labs with Deepthi Chand. Interestingly, the duo has registered Civic Data Labs as a startup and not as a non-profit. The team is made up of 52 strong members, including disaster risk production experts, climate scientists, data scientists, data engineers, technologists, design or user researchers, and technological architects.
Even though the company is bootstrapped, it raises funds for projects. For instance, the project with the Assam government is funded by the Rockefeller Foundation and the Patrick J Mcdovin Foundation.
The startup works closely with non-profits, volunteer groups, government agencies, and departments.
The post Machine Learning Enhances Assam Government’s Disaster Response Amid Floods appeared first on Analytics India Magazine.
