A Playbook to Scale MLOps

By Mike Caravetta & Brendan Kelly

Scaling MLOps for Your Teams

MLOps teams are pressured to advance their capabilities to scale AI. In 2022, we saw an explosion of buzz around AI and MLOps inside and outside of organizations. 2023 promises more hype with the success of ChatGPT and the traction of models inside enterprises.

MLOps teams look to expand their capabilities while meeting the pressing needs of the business. These teams start 2023 with a long list of resolutions and initiatives to improve how they industrialize AI. How are we going to scale the components of MLOps (deployment, monitoring, and governance)? What are the top priorities for our team?

AlignAI teamed up with Ford Motors to write this playbook to guide MLOps teams based on what we have seen be successful to scale.

What Does MLOps Mean?

To start, we need a working definition of MLOps. MLOps is an organization’s transition from delivering a few AI models to delivering algorithms reliably at scale. This transition requires a repeatable and predictable process. MLOps means more AI and the associated return on investment. Teams win at MLOps when they focus on orchestrating the process, the team, and the tools.

Foundational components of MLOps to scale

Let’s walk through each area with examples from Ford Motors and ideas to help get you started.

  • Measurement and Impact: how teams track and measure progress.
  • Deployment & Infrastructure: how teams scale model deployments.
  • Monitoring: maintaining the quality and performance of models in production.
  • Governance: creating controls and visibility around models.
  • Evangelizing MLOps: educating the business and other technical teams on why and how to utilize MLOps methods.

Measurement and Impact

One day, a business executive walked into Ford’s MLOps command center. We reviewed the usage metrics of a model and had a productive conversation about why usage had dropped. This visibility of the impact and adoption of models is crucial to building trust and reacting to the needs of the business.

A fundamental question for teams leveraging AI and investing in MLOps capabilities is how do we know if we are progressing?

The key is to align our team on how we provide value to our customers and business stakeholders. Teams focus on quantifying performance in the business impact they provide and the operational metrics enabling it. Measuring impact captures the picture of how we generate.

Ideas to get started:

  1. How do you measure the value of models in development or production today? How do you track the usage and engagement of your business stakeholders?
  2. What are the operational or engineering metrics for your models in production today? Who owns the improvement of these metrics? How do you give people access to see these metrics?
  3. How do people know if there is a change in user behavior or solution usage? Who responds to these issues?

Deployment & Infrastructure

The first hurdle the team faces in MLOps is deploying models into production. As the number of models grows, teams must create a standardized process and shared platform to handle the increased volume. Managing 20 models deployed using 20 different patterns can make things cumbersome. Enterprise teams typically create centralized infrastructure resources around X models. Choosing the right architecture and infrastructure across models and teams can be an uphill battle. However, once it is established, it provides a strong foundation to build the capabilities around monitoring and governance.

At Ford, we created a standard deployment function using Kubernetes, Google Cloud Platform, and a team to support them.

Lucid Link

Ideas for your team:

  1. How will you centralize the deployment of models? Can you create or designate a centralized team and resources to manage the deployments?
  2. What deployment patterns (REST, batch, streaming, etc.)?
  3. How are you going to define and share those with other teams?
  4. What are the most time-consuming or difficult aspects for your modeling teams to overcome to get a model in production? How can the centralized deployment system be designed to mitigate these issues?

Monitoring

A unique and challenging aspect of machine learning is the ability of models to drift and change in production. Monitoring is critical in creating trust with stakeholders to use the models. Google’s Rules of Machine Learning says to “practice good alerting hygiene, such as making alerts actionable.” This requires teams to define the areas to monitor and how to generate these alerts. A challenging piece becomes making these alerts actionable. There needs to be a process established to investigate and mitigate issues in production.

At Ford, the Model Operations Center is the centralized location with screens full of information and data to understand if the models are getting what we expect in near real-time.

Here is a simplified example of a dashboard looking for usage or record counts dropping below a set threshold.

Monitoring Metrics

Here are monitoring metrics to consider for your models:

  • Latency: Time to return predictions (e.g., batch processing time for 100 records).
  • Statistical Performance: the ability of a model to make correct or close predictions given a test data set (e.g., Mean Squared Error, F2, etc.).
  • Data Quality: quantification of the completeness, accuracy, validity, and timeliness of the prediction or training data. (e.g., % of prediction records missing a feature).
  • Data Drift: changes to the distribution of data over time (e.g., lighting changes for a computer vision model).
  • Model Usage:how often the model predictions are used to solve business or user problems (e.g., # of predictions for model deployed as a REST endpoint).

Ideas for your team:

  1. How should all models be monitored?
  2. What metrics need to be included with each model?
  3. Is there a standard tool or framework to generate the metrics?
  4. How are we going to manage the monitoring alerts and issues?

Governance

Innovation inherently creates risk, especially in the enterprise environment. Therefore, successfully leading innovation requires designing controls into the systems to mitigate risk. Being proactive can save a lot of headaches and time. MLOps teams should proactively anticipate and educate stakeholders on the risks and how to mitigate them.

Developing a proactive approach to governance helps avoid reacting to the needs of the business. Two key pieces of the strategy are controlling access to sensitive data and capturing lineage and metadata for visibility and audit.

Governance provides great opportunities for automation as teams scale. Waiting for data is a constant momentum killer on data science projects. At Ford, a model automatically determines if there is personally identifiable information in a data set with 97% accuracy. Machine learning models also help with access requests and have reduced the processing time from weeks to minutes in 90% of the cases.

The other piece is tracking meta-data throughout the model’s life cycle. Scaling machine learning requires scaling trust in the models themselves. MLOps at scale require built-in quality, security, and control to avoid issues and bias in production.

Teams can get caught up in the theory and opinions around governance. The best course of action is to start with clear access and controls around user access.

From there, meta-data capture and automation are key. The table below outlines the areas to collect meta-data. Wherever possible, leverage pipelines or other automation systems to capture this information automatically to avoid manual processing and inconsistencies.

Meta-data to Collect

Here are the items to collect for each model:

  • Version/ Trained Model Artifact:Unique identifier of the trained model artifact.
  • Training Data- Data used to create the trained model artifact
  • Training Code- Git hash or link to the source code for inference.
  • Dependencies- Libraries used in training.
  • Prediction Code- Git hash or link to the source code for inference.
  • Historical Predictions- Store inferences for audit purposes.

Ideas for your team:

  1. What issues have we encountered across projects?
  2. What problems are our business stakeholders experiencing or concerned about?
  3. How do we manage access requests for data?
  4. Who approves them?
  5. Are there automation opportunities?
  6. What vulnerabilities do our model pipelines or deployments create?
  7. What pieces of meta-data do we need to capture?
  8. How is it stored and made available?

Evangelizing MLOps

Many technical teams fall into the pitfall of thinking: “if we build it, they will come.” There’s more to solving the problem. It also involves sharing and advocating for the solution to increase organizational impact. MLOps teams need to share the best practices and how to solve the unique problems of your organization’s tools, data, models, and stakeholders.

Anybody in the MLOps team can be an evangelist by partnering with the business stakeholders to showcase their success stories. Showcasing examples from your organization can illustrate the benefits and opportunities clearly.

People across the organization looking to industrialize AI need education, documentation, and other support. Lunch and Learns, onboarding, and mentorship programs are great places to start. As your organization scales, more formalized learning and onboarding programs with supporting documentation can accelerate your organization’s transformation.

Ideas for your team:

  1. How can you create a community or recurring learnings and best practices for MLOps?
  2. What are the new roles and capabilities we need to establish and share?
  3. What problems have we solved that can be shared?
  4. How are you providing training or documentation to share best practices and success stories with other teams?
  5. How can we create learning programs or checklists for data scientists, data engineers, and business stakeholders to learn how to work with AI models?

Getting Started

MLOps teams and leaders face a mountain of opportunities while balancing the pressing needs of industrializing models. Each organization faces different challenges, given its data, models, and technologies. If MLOps were easy, we probably would not like working on the problem.

The challenge is always prioritization.

We hope this playbook helped generate new ideas and areas for your team to explore. The first step is to generate a big list of opportunities for your team in 2023. Then prioritize them ruthlessly based on what will have the largest impact on your customers. Teams can also define and measure their maturity progress against emerging benchmarks. This guide from Google can provide a framework and maturity milestones for your team.

Ideas for your team:

  1. What are the largest opportunities to advance our maturity or sophistication with MLOps?
  2. How do we capture and track our progress on the projects that advance maturity?
  3. Generate a list of tasks for this guide and your team. Prioritize based on the time to implement and the expected benefit. Create a roadmap.

References

  • https://www2.deloitte.com/content/dam/insights/articles/7022_TT-MLOps-industrialized-AI/DI_2021-TT-MLOps-industrialized-AI.pdf
  • https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf
  • https://developers.google.com/machine-learning/guides/rules-of-ml
  • https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning

Mike Caravetta has delivered hundreds of millions of dollars in business value using analytics.He currently leads the charge to scale MLOps in manufacturing and complexity reduction at Ford.
Brendan Kelly, Co-Founder of AlignAI, has helped dozens of organizations accelerate MLOps across the banking, financial services, manufacturing, and insurance industries.

More On This Topic

  • Scale and Govern AI Initiatives with ModelOps
  • TransformX by Scale AI is Oct 19-21: Register for free!
  • Computer Vision at Scale With Dask And PyTorch
  • How to get started managing data quality with SQL and scale
  • From Google Colab to a Ploomber Pipeline: ML at Scale with GPUs
  • Discover What It Takes to Scale Innovation & Data Science

The Future of ChatGPT in Healthcare: Potential Applications

image-2

Artificial Intelligence has often been hailed as the ‘next big thing’ in technology. This idea gained more traction when ChatGPT introduced the world to the infinite possibilities of trained AI systems. Gradually, businesses identified the benefit of ChatGPT for easing daily operations and acquiring expert insights. Healthcare is no different.

The primary benefit of ChatGPT in healthcare is the significant personalization it offers. As big data takes centre stage in healthcare, the medical industry has more data than it can analyze. Using a digital program that assesses these large datasets in seconds is crucial for enhancing patient care.

But this is the tip of the iceberg. ChatGPT is believed to have more applications in healthcare as the use cases become clearer. Here is a brief look at some major uses of this AI chatbot in improving healthcare.

Upcoming ChatGPT Applications in Healthcare

A survey published in the Cancer Research, Statistics, and Treatment Journal highlighted the perception of the healthcare industry towards generative AI systems like ChatGPT. The overall results were mostly positive, as the following results were observed:

  • 52.2% of the respondents believed that ChatGPT would boost their careers
  • 72% of healthcare professionals stressed that ChatGPT would have a lasting impact on the human civilization

But how much weight do these claims carry? Here are a few upcoming use cases of ChatGPT in healthcare to give you a clearer idea:

Evidence-Based Recommendations

ChatGPT harbors the ability to look up digital records in seconds. It can provide concise information about latest studies and insights into medical interactions. In turn, this will help doctors make better decisions in the context of patient care.

The Future of ChatGPT in Healthcare: Potential Applications

Furthermore, it could use datasets to analyze the ideal treatment course. This would help doctors detect potentially life-threatening conditions earlier.

Here’s how it can provide comprehensive support to the healthcare domain:

  • Enable faster diagnosis by matching the patient’s symptoms with the relevant condition
  • Provide insights into the latest research in healthcare to power decisions

But the ultimate call will still belong to the doctor. ChatGPT is an assistive technology that will only help improve the quality of decisions and lead to better treatment.

Round-the-Clock Patient Care

Devices like the Apple Watch and Fitbit already comprise AI assistants that help track calories and measure heart rate. As the functions of ChatGPT expand with future updates, it is completely possible to embed it as a virtual assistant in medical devices.

The Future of ChatGPT in Healthcare: Potential Applications

For instance, it can ensure complete patient care in the following ways:

  • Tracking the body systems of the patient throughout the day
  • Predicting and tracking their next medical appointment
  • Alerting patients about their health and providing updates

While it might not be affordable to hire a 24/7 nurse, AI systems can ensure comprehensive patient care at a lower cost.

Reducing Manual Errors

Did you know that out of all patients undergoing surgery in a year, 3% of them experience a problem during surgeries?

Even more astounding is the fact that 54% of these incidents are preventable.

But with ChatGPT in the medical domain, it would be easier to spot and solve these errors. Here’s how it can help:

· Automate the error-reporting process for a more streamlined approach

· Assess datasets to check for the common errors

· Chart out customized patient safety initiatives to reduce errors

Plus, it can offer a simulation for medical students and provide real-time advice. ChatGPT promises to create a culture centred around patient safety and reduce manual errors.

Medication Management

It can be challenging for patients to keep track of their dosages. Individuals taking several medications at once may find it especially more difficult. Incorrect dosage can lead to severe complications that might need immediate medical intervention.

The Future of ChatGPT in Healthcare: Potential Applications

To prevent such cases, ChatGPT can perform the following functions:

  • Track medication dosages
  • Intimate patients about unfavorable drug interactions

Provide constant reminders to take the appropriate medicine

Sorting Volunteers for Clinical Trials

Clinical trials form an important step in the drug approval process. Hence, it is vital to ensure that these medicines are tried on the intended audience to produce accurate results.

ChatGPT can analyze thousands of datasets in a few seconds. This is useful to sort out volunteers who match the eligibility criteria of the clinical trials.

Summing Up

ChatGPT in healthcare promises to change the landscape of the medical domain and make it easier for doctors to handle patient care. It is a valuable tool to assist medical professionals and empower them to make faster diagnoses and better decisions for improved medical practice.

As Open AI develops ChatGPT and makes it more equipped to handle various applications, the medical industry might be one of the first adopters of this technology. But it remains to be seen how AI is adapted to suit medicine and healthcare in general.

AI As A Catalyst For Financial Success In ASCs: Unlocking Revenue Potential

AI as a Catalyst for Financial Success in ASCs

Ambulatory surgery centers face unique financial challenges in the fast-paced healthcare industry. With AI, ASCs can unlock untapped revenue potential. AI revolutionizes revenue cycles, optimizes billing processes, and drives significant financial growth in ASCs.

Healthcare is slower to adopt new technologies than manufacturing and retail. In our blog “Must Have Medical Practice Technologies to Boost Revenues,” we discussed the essential client experience-impacting technologies for modern medical practices: EHR, robust practice website, patient portals, and RPM or remote patient monitoring. AI’s role in medical billing and claims collections continues the tech in healthcare theme.

Clinical settings have embraced AI. You’ve probably heard about AI’s impressive cancer, retinal, and COVID-19 diagnosis results. Did you know that medical billing companies and payers use AI in claims collection and revenue cycle management (RCM) to improve efficiency and accuracy?

What Is AI, and How Can It Benefit the Healthcare Revenue Cycle?

Medical billing is one of the most complicated administrative tasks in a medical practice. Medical billing can only be scaled without technology by hiring highly trained billing staff, which is difficult. Many healthcare entrepreneurs and medical billing companies are investing in AI-powered medical billing software to streamline and reduce human dependence.

AI in medical billing improves data collection, analysis, and decision-making throughout the transaction-centric RCM process, boosting practice revenues and patient engagement. AI improves medical billing and revenue collection.

Two-thirds of hospitals and health systems use AI for revenue cycle management, according to Change Healthcare. 72 percent use AI for eligibility/benefits verification and 64 percent for payment estimation (likely due to the No Surprises Act). AI affects revenue cycle management (RCM) beyond these areas. The National Association of Healthcare Revenue Integrity (NAHRI) 2022 State of Revenue Integrity survey lists AI-enabled RCM products and functions in charge description master (CDM) maintenance, charge capture, denials management, payer contract management, physician credentialing, and claim to audit.

AI can analyze claim denial trends and warn HIM professionals before billing. Pre-bill review and correction are possible. AI dashboards that track denials by type or payer can help reduce repeat denials and improve workflows and education. AI can help self-pay patients find financial aid and provide price transparency.

AI-Enhanced Revenue Cycles

Role of AI in Prior Authorizations

The Council for Affordable Quality Healthcare, Inc. (CAQH) reported that manual authorizations cost $4.31 more in 2019 than in 2018. Automating prior authorizations reduces transaction costs to $2.11 and processing time to four minutes. Fully electronic prior authorizations could save billions in healthcare administration.

Prior authorizations are the best healthcare AI use case because they are transactional. Real-time analytics, machine learning (ML), and payer follow-ups are used in authorizations.

AI informs providers about payer approval for medical treatment, making decisions easier. Real-time reports show patients’ financial clearance status and next steps. Accuracy reduces claim rework and increases revenue by capturing every dollar.

Role of AI in Patient Eligibility

The U.S. government has required patient care estimates for two years. In healthcare, buyer cost estimates are complicated. The manual patient-cost calculation requires practice staff to check many systems with limited precision. AI-powered software checks patient eligibility in real-time using contracted rates and patient-specific costs. The AI software uses patient data from the practice to improve patient eligibility estimates.

AI-powered patient eligibility software estimates out-of-pocket costs before patients receive care, so they know what they owe. It reduces last-minute cancellations due to treatment costs, which pleases providers. The medical practice can increase point-of-service collections by discussing patient payment options in advance.

Role of AI in Claims Tracking

AI is also tracking insurance claims. Medical billers manually check claim statuses on multiple payer websites. RPA is now mimicking payer website logins. The medical billing software alerts the collector if a claim is denied or needs intervention. Payment-approved claims never reach the collector.

RPA robots ping or call payer firms to check claim denial appeals and other statuses for software companies. Payers increasingly use robots to respond.

Some AI claims processing software provides information on the likelihood of a claim being rejected before it is submitted to the payer, allowing practice staff to scrub claims before submission and speed up claim submissions.

Change Healthcare and ENGINE Insights surveyed 200 revenue cycles, IT, finance, and C-suite decision-makers about AI in healthcare. Most respondents intend to use AI in RCM extensively in the coming years. Prior authorization (68%) and payment amount/timing estimation (62%) will lead RCM AI applications by 2023.

AI-based predictive analytics

AI-powered predictive analytics is a game-changer for Ambulatory surgery center optimizing revenue cycles. AI algorithms can predict revenue trends, identify bottlenecks, and provide actionable insights by analyzing historical data and patterns. ASCs can proactively address financial issues, identify areas for improvement, and make informed decisions that boost their bottom line.

AI-Powered Billing

AI can speed up and reduce errors in billing. AI-enabled billing workflows reduce coding errors and ensure accurate documentation. Ambulatory surgery centers can reduce revenue leakage and streamline billing by using AI.

Should you invest in AI?

Healthcare practices should make an investment in AI, but do you have the budget to develop your own AI-based software? Also, it requires a lot of expertise & time, so should you invest is a real question because all this revenue betterment is only possible if you do it today while not making a dent in your pocket.

So, what’s the solution? Third-party billing companies have it all. They developed an AI-based EHR-based medical billing solutions for ASCs that leverage the power of AI for just 399$ monthly. Also, you do not have to make any investments and wait. So don’t DIY rather outsource & leverage its benefits.

Can those with AI expertise be left behind?

dog-g2358a33e1_1280

I read an interesting post from NVIDIA CEO Jensen Huang who said in a commencement note for university students in Taiwan that, “Those without AI expertise will be left behind.”

I totally agree with this – and the need to work with AI, but there is a caveat

I think over the last six months since the launch of GPT- the world of AI has moved so fast that even those WITH AI expertise could be left behind if they do not skill up.

However, the good thing is AI and low code make it more inclusive to learn AI.

A low-code development platform provides a development environment used to create application software through a graphical user interface. A low-coded platform may produce entirely operational applications, or require additional coding for specific situations. Low-code development platforms can reduce the amount of traditional time spent, enabling accelerated delivery of business applications. A common benefit is that a wider range of people can contribute to the application’s development—not only those with coding skills but require good governance to be able to adhere to common rules and regulations. LCDPs can also lower the initial cost of setup, training, deployment, and maintenance (wikipedia).

Now, it gets interesting when we combine Generative AI with Low code.

The integration of generativeAI and low code is interesting because it changes the developer workflow and more importantly, it democratises the application development workflow paving the path to many more people to become developers.

We are evolving rapidly to the idea that you can imagine a solution and describe it in natural language – and you can then generate that solution

The primary interface to the LLM is not coding but rather prompt engineering. This takes us to the realm of LLMs as a true application development platform further democratizing development and enabling even more people to create innovative solutions through natural language. One of the biggest impacts will be on the UX. We can start with the UX and turn drawings, images and Figma documents into app user interfaces (UIs). AI and low code with transform the way applications are designed. Once the UX is designed, using low code tools like the Copilot, Power Apps, Power Virtual Agents, and Power Automate (in case of azure), the application can be generated. Thus, AI can be integrated across the stack for example:

a) Generating tables from data structures

b) Automate workflows

c) Develop chatbots

d) GPT enabled data exploration and

e) Responsible AI

To conclude, on an optimistic note, once again to another reference from Nvidia,” AI means everyone can now be a programmer” Nvidia chief says.

References:

https://www.reuters.com/technology/ai-means-everyone-can-now-be-programmer-nvidia-chief-says-2023-05-29/

https://www.bloomberg.com/news/articles/2023-05-28/nvidia-ceo-says-those-without-ai-expertise-will-be-left-behind?leadSource=uverify%20wall
https://pixabay.com/photos/dog-return-domestic-animal-lawn-7032262/

It’s Time to Oust The Imitation Game

The imitation game, known better by the name ‘The Turing Test’ seems to be obsolete now. Crafted by computer scientist extraordinaire Alan Turing in 1950, this test has long been a rule of thumb to gauge the impact of AI on humans. The test works by putting a human evaluator in charge of distinguishing whether a set of responses is being provided by a computer or a human. This has been largely able to determine the efficacy of a human-like AI algorithm, but is now falling to the wayside.

A new research, based on the world’s largest Turing test, shows that AI algorithms have advanced to the point where it simply does not work. To his credit, Turing predicted that in 50 years time, computers would play the imitation game so well that an average interrogator would have no more than 70% chance of guessing whether it was an AI or a human. In other words, AI can fool 30% of the people.

With modern algorithms, AI researchers have found that this number hovers around Turing’s prediction, with 68% guessing their partners correctly when talking to humans. However, in situations where the test-takers were facing an AI bot, users only guessed it right 60% of the time. This shows that 40% of people did not know that they were talking to an AI agent, showing that it is now possible for AI to fool humans.

The World’s Largest Turing Test

AI21 Labs, a company offering NLP and other AI solutions, recently conducted a game called ‘Human or Not?’. This app is a web version of the Turing Test where users can chat with someone for 2 minutes. After this, they are given a chance to find out whether they were talking to a human or a bot. This gamified test went unexpectedly viral, garnering over 2 million conversations between humans and bots.

Some key insights from the experiments found that humans were able to find it easier to identify a fellow human. Interestingly, India had the lowest percentage of correct guesses, sitting at 63.5% compared to France’s 71.3%, which was the highest. Moreover, younger age groups tend to guess better than older ones.

The game pitted human users against leading LLMs like GPT-4 or Jurassic-2, albeit with their own quirks and tricks to throw off human users. The researchers brought up an interesting point, stating that many users’ perceived limitations of large language models stemmed from their experience of using ChatGPT and other similar interfaces. As such, they added an additional layer of complexity to their algorithms to make it harder to guess.

For example, AI21 Labs made use of the assumption that bots usually do not make grammar mistakes or use slang. The researchers purposely trained their models to make common spelling mistakes and use trendy terms to make them seem more human. Similarly, humans also felt that asking or answering a personal question was something algorithms struggle with. However, the bots were able to reference their training data to easily come up with personal stories, further fooling the humans.

Some other human biases against AI include assumptions that bots aren’t aware of current events, that they aren’t capable of philosophical or ethical queries, that they are extremely polite (to a fault), and that they are incapable of answering certain sensitive questions. Interestingly, even in the 2-minute time span given per attempt, users tried to jailbreak the LLM using methods like DAN.

While this not only sheds light into the capabilities of modern LLMs, it also revealed some limitations of the Turing Test itself. By phrasing their sentences in tricky manners that bots would not understand, some test-takers were able to weed out the humans from the bots. This is a limitation of the test, which only tests capability in natural language.

The research clearly shows that the test is becoming obsolete, thanks to the advancements of LLMs and knowledge of the biases humans have against AI. However, the scientific community has also offered other, more capable benchmarks that provide a more comprehensive picture of what AI is capable of. These tests are also up to date with the capabilities of modern AI.

Thinking Beyond Turing

Gary Marcus, an American psychologist and AI expert, wrote in the past about an algorithm called Goostman, which was the first AI to pass the Turing test. Speaking about the importance of this test, he stated, “The real value of the Turing Test comes from the sense of competition it sparks amongst programmers and engineers.”

To this end, Marcus offered up his own version of the test, which is now known as the Marcus test. Reductively put, he stated that if an AI can watch an episode of ‘The Simpsons’ and tell the viewer when to laugh, it has passed the test.

However, the Lovelace Test 2.0, named after Ada Lovelace, the world’s first computer programmer, takes a different course. The AI can pass the Lovelace test if it can ‘develop a creative artifact from a subset of artistic genres deemed to require human-level intelligence’. Simply put, this means that AI agents which can create human-level art are considered to pass. By this logic, Midjourney has already passed this test, as an artist won a competition by using the image generation algorithm.

Many such stand-ins for the Turing test have been created, but have now been replaced by benchmarks, like Francois Chollet’s ARC. Instead of using a subjective human outlook to gauge the efficacy of an algorithm, ARC instead relies on reasoning and logic to find the capabilities of certain algorithms. Even as these methods are becoming more widely adopted, it seems that Turing’s original vision has fallen to the wayside.

While a thinking computer has not yet been created, AI has reached human parity, and, in some instances, surpassed humanity in certain fields. However, the creation of a true, fluid, generalised intelligence is still far away. Until then, we must find better ways of not only gauging an algorithm’s effectiveness, but also its humanity.

The post It’s Time to Oust The Imitation Game appeared first on Analytics India Magazine.

Artificial Intelligence: A Board of Directors Challenge – Part I

Slide1

This two-part series outlines the challenges and actions that the Board of Directors for organizations must address as they guide their organization’s responsible and ethical deployment of Artificial Intelligence (AI). Part one will cover mitigating the impacts of AI Confirmation Bias.

Leaders across various business, technology, social, educational, and government institutions are deeply concerned about the potential negative impacts of the untethered deployment of Artificial Intelligence (AI).

In the article “Experts are warning AI could lead to human extinction. Are we taking it seriously enough?” Dan Hendrycks, Executive Director of the Center for AI Safety, states: “There are many ‘important and urgent risks from AI,’ not just the risk of extinction; for example, systemic bias, misinformation, malicious use, cyberattacks, and weaponization.”

There is certainly an immediate need for global collaboration to create policies, regulations, education, and tools that stave off dire actions from nefarious actors and rogue nations who might leverage AI for misinformation, malicious use, cyberattacks, and weaponization.

But there are other equally dangerous risks from the careless application of AI, including:

  • Confirmation Bias from poorly-constructed AI models that deliver irrelevant, biased, unethical outcomes and impact longer-term operational viability.
  • Unintended Consequences from decision-makers making decisions without consideration of the second and third (and fourth and fifth…) order ramifications of what “might” go wrong.

This is where an organization’s Board of Directors must step up. The Boards across organizations are facing a challenging dilemma caused by the emergence of Artificial Intelligence (AI).

On the one hand, AI can bring compelling benefits to organizations by helping them more quickly exploit changes in their business and operating environments. The thoughtful deployment of AI can improve efficiencies in key business processes, policies, and use cases and reduce compliance, regulatory, and market risks. AI can help organizations uncover new sources of customer, product, service, and operational value while creating a differentiated, compelling customer experience.

On the other hand, AI can have adverse effects on an organization’s long-term viability and unintentionally expose them to compliance and regulatory risks. For instance, AI models that exhibit confirmation bias hinder an organization from attracting new customers or offering new products/services. Moreover, poorly designed AI models can impede necessary operational changes required to adapt to market, societal, regulatory, and environmental changes.

Let’s start this two-part blog series by covering what the Board of Directors needs to understand about Confirmation Bias.

Addressing AI Confirmation Bias

Confirmation bias is a cognitive bias that occurs when people give more weight to evidence supporting their existing beliefs while downplaying or ignoring evidence contradicting those beliefs.

AI model confirmation bias can occur when the data used to train the model is unrepresentative of the population the model is intended to predict. AI model confirmation bias can lead to inaccurate or unfair decisions and is of significant concern in fields such as employment, education, housing, healthcare, finance, and criminal justice, where the consequences of incorrect predictions can be destructive.

Creating AI models that overcome AI confirmation bias takes upfront work…and creativity. That work starts by 1) understanding the costs associated with the AI model’s False Positives and False Negatives and 2) building a feedback loop where the AI model is continuously learning and adjusting from the False Positives and False Negatives. For example, in a job application use case (Figure 1):

  • A Type I Error, or False Positive, is hiring someone the AI model believes will be successful, but ultimately, they fail. The False Positive situations are easier to track because most HR systems track the performance of every employee, and one can identify when an employee is not meeting performance expectations. The “bad hire” data can be fed back into the AI training process so that the AI model can learn how to avoid future “bad hires.” Easy.
  • A Type II Error, or False Negative, is not hiring someone the model believes will not be successful, but ultimately, that person goes on to be successful elsewhere. These False Negatives are hard to track and require creativity to capture False Negatives data. But tracking these False Negatives is necessary if we want to leverage them to improve the AI model’s effectiveness.
Slide2

Figure 1: AI Model Confirmation Bias

Another danger is that AI confirmation bias can feed upon itself, creating an echo chamber targeting the same customers and recommendations. AI Confirmation Bias can lead to a decay in your Total Addressable Market (TAM) because your AI models focus on the same customers and recommendations, missing out on opportunities to reach new customers outside the scope of your AI model’s knowledge (Figure 2).

Slide3

Figure 2: AI Confirmation Bias Can Lead to Shrinking TAM

The good news is that this problem is solvable. We can construct AI models that can continuously learn and adapt based on the effectiveness of the model’s predictions. We can build a feedback loop to measure and provide feedback on the effectiveness of the model’s predictions so that the AI model can get more accurate with each prediction (Figure 3).

Slide4

Figure 3: Constructing AI Model Feedback Loop

The Danger of Proxy Measures

Proxy Measures are indirect or substitute variables used to estimate or approximate a target variable of interest.

One deceptive way confirmation bias can sneak into our AI models is through proxy measures that approximate the variable we are trying to omit from the models. Proxy measures become a particular issue when dealing with protected classes[1].

For example, let’s say we are trying to remove the protected class of gender from AI model consideration. Unfortunately, there are many proxy measures in the data that can be indicators of gender (Figure 4).

Slide5

Figure 4: Identifying and Quantifying Proxy Measures

For example, one could easily detect the gender of my wife based on the types of movies watched (Romantic Comedies, Art Films), whether on a diet (always), retail outlet preferences (RealReal, J. Peterman), and favorite book authors (Anne Lamott).

To address the proxy measure problem, we can use knowledge graphs to identify potential “proxy” measures that correlate to the protected trait (Gender) we want to remove from the AI model training and inference data sets. We can calculate the correlation strength between the protected trait and any “proxy” measures to the protected trait to determine which “proxy” measures to remove from the AI Utility Function.

Summary: Board of Directors AI Confirmation Bias Challenge

In part one of this two-part series, we discussed what the Board of Directors needs to know to mitigate the impacts of AI Confirmation Bias.

In part two, we’ll discuss unintended consequences and conclude with a checklist of items that the Board of Directors needs to consider when advising senior management on mitigating the impacts of AI confirmation bias and the unintended consequences of management decisions.

[1] Protected classes include race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age (40 or older), disability, and genetic information

10 Jupyter Notebook Tips and Tricks for Data Scientists

10 Jupyter Notebook Tips and Tricks for Data Scientists
Image by Author

Whether you are a beginner or a data professional, you must have used the Jupyter Notebook and found out how easy it is for you to run Python code and visualize the output in a report format.

But what if I told you that you can improve your Jupyter development experience? In this post, we will be learning about 10 Jupyter Notebook tips for enhancing the productivity and performance of data professionals.

1. Keyboard Shortcuts

Keyboard shortcuts are important for performing repetitive tasks and saving time. You can learn about all of the default keyboard shortcuts by Help > Keyboard Shortcuts or pressing the H key.

The easiest and most popular way of accessing commands on the go is a command palette similar to VSCode. You can press Ctrl + Shift + P to invoke the command palette. It allows you to search and execute commands or scroll through all of the commands to discover the one you want to run.

10 Jupyter Notebook Tips and Tricks for Data Scientists
Gif by Author 2. IPython Magic Commands

You can access all of the IPython Magic commands within the Jupyter Notebook. These commands give you extra capabilities in executing the code.

For example, you can use %%time Magic command to display the cell execution time. In our case, it took the code 1.09 seconds to run 1000 iterations.

%%time    import time  for i in range(1_000):      time.sleep(0.001)
CPU times: user 10.2 ms, sys: 1.68 ms, total: 11.9 ms  Wall time: 1.09 s

You can learn about all of the available Magic commands by running the %lsmagic command or checking out the Built-in magic commands.

List of commonly used commands:

  • %env for setting environment variables.
  • %run for executing the Python code.
  • %store for accessing the variables in between multiple notebooks.
  • %%time gives the execution time of a cell.
  • %%writefile saves the content of the cell into a file.
  • %pycat shows the content of the external file.
  • %pdb is used for debugging.
  • %matplotlib inline for suppressing the output of the function in the final line.

3. Executing Shell Commands

You can run Shell and Bash commands within the Jupyter Notebook cell by using ! as shown below. It provides you extra ability to run Unix or Linux based commands and tools.

!git push origin

Most popular use of this command is to install Python packages on the go.

!pip install numpy

You can also install Python packages using Magic command %pip

%pip install numpy

4. Using LaTeX for Formulas

While creating the data analytics report, you need to provide statistical or mathematical equations, and Jupyter Notebook lets you render complex equations using Latex formulas.

Just create a Markdown cell and enclose your Latex formula with a dollar $ sign, as shown below.

$int frac{1}{x} dx = ln left| x right| + C$

Output:

10 Jupyter Notebook Tips and Tricks for Data Scientists 5. Installing Other Kernels for Jupyter Notebook

We all know about the Python Kernel, but you can also install other Kernels and run the code in any language.

For example, if you want to run the R programming language in Jupyter Notebook, you need to install R and install IRkernel within the R environment.

install.packages('IRkernel')  IRkernel::installspec()

Or, if you have installed Anaconda, you can simply run the command below in the terminal to set up R for Jupyter Notebook.

conda install -c r r-essentials

For Julia language lovers, I have created a simple guide on how to set up Julia on Jupyter Notebook.

6. Run Code from a Different Kernel

You can also run code from multiple kernels within Python Jupyter Notebook by using Magic commands like:

  • %%bash
  • %%html
  • %%javascript
  • %%perl
  • %%python3
  • %%ruby

In the example, we will try to run HTML code within the Python Kernel using %%HTMLMagic command.

%%HTML    <html>    <body>    <h1>Hello World</h1>    <p>Welcome to my website</p>    </body>    </html>

Output:

10 Jupyter Notebook Tips and Tricks for Data Scientists

Similar to !, you can run Shell script using %%script, which allows you to run all of the kernels installed on your machine. For example, you can run an R script.

%%script R --no-save  print("KDnuggets")

Output:

> print("KDnuggets")  [1] "KDnuggets"  >

7. Multicursor Support

You can use multiple cursors for editing multiple variables and syntax or adding multiple lines of code. To create multiple cursors, you need to click and drag your mouse while holding down the Alt key.

10 Jupyter Notebook Tips and Tricks for Data Scientists
Gif by Author 8. Output Images, Videos, and Audio

You can display Images, Videos, and Audio without installing additional Python packages.

You just have to import IPython.display to get the Image, Video, and Audio functions. It is quite useful when you are dealing with unstructured datasets and machine learning applications.

10 Jupyter Notebook Tips and Tricks for Data Scientists
Image by Author 9. Processing Large Data

You can process and query large datasets by using the IPython Parallel library. It is the collection of CLI scripts for controlling clusters of IPython processes built on the Jupyter protocol.

Moreover, you can use the PySpark session with the sparkmagic command.

Check out the example from the sparkmagic repository.

%%spark -c sql -o df_employee--maxrows 5  SELECT * FROM employee

Output:

	age	name  0	40.0	abid  1	20.0	Matt  2	36.0	Chris

10. Sharing Notebooks

Sharing the reports or code source with outputs is important, and you can do it in multiple ways:

  1. Convert Notebooks into HTML files using the File > Download as > HTML.
  2. Save Notebooks as PDFs using File > Download as > PDF.
  3. Save Notebooks as Markdown File > Download as > Markdown.
  4. Create blogs using Pelican.
  5. Upload the .ipynb file to Google Colab and share it among colleagues.
  6. Share the Notebook file with the public using GitHub Gits.
  7. Host your file on the cloud or the external server and use nbviewer to render the Notebook.

I hope you found my list of 10 tips on Jupyter Notebook helpful. If you have any additional suggestions or tips for Jupyter Notebook that you'd like to share, please feel free to mention them in the comments below. Thank you for reading.
Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • Streamlit Tips, Tricks, and Hacks for Data Scientists
  • How to Setup Julia on Jupyter Notebook
  • Quick Data Science Tips and Tricks to Learn SAS
  • 12 VSCode Tips and Tricks for Python Development
  • Tips & Tricks of Deploying Deep Learning Webapp on Heroku Cloud
  • MathWorks Deep learning workflow: tips, tricks, and often forgotten steps

Data Science Hiring Process at MediBuddy

Founded in 2015 by IIT graduates, Satish Kannan and Enbasekar Dinadayalane, MediBuddy, a prominent player in India’s digital healthcare sector, offers online and offline consultations, doctor appointments, clinics, prescription delivery, home lab tests, and more.

With over 2000 employees, MediBuddy covers primary and tertiary care, including surgeries and second opinions, and offers preventive healthcare services. The portal boasts an alliance grid comprising over 90,000 physicians, more than 7,100 medical facilities and clinics, over 4,000 testing and examination hubs, and over 2,500 pharmaceutical outlets.

MediBuddy is valued at $407 million as of Dec 19, 2022, with backing from India Life Sciences Fund III, Quadria Capital, InvAscent and Lightrock India, among others. In February 2023, MediBuddy acquired ‘vHealth by Aetna’ to enhance MediBuddy’s market dominance and strengthen its overall presence.

MediBuddy also expanded in rural India by acquiring Clinix, a healthcare provider with an extensive network across 20 tier-two and tier-four cities. This acquisition enabled MediBuddy to reach and serve the healthcare needs of people in remote areas of India.

MediBuddy is currently hiring data scientists (level 2 and level 3).

Inside MediBuddy’s Data Science Team

“We have a diverse tech team consisting of over 200 people, with a unique multidisciplinary data team that includes various professionals such as doctors, nurses, and clinicians besides data scientists, data engineers, analysts, product managers, lab pathologists, lab technicians, and clinical experts,” Dinadayalane told AIM in an exclusive interview.

The team also includes data annotators, who are nurses and clinical staff responsible for tagging and cleaning the data.

“While we have doctors who can code, we don’t have engineers who have transitioned to become doctors,” he added jokingly.

Over the years, MediBuddy addressed various problems using AI and analytics. In 2015, the digital healthcare startup launched its app, placing digital healthcare at the forefront. Initially, they focused on online consultations, which became essential during the COVID-19 pandemic.

One of the primary impacts of the data science team can be seen in Clara, MediBuddy’s proprietary data science system developed by the company. It includes a chatbot for streamlined patient-doctor interactions, a clinical decision support system for doctors, quality control measures for improved patient experience, and an AI-based database engine for prescription accuracy. Clara also assists in providing additional healthcare services like medicine delivery, lab tests, and surgeries.

Another problem that the company is solving through AI is handwriting recognition from prescriptions which often turns out to be a deciding factor in getting the right medication. They are exploring technology solutions like AWS TextTrack and OCR-based technologies to automate interpretation for this.

Another priority for MediBuddy is quality control, with robust data-based analytics and quality control engines. These engines analysed audio, video, and text data to assess interactions between doctors and patients. Critical insights were flagged and forwarded to internal clinical experts for improvement.

Taking into account India’s rich linguistic diversity, with more than 350 languages spoken, MediBuddy has undertaken a thoughtful approach to cater to the larger population who may not be proficient in English. They have implemented word embeddings to effectively tackle the unique challenges encountered in Indian English, such as the blending of Hindi words within English text.

Moreover, the MediBuddy team has dedicated efforts towards constructing models that are capable of language classification, named entity recognition, and establishing meaningful connections across various languages, leveraging extensive datasets. They recognise the significance of language teams in accurately tagging and comprehending subtle nuances within the linguistic landscape.

The company also created a doctor assistant engine, integrating AI-based auto-suggestions and recommendations. Based on chat conversations, the system suggests potential diagnoses, additional questions, and appropriate medications for doctors to review and validate. However, no decision is made without a doctor. It is just for suggestions to empower doctors to make well-informed decisions and prevent prescription errors. To address this set of unique challenges, MediBuddy also has a diverse range of tools and frameworks.

On the data science side, Python is primarily used as the scripting language. The data science team employs various NLP models as their work involves chat, call, and video consultations. Speech processing, audio and video processing, and text processing are key components, as chatbots are built using NLP engines. Additionally, the team works on video quality and conversation smoothness. In terms of data modeling, the team explores emerging models like GPT-2 and GPT-3 and SVM (support vector machines) for NLP.

MediBuddy uses Superset, an open-source analytics platform, and R for data analytics tasks. Airflow is employed for data engineering and managing data pipelines. Cloud platforms such as AWS, Google, and Microsoft are used to address specific AI and healthcare requirements, including the use of TensorFlow.

Read more: Data Science Hiring Process at Meesho

Interview Process

Candidates have introductory conversations to assess their fit for the role. This is followed by two or three rounds of technical interviews, consisting of in-person or take-home assignments r depending on the role and problem being addressed. For higher-level positions, there may be interviews with managers to evaluate leadership potential.

“Our data science work is not limited to a separate R&D team but is integrated into everyday work across various healthcare services,” said Enbasekar .

When it comes to the skills that the company looks for in data science candidates, there are two core skill sets. Firstly, a strong understanding of fundamental modeling concepts is important. This includes expertise in mathematics, algorithms, and data science fundamentals. Secondly, candidates should have the ability to work with multiple functional teams and have a customer-oriented approach. Soft skills such as effective communication, teamwork, and customer understanding are highly valued.

From candidates, the company expects a willingness to learn and adapt to new technologies and challenges, as the field of data science is constantly evolving. MediBuddy values individuals who can bring solutions to life by collaborating with multiple teams and taking full ownership of projects.

Work Culture

Enbasekar describes his company’s work culture as being focused on healthcare, with a strong sense of pride and humility about the impact they can make in the field. “We have a high-performance environment with continuous learning and problem-solving, driven by the desire to improve and serve our customers, doctors, and hospital partners”, he opined.

End-to-end ownership is a key cultural value, where employees go beyond their core roles to ensure value for the end user. MediBuddy also encourages open interaction and collaboration across teams, promoting a non-hierarchical structure where individuals freely seek help and get things done.

Why Join MediBuddy?

Joining Medibuddy offers the opportunity to have a significant impact on healthcare at a large scale.

“We address unique technological challenges in healthcare, providing a valuable learning experience. Operating in both B2C and B2B sectors, MediBuddy offers exposure to various aspects of the healthcare industry,” he concluded.

So if you want to contribute to solving important challenges and ultimately benefit millions of people, making a meaningful difference in healthcare, MediBuddy is the right place for you. Check out their careers page here.

Read more: Data Science Hiring Process at ZS

The post Data Science Hiring Process at MediBuddy appeared first on Analytics India Magazine.

5 Best Practices for Data Science Team Collaboration

5 Best Practices for Data Science Team Collaboration
Image by Author

A data science project is made up of a wide range of skills, with different team members playing different roles. Everybody has their set of skills and responsibilities, which all play a big role in collaborative technical work.

However, we’re still facing the backlash of the global pandemic and the rise in people continuing to work from home. Naturally, that will cause a shift in the way people work and operate.

So what can data science teams do to collaborate more effectively? Let’s look into it.

Ensure Models Make it to Production

It's a known fact that there are so many models that have taken time, energy and money to build, but they rarely make it into production. According to VentureBeat AI, 87% of data science projects never make it into production. That’s a pretty high number! But why is it so high?

This is due to the data science element of the business and the actual aim of the business do not connect. And the main reason why they do not connect is that there is a gray area of what the data science team needs to produce to ensure they are meeting the business's needs.

Better communication between the data science team and the decision makers of a business will allow members of the data team to effectively produce what is required. This can be done by answering the following questions:

  1. What is the business problem?
  2. Is it possible to solve this problem?
  3. Will the business adopt solutions from the data insight?

Answering these three questions allows the data science team to have an in-depth understanding of what needs to be done.

Document Projects

A data science project consists of people with different roles, from data scientists to data engineers, product managers, IT admin, and more. When working on a project, documenting everything you do provides everybody in the team with a clearer understanding of the process of the project, and what needs to be done next.

Data science projects will not always be successful, but documenting your every move allows you to learn lessons from the project and what to do next time to ensure success.

Two rules that you should take with you when documenting projects are:

  1. Although documenting is helping to collaborate with your employees now, it is also collaborating with future employees.
  2. Walk before you run. Operate your data science project like a research paper. Don’t rush to produce the end product, but build an end product that is effective and successful at meeting the goal of the business.

Knowledge Sharing

By documenting everything, you are also providing knowledge sharing across the company. The data science team holds a lot of ??valuable assets in the company. One of the biggest challenges a lot of companies face is the multiple production of the same work or resource.

Creating a knowledge share where everyone can have access to information such as code, projects, and models will save your organization a lot of time not reproducing the same thing twice.

Knowledge sharing works hand in hand with documenting your projects, as employees should be able to see what data sources the data scientist used, the modeling approach, the environment versions, and more.

Version your Work

Now to get a bit more into the technical elements of data science projects. The majority of data is stored as flat files or can be accessed through relational database systems. However, the biggest challenge that data science teams face is when members of the team download the raw data and produce their work locally without pushing the intermediate data versions back for other members of their team.

Unfortunately, other members of the data science team will complete the same work, causing a repetition of workloads. Sharing your work is very valuable, as it gives your coworkers a chance to take advantage of the work you have done, so they can work from that.

All your work should be versioned and pushed back to a non-local system, allowing others to see the changes and pull the changes for them to work on it.

You can ensure this by:

  1. Use a shared server for your team.
  2. Automation tools that push intermediate data files back to the appropriate location.
  3. Make use of integration tools such as Slack and GitHub, so you can get notified of changes being made.

Data Pipeline

Data pipelines allow for the flow of data science projects as the data processing elements are connected in series, where the output of one element is the input of the next one. Rather than spending extra time running two or more commands to go from your raw data to your end result, using a data pipeline allows you to see the entire transformation with a single command.

Not only will this reduce the amount of time spent trying to rebuild your project from scratch, but it also provides you with a structural understanding of your data transformation.

Wrapping it up

Although there are other practises that you can use to ensure an even better data science team collaboration method. However, these 5, if done correctly and effectively will allow your team to progress in a more effective and productive manner.

Want to learn about automating your data science workflow, have a read of this: Automation in Data Science Workflows
Nisha Arya is a Data Scientist, Freelance Technical Writer and Community Manager at KDnuggets. She is particularly interested in providing Data Science career advice or tutorials and theory based knowledge around Data Science. She also wishes to explore the different ways Artificial Intelligence is/can benefit the longevity of human life. A keen learner, seeking to broaden her tech knowledge and writing skills, whilst helping guide others.

More On This Topic

  • Context, Consistency, And Collaboration Are Essential For Data Science…
  • The Maslow’s hierarchy your data science team needs
  • Six Tips on Building a Data Science Team at a Small Company
  • How to Build a Data Science Enablement Team: A Complete Guide
  • Integrating ChatGPT Into Data Science Workflows: Tips and Best Practices
  • Where Collaboration Fails Around Data (And 4 Tips for Fixing It)

Australia launches consultative review of AI

chatgpt on phone

Amid increasing global alarm that artificial intelligence (AI) is poised to cause irreparable harm to human society in the near future, the Australian government headed by Liberal party leader Anthony Albanese has launched its own review of this rapidly evolving technology.

Industry and Science Minister Ed Husic has released two papers that kickoff an eight-week consultative process that seeks to get input from a variety of stakeholders on a new framework.

Also: Today's AI boom will amplify social problems if we don't act now, says AI ethicist

One is a 'rapid response' report commissioned by the National Science and Technology Council (NSTC) that explores the opportunities and risks posed by generative AI.

This analysis has become expedient because of the speed at which existing tech companies in Australia and globally are pivoting to AI, and the pace with which AI is seeping into almost every industry.

It's a transformation that has triggered increasing concern about AI's intrusiveness or tendencies toward bias as well as concerns about truthfulness and 'hallucinations'.

Also: Today's AI boom will amplify social problems if we don't act now, says AI ethicist

For instance, Australia's leading medical association has called for regulations that will keep a watch over AI in healthcare — a sector that is vulnerable to racial or age-related bias, with potentially catastrophic consequences.

The paper looks at a variety of similarly disruptive scenarios that AI could foment, including rampant misinformation and subsequent polarization of the population, along with widespread job displacement and increasing inequity.

The next stage is a consultation paper that examines what other nations around the world are doing to address AI regulation.

Toward a framework

The Australian government is looking closely at an almost identical review that the U.K.'s antitrust agency Competition and Markets Authority announced that it was launching, as well as an AI-specific Act that the European Union is deliberating.

The government will also no doubt look at recent actions of the U.S. government, which launched similar public assessments of all major AI generative systems, including by communities of hackers, data scientists, independent community partners, and AI experts.

Also: How to use ChatGPT: Everything you need to know

"Given the developments over the last, in particular, six months, we want to make sure that our legal and regulatory framework is fit-for-purpose, and that's why we're asking people, either experts or the community, to be involved in this process, the discussion process, with the papers that we've put out, to let us know what their expectations are and what they want to see," said Husic.

"We need the framework right, that people are confident that it's working in favor or for the benefit of communities — it's really important," he added.

The Australian government's consultation paper emphasizes that AI still has to operate under the aegis of existing rules in the country, as they currently are in the U.K. or Europe, which range from those that are sector-specific (such as healthcare and energy) or those that are general to all industries (privacy, security, and consumer safeguards).

The deliberation will involve a consideration of whether to simply strengthen each sector accordingly or to introduce specific AI legislation, or both.

Also: AI is more likely to cause world doom than climate change

Husic mentioned that if there are "high-risk" areas that emerge from this process — he singled out a hypothetical abuse of facial recognition as an example — that his government will be proactive in addressing these concerns seriously within an emerging framework.

"We want people to be confident that the technology is working for us and not the other way around," Husic added. "Governments have got a clear role to play in recognizing the risk and responding to it, putting the curbs in place."

Artificial Intelligence