Рубрика: AI

Data engineers are professionals who specialize in designing, implementing, and managing the systems and processes that transform raw data into usable and trusted information. They play a crucial role in ensuring data integrity and accessibility for downstream analytics and machine learning applications. If you’re interested in entering this field but worry it might seem impossible or take too long to achieve, rest assured.

Stepping into the role of a data engineer is far more attainable than you might imagine. With the right support system in place—a knowledgeable mentor and access to quality learning resources—and with dedicated effort, you can successfully navigate this journey. In this article, we’ll address the most pressing question: How long does it truly take to master the art of data engineering? Read along to find out.

Phases of mastering data engineering

Learning data engineering involves traversing through several distinct phases, each varying in duration and complexity.

Phase 1: Introduction

In this initial phase, you need to acquaint yourself with foundational tools and technologies in data engineering. They explore basic Extract, Transform, Load (ETL) concepts and data pipelines, gaining a fundamental understanding of data movement and transformation.

Phase 2: Establishing foundations

The second phase involves building a robust base. You must delve deeper into different types of databases: relational (e.g., IBM DB2, PostgreSQL) and NoSQL (e.g., MongoDB, Cassandra). and data modeling, grasping how data structures influence system design. Also learn SQL for data querying, manipulation, and basic database management. With this you can gain practical experience with ETL frameworks like Apache Spark, honing your data processing skills.

Phase 3: Intermediate proficiency

Progressing to intermediate proficiency entails practical application. You need to start tackling real-world projects and apply your knowledge in practical contexts. Additionally, you should also master advanced data storage solutions such as HDFS (Hadoop Distributed File System) and cloud-based platforms like Amazon S3. Gain proficiency in popular file formats such as Parquet, Avro, and ORC commonly utilized in data engineering.

Phase 4: Advanced exploration

In the final phase delve into advanced topics and deepen your SQL expertise with topics like window functions, indexing, and query optimization. Explore intricate aspects of data engineering, such as advanced ETL orchestration with tools like Apache Airflow. Get acquainted with leading cloud platforms such as AWS, GCP, or Azure, and their data-centric services. Learn about big data processing tools like Hadoop, Hive, and HBase for scalable data management. Continue to develop an understanding of distributed computing and parallel processing mechanisms, specializing in areas like real-time processing and streaming data management.

Continuous learning and growth

Continuous learning and growth are vital for mastering data engineering. As technology evolves rapidly, staying updated with emerging tools and trends is crucial. Active engagement within the data engineering community fosters knowledge exchange and skill enhancement. However, theoretical knowledge alone isn’t sufficient; practical application through personal projects solidifies understanding and nurtures adaptability. Practical experience is paramount in mastering data engineering. Moreover, consider enrolling in a data engineering certification program to keep upskilling yourself. This combination of hands-on practice and formal education will greatly enhance your proficiency and confidence in the field.

Factors affecting learning duration

Several factors influence the duration of learning data engineering. Individuals with prior technical experience may grasp concepts more quickly, while the quality of learning resources and instruction also plays a significant role. Consistency in practice, engagement, and project complexity are additional factors impacting learning speed.

Final thoughts

It’s essential to recognize that these phases’ durations are approximate and may vary based on individual backgrounds, learning pace, and engagement levels. The journey’s significance lies not only in the duration of each phase but also in the cumulative expertise gained. Embracing continuous learning and remaining open to new developments is crucial for success in the ever-evolving field of data engineering.

A recent study explored the capability of AI, particularly LLMs like ChatGPT (GPT-4), Google Bard, and Bing, in understanding and responding to emotional and psychological needs of patients during therapy sessions and compared the results to human psychologists.

The research involved 180 male psychologists from King Khalid University in Saudi Arabia, who were grouped based on their educational levels into either bachelor’s or doctoral students.

Each participant, both human and AI, responded to 64 different scenarios presented through the Social Intelligence Scale. Interestingly, ChatGPT-4 outperformed all participating psychologists by scoring 59 out of 64 on the scale.

These advancements hint towards a future where, all thanks to AI, basic mental healthcare may be right in our hands; always by our side when we are struggling to get out of bed in the morning or up at 2 am.

OpenAI recently introduced a ‘memory’ feature for all its ChatGPT Plus users. It allows the chatbot to retain information more permanently, enabling it to enhance its responses by remembering details and learning from conversations.

This latest development might prove to be another feat for AI when it comes to donning the role of a therapist. By providing that missing ‘context’ to LLMs and eliminating the need for a user to repeat oneself in every new conversation, this can enhance the efficiency of future interactions.

That being said, I would have loved to have some sort of life coach powered by AI whom I could talk to when humans were failing to be present for me.
There’s a place for both but one definitely does not replace the other!

— Damien Terwagne (@ropesandhopes) April 30, 2024

For example, you can vent about someone or something that annoys you and won’t have to introduce that person or thing again to the chatbot if you’ve already done that before. It might just feel like venting out to a friend who knows what you’re talking about without needing new references or introductions every time.

Can AI Really Solve Mental Health Problems?

According to a 2017 Lancet Report, close to 200 million Indians, which is one out of every seven individuals in India, suffered from mental disorders of varying severity. According to the McKinsey Health Institute’s 2023 survey, India topped the rank in workplace burnout, with 59% employees reporting symptoms.

With only 0.7 psychiatrists for every 100,000 people in India, lack of access to mental health resources, expensive therapy, and excessive stigma associated with it, the road to availing mental healthcare in India is filled with hurdles. This is where technology-based interventions can help.

AI has been gaining traction in the mental health field, presenting fresh avenues for diagnosis, treatment, and prevention of mental health disorders. Its potential lies in its capacity to deliver personalised and cost-effective solutions to individuals, especially in underserved or remote areas, who need it the most.

An AI Therapist in Every Pocket

Imagine a world where our devices, from smartphones to smartwatches, become silent guardians of our mental health. AI is making this a reality.

AI-driven chatbots or virtual assistants can offer immediate solace, suggest coping mechanisms, or simply lend a virtual ear during midnight moments of anxiety or despair. It’s like having a supportive friend available whenever one needs to talk, enhancing the quality of life for countless individuals.

Unlike traditional therapy, which is limited by session times and availability, the 24/7 support that AI tools provide can be incredibly reassuring. The anonymity it promises also helps people open up without fear of judgment.

Source: Reddit

Many individuals hesitate to seek help due to stigma, privacy concerns, or logistical issues. AI applications, accessible via smartphones and computers, provide a private, stigma-free environment for people to begin addressing their mental health needs.

Currently, it might not be tailor-made for our problems or offer long-lasting solutions. Still, AI is democratising access to mental well-being by providing instant solutions for temporary problems without burning a hole in the pocket. This type of support can be particularly useful for individuals who may not have access to traditional mental health services.

One of the most significant benefits of AI in mental health lies in its capacity for early detection and diagnosis of mental health disorders.

AI algorithms can sift through large amounts of data, scanning everything from tweets, data from wearable devices like smartwatches, speech patterns, text analysis, and facial expressions to subtle changes in digital behavior to identify potential mental health challenges long before they spiral out of control.

This early detection can help mental health professionals intervene before the condition worsens. The use of AI/ML to assess mental health status and suicidal tendencies through apps like Wysa, Woebot, and Replika among others presents a new frontier in the battle against mental health disorders.

Finally, leveraging AI can help mental health professionals automate tasks, enhance their understanding of the causes and origins of complex disorders. Besides, this can formulate personalized treatment strategies based on a patient’s medical history, inclinations, and therapy responses, and provide better care and support to their patients, leading to better outcomes.

What Next?

As AI advances, its role in enhancing accessibility and reducing mental health stigma is expected to expand. The ongoing refinement of more sophisticated, empathetic, and user-friendly AI tools will further remove barriers to care.

For example, Hume AI, an Empathic Voice Interface (EVI), a conversational chatbot that can differentiate 28 types of vocal expressions including disappointment and disgust, excitement, fear, confusion and even anger, among others, adds the missing ingredient of emotional intelligence to AI systems. It lets users interact with it expressing their emotions in a way they would do naturally.

This is one of the most impressive applications of AI I've seen that I'm immediately going to start using. Major "aha" moment feeling like I'm going to use @hume_ai on a regular basis as a mix of an on-demand sounding board/coach/therapist.https://t.co/wWJpfbtLD1

— Vinay Hiremath (@vhmth) March 27, 2024

When implemented ethically, incorporating AI into mental health services can serve as a cornerstone of a more inclusive, effective, and compassionate healthcare system. With its ability to offer privacy, convenience, and immediate support, AI promises a future where everyone can seek and receive the help they need.

Experts believe that there is a need for collaborative approach from both the clinical psychologists and developers building AI solutions for mental health. Notably, to raise awareness, alongside making tech accessible, affordable, accurate and ensure appropriateness of the treatment – aka the 5As.

The post GPT-4 Beats Human Psychologists in Understanding Complex Emotions appeared first on Analytics India Magazine.

A mysterious AI model named ‘gpt2-chatbot’ recently appeared on the LMSYS Chatbot Arena. According to several users on X, the model showcased better reasoning and math abilities than OpenAI’s GPT-4. This left many users surprised, questioning whether it’s a new model by OpenAI.

i do have a soft spot for gpt2

— Sam Altman (@sama) April 30, 2024

Interestingly, the model was released without official documentation, and there are no details to be found. However, soon after, OpenAI chief Sam Altman posted a cryptic message: ‘I do have a soft spot for GPT-2.’ This led to speculation that this could hint at a new version beyond GPT-4, possibly GPT-5.

AI influencer Rowan Cheung highlighted several notable features of the gpt2-chatbot, with its enhanced reasoning skills being praised by several users on X who posted screenshots.

A mysterious new AI model called “gpt2-chatbot” is going viral.
It was released without official documentation, and there is speculation that it could be OpenAI's next model.
Here's everything we know so far (and how to try it for free):

— Rowan Cheung (@rowancheung) April 30, 2024

A user on X tested the chatbot’s mathematical capabilities, presenting it with an International Math Olympiad problem. Impressively, the chatbot solved it on the first attempt, although it couldn’t tackle all problems on the test. Despite this, its performance remained outstanding.

Moreover, the chatbot’s coding skills surpassed those of GPT-4 and Claude Opus, according to Chase, founding engineer at Codegen. He said the gpt2-chatbot excelled in complex code manipulation tasks, outperforming newer models.

Furthermore, the chatbot’s proficiency in ASCII art was lauded by Cheung, who described it as “miles ahead of any other model” in this domain.

Interestingly, this development comes as the tech ecosystem eagerly awaits GPT-5. Recently, Altman said that the company will release GPT-5 in the ‘coming months,’ adding that OpenAI has more important things to release before GPT-5. “Before we talk about a GPT -5-like model… I know we have a lot of other important things to release first,” said Altman. Meta also stirred the air with Llama 3, released about two weeks ago.

The post Mysterious gpt2-Chatbot takes Everyone by Surprise appeared first on Analytics India Magazine.

Image by Author

MLOps is essential for the success of any machine learning system in production. So it's not surprising that organizations are looking for skilled MLOps engineers. But what does an MLOps engineer do?

The role of the MLOps engineer is fluid and varies from one organization to the other. However, it is both convincing and simple to think of an MLOps engineer to be more end-to-end than a data scientist. Meaning their job goes beyond building machine learning models—with roles in model building, deployment and monitoring amongst others.

This article is a compilation of MLOps courses from Google. Which will help you learn the fundamentals of production machine learning systems with focus on Google’s Vertex AI platform.

Let’s get started!

1. Production Machine Learning Systems

To understand and appreciate MLOps, it’s important to first understand how machine learning systems work in production. The Production Machine Learning Systems course will help you learn about the implementation of machine learning systems in production focusing on:

• Static, dynamic, and continuous training
• Static and dynamic inference
• Batch and online processing

Here some of the key modules in this course:

• Architecting production ML systems
• Designing adaptable ML systems
• Designing high performance ML systems
• Building hybrid ML systems

Link: Production Machine Learning Systems

2. Machine Learning Operations (MLOps): Getting Started

The Machine Learning Operations (MLOps): Getting Started course is an introduction to machine learning operations. So you’ll learn you will learn how to deploy, test, monitor, and evaluate machine learning systems in production.

You’ll be introduced to the tools and best practices for MLOps learn about Google’s Vertex AI platform. The modules in this course are as follows:

• Employing machine learning operations
• Vertex AI and MLOps on Vertex AI

Link: Machine Learning Operations (MLOps): Getting Started

3. Machine Learning Operations (MLOps) with Vertex AI: Manage Features

The Machine Learning Operations (MLOps) with Vertex AI: Manage Features course will help you further your knowledge of doing MLOps on Google Cloud platform with focus on the Vertex AI feature store.

So you’ll become familiar with deploying, monitoring, and operating ML systems on Google Cloud. It introduces you to Vertex AI feature store and its key capabilities.

Link: Machine Learning Operations (MLOps) with Vertex AI: Manage Features

4. ML Pipelines on Google Cloud

This course ML Pipelines on Google Cloud is an in-depth course focusing on building and orchestrating ML pipelines on the Google Cloud Platform. This course has several modules covering the following key topics:

• Building and orchestrating ML pipelines using TensorFlow Extend (TFX), Google's production ML platform
• CI/CD for machine learning
• Automating ML pipelines
• Using Cloud Composer to orchestrate continuous training pipelines

Link: ML Pipelines on Google Cloud

5. Build and Deploy Machine Learning Solutions on Vertex AI

In the Build and Deploy Machine Learning Solutions on Vertex AI course, you’ll work on real-world use cases to train and deploy machine learning solutions.

In this course, you’ll get to dive into the following enterprise ML use cases:

• Retail customer lifetime value prediction
• Mobile game churn prediction
• Visual car part defect identification
• Fine-tuning BERT for sentiment classification of reviews

Along the way, you’ll also learn how to leverage AutoML.

Link: Build and Deploy Machine Learning Solutions on Vertex AI

Wrapping Up

I hope working through these courses and the labs that are part of these courses will help you gain a good grasp of building and deploying machine learning solutions with Vertex AI.

If you’re looking for a comprehensive bootcamp to learn MLOPs, you can check out the MLOps Zoomcamp by DataTalks.Club. You can learn more about this bootcamp in The Only Free Course You Need to Become a Professional MLOps Engineer.

Bala Priya C is a developer and technical writer from India. She likes working at the intersection of math, programming, data science, and content creation. Her areas of interest and expertise include DevOps, data science, and natural language processing. She enjoys reading, writing, coding, and coffee! Currently, she's working on learning and sharing her knowledge with the developer community by authoring tutorials, how-to guides, opinion pieces, and more. Bala also creates engaging resource overviews and coding tutorials.

More On This Topic

• Easily Integrate LLMs into Your Scikit-learn Workflow with Scikit-LLM
• RAPIDS cuDF to Speed up Your Next Data Science Workflow
• Streamline Your Machine Learning Workflow with Scikit-learn Pipelines
• 7 GPTs to Help Improve Your Data Science Workflow
• The 7 Best AI Tools for Data Science Workflow
• Advance your data science career to the next level

SafeBase taps AI to automate software security reviews Kyle Wiggers 8 hours

Entrepreneurs Al Yang and Adar Arnon met at Harvard Business School and quickly realized that they had an interest in common: cybersecurity.

“We’ve witnessed an evolving business climate that brought along with it an unprecedented need for improved security processes,” Arnon told TechCrunch. “Security’s importance has increased exponentially … [it’s] non-negotiable for technology buyers.”

Yang and Arnon decided to turn this interest into something more, so they started SafeBase, which was accepted into Y Combinator’s accelerator program during the pandemic.

SafeBase on Tuesday announced that it raised $33 million in a Series B round led by Touring Capital. The company helps customers fill out security questionnaires, which are reviews organizations normally kick off before buying a new piece of software. It’s a governance and compliance thing.

Security questionnaires can be painstaking, taking teams weeks to months to complete for more complex pieces of software. But Arnon makes the case that SafeBase can save time through automation — and AI.

SafeBase employs AI models “specifically trained on security documentation use cases” to read, interpret security information and questions and then automatically respond to security questionnaires. “[Our platform] takes the pain out of the cumbersome security review process by empowering security, governance, risk and compliance and revenue teams,” he said.

Image Credits: SafeBase

Being the cynic about AI I am, I asked Arnon about the accuracy of these models; AI is a notorious liar, after all. He claimed that it’s superior thanks to a “mix of large and small language models” that deliver “greater answer coverage [and].” Take that how you will.

Beyond the custom models, SafeBase provides an engine that allows a company to assign “rules-based behavior” for customer access, as well as dashboards that show insights and analytics on the company’s security posture.

SafeBase isn’t the only vendor out there offering tools to automate security questionnaires and reviews. Rivals include Conveyor, which recently raised $12.5 million; Kintent; and Quilt, which claims that it can also automate due diligence reviews in addition to security reviews.

Arnon didn’t seem too worried. Perhaps that’s because of SafeBase’s 700-company-strong customer roster, which includes Palantir, LinkedIn, Asana and Instacart.

“SafeBase saw massive growth in the past couple of years,” Arnon said. “Customers love the product and adoption continues to accelerate. The company benefits from increased visibility across its vendor network as more and more high-volume customers launch trust centers that replace the need for tens of thousands of manual security reviews.”

SafeBase, which is based in San Francisco, has 55 employees.

The company’s Series B had the participation of strategic investor Zoom Ventures (Zoom’s corporate venture arm), NEA, Y Combinator, Comcast Ventures and Cerca Partners as well as angels including former Salesforce chief trust officer Jim Alkove. It brings SafeBase’s total raised to over $50 million; Arnon says a significant portion will be put toward expanding the team.

Discussion with Tim Rohrbaugh

For our 4th episode of the AI Think Tank Podcast, we explored cybersecurity and artificial intelligence with the insights of Tim Rohrbaugh, an expert whose career has traversed the Navy to the forefront of commercial cybersecurity. The discussion focused on the strategic deployment of AI in cybersecurity, highlighting the use of open-source models and the benefits of local deployment to secure data effectively.

Bridging the Gap Between Cybersecurity and AI

Tim’s approach to cybersecurity is deeply influenced by his early encounters with Neural Networks used by Gensym and his subsequent roles, including as the CISO for major enterprises like JetBlue. His career showcases a commitment to leveraging technology not only to defend but also to anticipate and preempt potential threats.

Tim’s perspective is that AI significantly amplifies the ability to analyze and react to security challenges dynamically. “AI transforms the landscape of cybersecurity by turning defensive tactics into proactive strategies.” This insight underpins his belief in the power of AI to revolutionize security protocols by providing faster, more comprehensive threat detection and response capabilities.

The Advantages of Open Source AI in Cybersecurity

A key theme of this episode was the significance of open source models in the cybersecurity ecosystem. Tim emphasized the community-driven nature of open source projects, which fosters innovation and accessibility, making powerful tools available to a broader audience without the prohibitive costs associated with proprietary software.

“Open source models bring the community together in the collective pursuit of enhancing security. They allow for a level of customization and control that proprietary models do not,” Tim explained. By deploying these models locally, organizations can ensure that their sensitive data remains within their control, mitigating the risk of data breaches that could occur with cloud-based solutions.

Empowering Users with Local AI Deployments

Tim is particularly enthusiastic about the potential for individuals and organizations to implement AI tools locally. This approach not only secures data but also empowers users to harness the full potential of this technology without compromising their privacy or autonomy. Local deployment of open models is a powerful strategy that Tim advocates for enhancing cybersecurity measures efficiently within the physical and operational confines of an organization.

“By deploying AI locally, we harness its strengths in a controlled environment, optimizing security and functionality simultaneously,” Tim highlighted. This method offers users the dual benefits of advanced AI capabilities and robust data security, tailored to their specific operational needs.

Tools We Covered

Tim highlighted the Windows Subsystem for Linux (WSL) as a foundational tool for integrating Linux-based applications and workflows on Windows machines without the need for dual-boot setups. I personally run both Linux and Windows on a variety of systems but love this flexibility.

Ultimately, anything that makes entry into a local deployment easy, is what I support. So for Windows users, WSL is instrumental in facilitating the seamless execution of Linux-powered AI applications, ensuring that users can leverage the full capabilities of AI tools directly from their Windows environment.

By utilizing tools like FlowiseAI for workflow management, ollama for model deployment, and OpenWebUI for easy management and interaction, users can create a robust environment for managing AI tasks with enhanced security and efficiency. These tools collectively represent a powerful suite of resources that empower users to harness the full potential of AI while maintaining stringent data security standards.

Conclusion

Tim’s expertise and forward-thinking approach offered our audience and myself a deeper understanding of how AI can be a critical ally in the fight against cyber threats. This was a seriously empowering experience, and now I have much more to do with my home servers! Should you have a need for Tim’s expertise helping your company with on-prem deployments, visit LLM Strategic Solutions. Have a wonderful day and see you next week!

Subscribe to the AI Think Tank Podcast on YouTube.
Would you like to join the show as a live attendee and interact with guests? Contact Us

In a move that underscores its commitment to Indonesia’s digital transformation, Microsoft today announced a $1.7 billion investment over the next four years to bolster the country’s cloud and AI infrastructure, as well as provide AI skilling opportunities for 840,000 people and support the nation’s growing developer community.

The investment, the single largest in Microsoft’s 29-year history in Indonesia, is aimed at helping the country achieve its ambitious “Golden Indonesia 2045” vision, which seeks to transform the nation into a global economic powerhouse.

“This new generation of AI is reshaping how people live and work everywhere, including in Indonesia,” said Satya Nadella, Chairman and CEO of Microsoft. “The investments we are announcing today – spanning digital infrastructure, skilling, and support for developers – will help Indonesia thrive in this new era.”

Budi Arie Setiadi, Indonesia’s Minister of Communications and Information Technology, emphasized the significance of the partnership with Microsoft, stating that it “perfectly aligns with our ambition for a future driven by digital innovation” and will position the country as a “pivotal contributor to the global technological supply chain.”

Nadella highlighted the critical role of developers in harnessing AI to fulfill Indonesia’s potential as a digital economy. Microsoft will continue to help foster the growth of the country’s developer community through new initiatives such as AI Odyssey, which is expected to help 10,000 Indonesian developers become AI subject matter experts by learning new skills and earning Microsoft credentials. Microsoft is significantly boosting its investments in Southeast Asia. Recently, the company announced a $2.9 billion investment over the next two years to enhance its hyperscale cloud computing and AI infrastructure in Japan. Additionally, Microsoft aims to extend its digital skilling initiatives, aiming to train over 3 million individuals in AI skills within the next three years.

The post Microsoft announces $1.7 billion investment to advance Indonesia’s Cloud and AI ambitions appeared first on Analytics India Magazine.

According to a recent report on gender diversity by Pure Storage and Zinnov, Bengaluru leads among tier-1 cities in India in diversity representation.

The report found that diversity is around 31.4% in Global Capacity Centres (GCCs) based in the city and 14% in deeptech companies.

The report titled, ‘Towards a Gender Equitable World: Unveiling Diversity in DeepTech,’ highlights the need for greater focus on university enrolment in STEM courses and workplace retention to address the low representation of women in the DeepTech sector.

The report analyses women engineering graduates between 2004 and 2023 from 42 top engineering universities leveraged by GCCs for recruitment, with particular emphasis on 23 top institutions deemed to be preferred by DeepTech companies.

Moreover, it also highlights that the ongoing gender disparity is largely due to two main factors: a shortage of women’s enrolment in these institutions and a significant rate of mid to senior-level dropouts within the industry.

Some of the other key takeaways from the report include:

GCCs are leading the charge for a diverse workforce with 28% women in their workforce, yet they face unique challenges in achieving gender parity in DeepTech organizations, where the gender diversity stands at 23%​.

The median representation of women graduates from top engineering universities stands at 25% between 2020 and 2023, which directly affects the inflow of women candidates in GCCs, especially in the deeptech sector.

Despite this disparity in women’s representation, women graduates consistently outperformed in securing placements compared to the overall average in top-tier universities.

With a mere 6.7% of women at the Executive level in GCCs and 5.1% in deeptech organisations, there is a considerable decrease in the available talent pool of women as they move up their careers.

Family and caregiving responsibilities, limited access to career advancement and leadership opportunities, poor work-life balance are some of the key factors influencing women’s attrition.

The post Bengaluru Leads in Diversity Representation Among tier-1 Indian Cities: Report appeared first on Analytics India Magazine.

Image by Author

What if AI did not exist; in a way, no such technology furor has taken the entire industry by storm.

For a business leader whose only core focus is to drive business growth by leveraging technology, the very first thought is the customer – whom are we serving? Who is our audience? What is it that they want/expect from us?

And the immediate second thought is – their pain points. What is it they need that no one, not even the competitors is serving?

Customer-Focused Business Strategy

And there starts the series of questions which, when addressed, will set the business for success.

• What makes customers’ lives easier?
• What makes a seamless experience for them?
• What are their underserved needs?

And, so starts the path to discovering the means to an end – aka the technology.

Notably, we have not yet discussed AI. Listing down the business strategy, levers and prerogatives is the most crucial and utmost important step to deciding “what to solve” and “whom to solve for”.

Thereafter. comes the question of “how to solve”. Does AI make a good solution to solve this business problem?

At this juncture, businesses need a framework to decide what use cases AI is a good fit for. Here is what I suggest – the “PRS” framework. It stands for “Patterns that Repeat at Scale”.

Image by Author

Pattern

Let’s take an example to internalize this framework:

For example, cab service providers ensure providing cab drivers’ availability at a cost-effective price, which considers various factors –

• Proximity of the available pool of drivers to the cab requestor
• Distance to the destination
• Peak demand leads to price-surge due to more cab requestors as compared to cab drivers
• Reportedly, the lower battery status of the cab requestor’s phone likely suggests an increased fare price. This gives the cab service provider the signal that the low battery of a cellphone can increase the appetite of the cab requestor to pay more for the same ride owing to a sense of urgency.
• Cab availability and pricing also vary with factors such as regular vs premium cab service, hour of the day, or unfavorable weather conditions.

Image by Author

All this, while ensuring cab drivers are sufficiently incentivized to continue enriching customer experience.

So, we understand the data patterns.

Repetition at Scale

Next comes the repeatability – all these data attributes repeat for every cab requestor and every ride across the geographies, which inevitably leads to our last point, scale.

Think of how unachievable this problem would become, had there been a manual or non-AI workflow to solve this business case which is compute-heavy.

Data Strategy

Having built the business mindset, following which we have identified the problems that make a good case to solve via AI, let us put all our attention on data. After all, data is the core engine driving the success of all AI algorithms.

I have a framework for this too — AAA which stands for Availability, Accessibility, and Authorization.
Consider this:

Do I have the data?
Vs.
Do I have the exhaustive data?

There is a minor but crucial difference between these two statements.

Just having data is not enough. One needs all the data that is needed to model the phenomenon to ensure the model sees all those attributes that a human expert gets to see too. So, data availability is key.

Next is data accessibility. Having data at disposal is one thing, but being able to access it with ease is another. It is important to build data pipelines to ensure seamless data access.

By now, we have covered a lot of ground to get data in shape, but what if we are not allowed to use the data for model training or analytical purposes?

This is where most organizations slip up. Ensure getting the necessary authorizations or even better, only use data for which you have required permissions.

With the 3A’s of data strategy, there is still one question unanswered, that is, what is the sequence or order among business, data, and AI strategy?

So Many Strategies!!!

Largely, AI strategy is always a function of business strategy and is aligned with data strategy. It is prudent to keep working on AI use cases alongside keeping 3A’s of data in progress.

Similar to the iterative nature of AI projects, the AI roadmap needs continuous refinement while preparing and enhancing data infrastructure to maximize the potential of AI technologies within an organization.

Keep analyzing and tracking the key performance indicators (KPIs) such as accuracy, efficiency, and ROI to periodically assess the status of AI initiatives to gauge their effectiveness as well as identify areas for improvement.

Bonus Tip

Most of the AI projects and strategies suffer due to a lack of timely communication. It is crucial to perform milestone checks and actively solicit feedback from stakeholders, including end-users and business leaders. All successful AI projects go through several cycles of iterations through feedback that informs adjustments and enhancements to existing models or the development of new use cases.

Additionally, the models are not just developed once and never looked back at again. It could be entirely possible that business priorities have changed over time, which must be reflected in AI strategy and in the implementation too.

Vidhi Chugh is an AI strategist and a digital transformation leader working at the intersection of product, sciences, and engineering to build scalable machine learning systems. She is an award-winning innovation leader, an author, and an international speaker. She is on a mission to democratize machine learning and break the jargon for everyone to be a part of this transformation.

More On This Topic

• A Playbook to Scale MLOps
• The Ultimate Guide To Different Word Embedding Techniques In NLP
• Bark: The Ultimate Audio Generation Model
• The Ultimate Open-Source Large Language Model Ecosystem
• Your Ultimate Guide to Chat GPT and Other Abbreviations
• The Ultimate Guide to Mastering Seasonality and Boosting Business Results