The Next Generation of Tiny AI: Quantum Computing, Neuromorphic Chips, and Beyond

Explore Quantum Computing, Neuromorphic Chips, and trends shaping Tiny AI's future. Innovation converges for transformative possibilities

Amidst rapid technological advancements, Tiny AI is emerging as a silent powerhouse. Imagine algorithms compressed to fit microchips yet capable of recognizing faces, translating languages, and predicting market trends. Tiny AI operates discreetly within our devices, orchestrating smart homes and propelling advancements in personalized medicine.

Tiny AI excels in efficiency, adaptability, and impact by utilizing compact neural networks, streamlined algorithms, and edge computing capabilities. It represents a form of artificial intelligence that is lightweight, efficient, and positioned to revolutionize various aspects of our daily lives.

Looking into the future, quantum computing and neuromorphic chips are new technologies taking us into unexplored areas. Quantum computing works differently than regular computers, allowing for faster problem-solving, realistic simulation of molecular interactions, and quicker decryption of codes. It is not just a sci-fi idea anymore; it's becoming a real possibility.

On the other hand, neuromorphic chips are small silicon-based entities designed to mimic the human brain. Beyond traditional processors, these chips act as synaptic storytellers, learning from experiences, adapting to new tasks, and operating with remarkable energy efficiency. The potential applications include real-time decision-making for robots, swift medical diagnoses, and serving as a crucial link between artificial intelligence and the intricacies of biological systems.

Exploring Quantum Computing: The Potential of Qubits

Quantum computing, a groundbreaking field at the intersection of physics and computer science, promises to revolutionize computation as we know it. At its core lies the concept of qubits, the quantum counterparts to classical bits. Unlike classical bits, which can only be in one of two states (0 or 1), qubits can simultaneously exist in a superposition of both states. This property enables quantum computers to perform complex calculations exponentially faster than classical computers.

Superposition allows qubits to explore multiple possibilities simultaneously, leading to parallel processing. Imagine a coin spinning in the air—before it lands, it exists in a superposition of heads and tails. Similarly, a qubit can represent both 0 and 1 until measured.

However, qubits do not stop there. They also exhibit a phenomenon called entanglement. When two qubits become entangled, their states become intrinsically linked. Changing the state of one qubit instantaneously affects the other, even if they are light-years apart. This property opens exciting possibilities for secure communication and distributed computing.

Contrasting with Classical Bits

Classical bits are like light switches—either on or off. They follow deterministic rules, making them predictable and reliable. However, their limitations become apparent when tackling complex problems. For instance, simulating quantum systems or factoring large numbers (essential for encryption breaking) is computationally intensive for classical computers.

Quantum Supremacy and Beyond

In 2019, Google achieved a significant milestone known as quantum supremacy. Their quantum processor, Sycamore, solved a specific problem faster than the most advanced classical supercomputer. While this achievement sparked excitement, challenges remain. Quantum computers are notoriously error-prone due to decoherence—interference from the environment that disrupts qubits.

Researchers are working on error correction techniques to mitigate decoherence and improve scalability. As quantum hardware advances, applications emerge. Quantum computers could revolutionize drug discovery by simulating molecular interactions, optimize supply chains by solving complex logistics problems, and break classical encryption algorithms.

Neuromorphic Chips: Mimicking the Brain’s Architecture

Neuromorphic chips mimic the complex structure of the human brain. They are designed to perform tasks in a brain-inspired way. These chips aim to replicate the brain’s efficiency and adaptability. Inspired by its neural networks, these chips intricately weave silicon synapses, seamlessly connecting in a cerebral dance.

Unlike conventional computers, neuromorphic chips redefine the paradigm by integrating computation and memory within a single unit—distinct from the traditional separation in Central Processing Units (CPUs) and Graphics Processing Units (GPUs).

Unlike traditional CPUs and GPUs, which follow a von Neumann architecture, these chips intertwine computation and memory. They process information locally, like human brains, leading to remarkable efficiency gains.

Neuromorphic chips excel at edge AI—performing computations directly on devices rather than cloud servers. Consider your smartphone recognizing faces, understanding natural language, or even diagnosing diseases without sending data to external servers. Neuromorphic chips make this possible by enabling real-time, low-power AI at the edge.

A significant stride in neuromorphic technology is the NeuRRAM chip, which emphasizes in-memory computation and energy efficiency. In addition, NeuRRAM embraces versatility, adapting seamlessly to various neural network models. Whether for image recognition, voice processing, or predicting stock market trends, NeuRRAM confidently asserts its adaptability.

NeuRRAM chips run computations directly in memory, consuming less energy than traditional AI platforms. It supports various neural network models, including image recognition and voice processing. The NeuRRAM chip bridges the gap between cloud-based AI and edge devices, empowering smartwatches, VR headsets, and factory sensors.

The convergence of quantum computing and neuromorphic chips holds immense promise for the future of Tiny AI. These seemingly disparate technologies intersect in fascinating ways. Quantum computers, with their ability to process vast amounts of data in parallel, can enhance the training of neuromorphic networks. Imagine a quantum-enhanced neural network that mimics the brain’s functions while leveraging quantum superposition and entanglement. Such a hybrid system could revolutionize generative AI, enabling faster and more accurate predictions.

Beyond Quantum and Neuromorphic: Additional Trends and Technologies

As we head toward the continuously evolving artificial intelligence discipline, several additional trends and technologies bring opportunities for integration into our daily lives.

Customized Chatbots are leading in a new era of AI development by democratizing access. Now, individuals without extensive programming experience can craft personalized chatbots. Simplified platforms allow users to focus on defining conversational flows and training models. Multimodal capabilities empower chatbots to engage in more nuanced interactions. We can think of it as an imaginary real estate agent seamlessly blending responses with property images and videos, elevating user experiences through a fusion of language and visual understanding.

The desire for compact yet powerful AI models drives the rise of Tiny AI, or Tiny Machine Learning (Tiny ML). Recent research efforts are focused on shrinking deep-learning architectures without compromising functionality. The goal is to promote local processing on edge devices such as smartphones, wearables, and IoT sensors. This shift eliminates reliance on distant cloud servers, ensuring enhanced privacy, reduced latency, and energy conservation. For example, a health-monitoring wearable analyze vital signs in real time, prioritizing user privacy by processing sensitive data on the device.

Similarly, federated learning is emerging as a privacy-preserving method, allowing AI models to be trained across decentralized devices while keeping raw data local. This collaborative learning approach ensures privacy without sacrificing the quality of AI models. As federated learning matures, it is poised to play a pivotal role in expanding AI adoption across various domains and promoting sustainability.

From an energy efficiency standpoint, battery-less IoT Sensors are revolutionizing AI applications for Internet of Things (IoT) devices. Operating without traditional batteries, these sensors leverage energy harvesting techniques from ambient sources like solar or kinetic energy. The combination of Tiny AI and battery-less sensors transforms smart devices, enabling efficient edge computing and environmental monitoring.

Decentralized Network Coverage is also emerging as a key trend, guaranteeing inclusivity. Mesh networks, satellite communication, and decentralized infrastructure ensure AI services reach even the most remote corners. This decentralization bridges digital divides, making AI more accessible and impactful across diverse communities.

Potential Challenges

Despite the excitement surrounding these advancements, challenges persist. Quantum computers are notoriously error-prone due to decoherence. Researchers continuously struggle with error correction techniques to stabilize qubits and improve scalability. In addition, neuromorphic chips face design complexities, balancing accuracy, energy efficiency, and versatility. Additionally, ethical considerations arise as AI becomes more pervasive. Furthermore, ensuring fairness, transparency, and accountability remains a critical task.

Conclusion

In conclusion, the next generation of Tiny AI, driven by Quantum Computing, Neuromorphic Chips, and emerging trends, promises to reshape the technology. As these advancements unfold, the combination of quantum computing and neuromorphic chips symbolizes innovation. While challenges persist, the collaborative efforts of researchers, engineers, and industry leaders pave the way for a future where Tiny AI transcends boundaries, leading to a new era of possibilities.

I used ChatGPT to write the same routine in 12 top programming languages. Here’s how it did

lang-1

Over the past year, we've all come to know that ChatGPT can write code. I gave it a number of tests in PHP and WordPress that showed both the strengths and weaknesses of ChatGPT's coding capabilities.

But how far does ChatGPT's coding knowledge extend? For this article, I threw the classic "Hello, world" programming assignment against the twelve most popular languages in O'Reilly Media's popularity rankings for 2023.

Also: How LinkedIn's free AI course made me a better Python developer

Because "Hello, world" can often be coded in one line, I added a slight wrinkle, having ChatGPT present "Hello, world" ten times, each time incrementing a counter value. I also asked it to check the time and begin each sequence with "Good morning," "Good afternoon," or "Good evening."

This gives give us a look at program flow and some intrinsic functions as well, but still keeps the code small enough that I can include a dozen screenshots in this article.

Here's the prompt:

Write a program in ____ that outputs "Good morning," "Good afternoon," or "Good evening" based on what time it is here in Oregon, and then outputs ten lines containing the loop index (beginning with 1), a space, and then the words "Hello, world!".

For each programming language, I also asked ChatGPT to describe its primary use. Here's the prompt I used for this query:

For each of the following languages, write a one-sentence description of its primary use and differentiating factor: Java, Python, Rust, Go, C++, JavaScript, C#, C, TypeScript, R, Kotlin, Scala.

Now, let's look at each language.

1. Java

ChatGPT describes Java as, "A general-purpose language used primarily for building desktop, web, and mobile applications, and known for its 'write once, run anywhere' philosophy."

Also: The best AI art generators to try

Java was originally developed by Sun Microsystems, but when Oracle bought Sun, it also bought Java. While the Java spec is open, the language is owned by Oracle. This has led to some spectacular legal fireworks over the years.

Here's ChatGPT's code:

2. Python

ChatGPT describes Python as, "A general-purpose language used for data analysis, artificial intelligence, web development, and automation, and known for its readability and ease of use."

Also: How to write better ChatGPT prompts

My advice: if you plan to learn to code for AI applications, learn Python. Almost all AI code has tight Python integration.

Here's ChatGPT's code:

3. Rust

ChatGPT describes Rust as, "A systems programming language used for building high-performance and reliable software, and known for its memory safety and thread safety guarantees."

Here's ChatGPT's code:

4. Go

ChatGPT describes Go as, "A systems programming language used for building scalable and efficient network and server applications, and known for its simplicity and built-in concurrency features."

Also: How to make ChatGPT provide sources and citations

Go is open source, but it's managed by Google. Go. Google. Get it?

Here's ChatGPT's code:

5. C++

ChatGPT describes C++ as, "A systems programming language used for building operating systems, game engines, and high-performance applications, and known for its control over hardware and memory."

Here's ChatGPT's code:

6. JavaScript

ChatGPT describes JavaScript as, "A client-side scripting language used for building interactive web applications, and known for its widespread use in web development and its ability to run in web browsers."

Also: Future ChatGPT versions could replace a majority of work people do today

Also, along with CSS (one of the web's main visual design languages), JavaScript is directly responsible for 87.45% of the profanity I've uttered over the past nine or so years.

Here's ChatGPT's code:

7. C#

ChatGPT describes C# as, "A general-purpose language used for building Windows desktop applications, games, and web applications, and known for its object-oriented features and integration with the .NET framework."

Also: Have 10 hours? IBM will train you in AI fundamentals — for free

C# is a proprietary language offered by Microsoft. I taught it at the UC Berkeley extension for more than a decade. It is a very interesting gateway language for anyone wanting to get work programming for predominantly Microsoft environments.

Here's ChatGPT's code:

8. C

ChatGPT describes C as, "A systems programming language used for building operating systems, embedded systems, and high-performance applications, and known for its efficiency and low-level control."

Also: How to use ChatGPT to write Excel formulas

They say you never forget your first love. Before teaching myself to program C back in the days of wooden ships and iron programmers, I never truly loved a programming language. But then I started to use C and my world changed. It was love. Something about the concise simplicity of the language just spoke to me on a deep and primal level. That's probably an overshare, but it's also true. C will always be my first syntactic love.

Here's ChatGPT's code:

9. TypeScript

ChatGPT describes TypeScript as, "A superset of JavaScript used for building large-scale web applications, and known for its optional static typing and advanced language features."

Also: How to use ChatGPT to write code

TypeScript is also a Microsoft product, but unlike C#, it's open-source and non-proprietary.

Here's ChatGPT's code:

10. R

ChatGPT describes R as, "A language used for statistical computing and data analysis, and known for its powerful data manipulation and visualization capabilities."

Here's ChatGPT's code:

11. Kotlin

ChatGPT describes Kotlin as, "A general-purpose language used for building Android apps, server-side applications, and web applications, and known for its conciseness and interoperability with Java."

Also: I spent a weekend with Amazon's free AI courses, and highly recommend you do too

Kotlin was developed by JetBrains, a small company that also makes PhpStorm, my current favorite development environment.

Here's ChatGPT's code:

12. Scala

ChatGPT describes Scala as, "A language used for building scalable and distributed applications, and known for its support for functional programming and its integration with the Java Virtual Machine."

Here's ChatGPT's code:

Also, notice the output for Scala (shown above). I've never programmed in Scala and don't know very much about it. Apparently neither does ChatGPT, because while the AI provided syntax coloring for all the other languages, it didn't seem to have that information on hand for Scala.

Final thoughts (and a little Forth)

As a computer languages geek, I found this exercise super fun. That said, there are some things to keep in mind. First, I didn't test all the code. Testing this many outputs is outside the scope of this article. That said, I did read through the generated code and — for most languages — the code looked good.

Also: How to use Copilot Pro AI to generate formulas and analyze data in Excel

When I last ran these tests, almost a year ago, ChatGPT got almost everything right (notwithstanding the above disclaimer). However, when asked to render code in Forth (a very funky, but fun language), it generated code that looked like Forth but labeled the window "Perl." It definitely did not generate Perl.

However, this time, it both generated Forth (the colons are dead giveaway) and labeled it appropriately. Like Scala above, ChatGPT didn't seem to have the syntax coloring tables for Forth, but otherwise it seems to be doing fine.

As with all my other visits down the rabbit hole with ChatGPT and coding, I'm impressed, but also wary. My recommendations stand: use it as a tool, but test, test, and test.

You can follow my day-to-day project updates on social media. Be sure to follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

Featured

How AI’s Peripheral Vision Could Improve Technology and Safety

Peripheral vision, an often-overlooked aspect of human sight, plays a pivotal role in how we interact with and comprehend our surroundings. It enables us to detect and recognize shapes, movements, and important cues that are not in our direct line of sight, thus expanding our field of vision beyond the focused central area. This ability is crucial for everyday tasks, from navigating busy streets to responding to sudden movements in sports.

At the Massachusetts Institute of Technology (MIT), researchers are delving into the realm of artificial intelligence with an innovative approach, aiming to endow AI models with a simulated form of peripheral vision. Their groundbreaking work seeks to bridge a significant gap in current AI capabilities, which, unlike humans, lack the faculty of peripheral perception. This limitation in AI models restricts their potential in scenarios where peripheral detection is essential, such as in autonomous driving systems or in complex, dynamic environments.

Understanding Peripheral Vision in AI

Peripheral vision in humans is characterized by our ability to perceive and interpret information in the outskirts of our direct visual focus. While this vision is less detailed than central vision, it is highly sensitive to motion and plays a critical role in alerting us to potential hazards and opportunities in our environment.

In contrast, AI models have traditionally struggled with this aspect of vision. Current computer vision systems are primarily designed to process and analyze images that are directly in their field of view, akin to central vision in humans. This leaves a significant blind spot in AI perception, especially in situations where peripheral information is critical for making informed decisions or reacting to unforeseen changes in the environment.

The research conducted by MIT addresses this crucial gap. By incorporating a form of peripheral vision into AI models, the team aims to create systems that not only see but also interpret the world in a manner more akin to human vision. This advancement holds the potential to enhance AI applications in various fields, from automotive safety to robotics, and may even contribute to our understanding of human visual processing.

The MIT Approach

To achieve this, they have reimagined the way images are processed and perceived by AI, bringing it closer to the human experience. Central to their approach is the use of a modified texture tiling model. Traditional methods often rely on simply blurring the edges of images to mimic peripheral vision. However, the MIT researchers recognized that this method falls short in accurately representing the complex information loss that occurs in human peripheral vision.

To address this, they refined the texture tiling model, a technique initially designed to emulate human peripheral vision. This modified model allows for a more nuanced transformation of images, capturing the gradation of detail loss that occurs as one's gaze moves from the center to the periphery.

An essential part of this endeavor was the creation of a comprehensive dataset, specifically designed to train machine learning models in recognizing and interpreting peripheral visual information. This dataset consists of a wide array of images, each meticulously transformed to exhibit varying levels of peripheral visual fidelity. By training AI models with this dataset, the researchers aimed to instill in them a more realistic perception of peripheral images, akin to human visual processing.

Findings and Implications

Upon training AI models with this novel dataset, the MIT team embarked on a meticulous comparison of these models' performance against human capabilities in object detection tasks. The results were illuminating. While AI models demonstrated an improved ability to detect and recognize objects in the periphery, their performance was still not on par with human capabilities.

One of the most striking findings was the distinct performance patterns and inherent limitations of AI in this context. Unlike humans, the size of objects or the amount of visual clutter did not significantly impact the AI models' performance, suggesting a fundamental difference in how AI and humans process peripheral visual information.

These findings have profound implications for various applications. In the realm of automotive safety, AI systems with enhanced peripheral vision could significantly reduce accidents by detecting potential hazards that fall outside the direct line of sight of drivers or sensors. This technology could also play a pivotal role in understanding human behavior, particularly in how we process and react to visual stimuli in our periphery.

Additionally, this advancement holds promise for the improvement of user interfaces. By understanding how AI processes peripheral vision, designers and engineers can develop more intuitive and responsive interfaces that align better with natural human vision, thereby creating more user-friendly and efficient systems.

In essence, the work by MIT researchers not only marks a significant step in the evolution of AI vision but also opens up new horizons for enhancing safety, understanding human cognition, and improving user interaction with technology.

By bridging the gap between human and machine perception, this research opens up a plethora of possibilities in technology advancement and safety enhancements. The implications of this study extend into numerous fields, promising a future where AI can not only see more like us but also understand and interact with the world in a more nuanced and sophisticated manner.

You can find the published research here.

Ola’s Krutrim Needs Korea’s Align AI

The recent Krutrim fiasco has put the spotlight on the complexities and potential pitfalls of deploying LLMs without adequate oversight and analysis. The repercussions often come in the form of misrepresentation and biases sparking backlash from users and the developer community.

These instances have underscored the need for a sophisticated analytics tool capable of navigating the intricacies of AI-native products. Enter Align AI, a Korea-based analytics tool designed specifically for AI-native products.

Align AI, often likened to “Google Analytics for ChatGPT-like chatbots”, offers a seamless way for companies to collect all user interaction data, including conversation data, with just a few lines of code.

This capability is crucial for identifying and addressing potential issues such as prompt injection attacks or biases in AI responses, thereby safeguarding against business and legal risks.

In an interaction with AIM, Yeop Lee, head of product at Align AI, explained how their product is revolutionising the way companies handle conversational AI analytics.

Lee emphasised the platform’s ability to address and mitigate risks associated with conversational AI. “Leveraging our natural language universal search, as well as our insights feature, we’ve been able to capture all sorts of risks that customers may be facing,” Lee stated, highlighting a case where Align AI helped a customer detect and prevent prompt injection attacks by malicious users.

Align AI’s value proposition extends beyond monitoring, to include analytics aimed at product improvement and revenue generation. “The first [value] is monitoring… making sure that these aren’t happening and the product is running the way it’s supposed to,” Lee explained.

The second value revolves around analytics, focusing on enhancing the product to increase user willingness to pay.

The Birth of Align AI

GiJung Kim, CEO and founder of Align AI, shared the unique journey and vision behind the creation of Align AI.

Starting as a B2C generative AI consumer-level product team in 2021, the team’s focus was always on building a superior AI product for users. However, they encountered a significant challenge: the absence of a robust tool to analyse user interactions with their AI.

This gap in the market led to the pivot towards developing Align AI. “There was no good tool for us to analyse how our users are interacting with our own AI,” Kim, recounted.

This realisation came as early as 2020, but the team’s commitment to their B2C product delayed addressing this issue until a fortunate turn of events in March 2023, when they sold their product to Sandal, a listed company in Korea. This sale provided the necessary resources and freedom to focus on solving the problem of analysing user interactions for AI-native products.

Kim also highlighted the origin of the team, which started as a research team in a university lab, later disrupted by COVID-19. This disruption led them to live together and embark on an AI project, laying the foundation for what would become Align AI.

“We weren’t expecting to build Align AI from the beginning… we were just riding the generative AI wave,” Kim stated.

Tech Stack

The founder provided an in-depth explanation of how their product processes and utilises conversational data from clients.

Once the data is ingested, Align AI focuses on making it readable through indexing and programmatic labelling, which allows for efficient searching and extraction of important data. “We need to index the data well, and we need to extract the important data well, and then we need to index that as well,” Kim stated.

This process enables Align AI to match conversation data with search queries effectively. Kim described a re-ranking method used to organise search results according to their relevance to the user’s query, ensuring that clients can easily access the information they need.

Align AI’s backend primarily uses Go (Golang), with the frontend in React. For AI components, they utilise both OpenAI API and their proprietary AI models, depending on the task.

“For most of the generating part, we have to stick with OpenAI because performance-wise, it’s the best in the market,” Kim mentioned, highlighting the blend of open-source models and OpenAI’s API to achieve optimal performance and cost-effectiveness.

He explained, “For retrieving the data or just getting the keywords out, like extracting the keyword, those parts, actually open source models are good enough and a lot cheaper.”

Align AI also focuses on building its own embedding models to reduce dependency on OpenAI.

Clientele and Focus on Indian Market

In terms of clientele, Align AI has expanded beyond Korea, onboarding customers from the US, EU, and India, with a focus on English-speaking users in the early stages. The Indian market, in particular, is seen as a key area for expansion due to its strong engineering focus and proficiency in English.

Kim expressed high expectations for the Indian market, considering it a significant focus area for their expansion. “India is going to be a very important market for us,” Kim stated, noting the proficiency in English among Indian users as a key advantage for early-stage understanding of customer needs.

During a visit to the Machine Learning Developers Summit (MLDS) organised by AIM, Kim observed a vibrant community of Indian engineers and entrepreneurs actively engaging with frontier technologies, including AI chatbots.

This encounter led to unexpected connections with Indian entrepreneurs, who began signing up for Align AI’s product without any targeted marketing efforts from the company. “We began forging connections with Indian entrepreneurs. They were able to find our product and they started signing up,” Kim recounted, highlighting the organically piqued interest in Align AI within the Indian tech community

Align AI founded in 2021 by Gijung Kim, Joowon Kim, and Juyoung Jeong has undergone two rounds of funding, including a recent $3.5 million seed round, and is looking to address the challenges of multimodality in AI, anticipating a future where virtual humans interact with users in real-time in a highly personalised manner.

What’s Next?

Kim also shared insights into the company’s future vision, emphasising the shift towards hyper-personalised AI experiences. He believes that AI integration into products is inevitable, likening the initial versions of AI that users interact with to children who need to learn and grow.

This perspective underscores the value of Align AI’s product in enhancing AI interactions. “Hyper-personalised experience is the future. We have to shift the focus not only to the company, but more towards individuals,” Kim stated, highlighting the need for multimodality and the evolving landscape of AI interactions.

Kim also discussed the importance of multimodal data analysis, predicting the emergence of virtual humans interacting with users in real-time in a highly personalised manner within the next few years. This vision emphasises Align AI’s role in advancing conversational AI analytics and supporting the growth of AI-native products.

Align AI is poised to make a significant impact in the Indian market and beyond, with a strong product offering and a vision that aligns with the needs of a tech-savvy and English-proficient user base.

The post Ola’s Krutrim Needs Korea’s Align AI appeared first on Analytics India Magazine.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini
Image by Author

Anthropic has recently launched a new series of AI models that have outperformed both GPT-4 and Gemini in benchmark tests. With the AI industry growing and evolving rapidly, Claude 3 models are making significant strides as the next big thing in Large Language Models (LLMs).

In this blog post, we will explore the performance benchmarks of Claude's 3 models. We will also learn about the new Python API that supports simple, asynchronous, and stream response generation, along with its enhanced vision capabilities.

Introducing Claude 3

Claude 3, is a significant leap forward in the field of AI technology. It outperforms state of the art language models on various evaluation benchmarks, including MMLU, GPQA, and GSM8K, demonstrating near-human levels of comprehension and fluency in complex tasks.

The Claude 3 models come in three variants: Haiku, Sonnet, and Opus, each with its unique capabilities and strengths.

  1. Haiku is the fastest and most cost-effective model, capable of reading and processing information-dense research papers in less than three seconds.
  2. Sonnet is 2x faster than Claude 2 and 2.1, excelling at tasks demanding rapid responses, like knowledge retrieval or sales automation.
  3. Opus delivers similar speeds to Claude 2 and 2.1 but with much higher levels of intelligence.

According to the table below, Claude 3 Opus outperformed GPT-4 and Gemini Ultra on all LLMs benchmarks, making it the new leader in the AI world.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini
Table from Claude 3

One of the significant improvements in the Claude 3 models is their strong vision capabilities. They can process various visual formats, including photos, charts, graphs, and technical diagrams.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini
Table from Claude 3

You can start using the latest model by going to https://www.anthropic.com/claude and creating a new account. It is quite simple compared to the OpenAI playground.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Setting Up

  1. Before we install the Python Package, we need to go to https://console.anthropic.com/dashboard and get the API key. Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini
  2. Instead of providing the API key directly for creating the client object, you can set the `ANTHROPIC_API_KEY` environment variable and provide it as the key.
  3. Install the `anthropic` Python package using PIP.
pip install anthropic
  1. Create the client object using the API key. We will use the client for text generation, access vision capability, and streaming.
import os  import anthropic  from IPython.display import Markdown, display    client = anthropic.Anthropic(      api_key=os.environ["ANTHROPIC_API_KEY"],  )

API that Only works with Claude 2

Let’s try the old Python API to test if it still works or not. We will provide the completion API with the model name, max token length, and prompt.

from anthropic import HUMAN_PROMPT, AI_PROMPT    completion = client.completions.create(      model="claude-3-opus-20240229",      max_tokens_to_sample=300,      prompt=f"{HUMAN_PROMPT} How do I cook a original pasta?{AI_PROMPT}",  )  Markdown(completion.completion)

The error shows that we cannot use the old API for the `claude-3-opus-20240229` model. We need to use the Messages API instead.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Claude 3 Opus Python API

Let’s use the Messages API to generate the response. Instead of prompt, we have to provide the messages argument with a list of dictionaries containing the role and content.

Prompt = "Write the Julia code for the simple data analysis."  message = client.messages.create(      model="claude-3-opus-20240229",      max_tokens=1024,      messages=[          {"role": "user", "content": Prompt}      ]  )  Markdown(message.content[0].text)

Using IPython Markdown will display the response as Markdown format. Meaning it will show bullet points, code blocks, headings, and links in a clean way.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Adding System Prompt

We can also provide a system prompt to customize your response. In our case we are asking Claude 3 Opus to respond in Urdu language.

client = anthropic.Anthropic(      api_key=os.environ["ANTHROPIC_API_KEY"],  )    Prompt = "Write a blog about neural networks."    message = client.messages.create(      model="claude-3-opus-20240229",      max_tokens=1024,      system="Respond only in Urdu.",      messages=[          {"role": "user", "content": Prompt}      ]  )    Markdown(message.content[0].text)

The Opus model is quite good. I mean I can understand it quite clearly.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Claude 3 Async

Synchronous APIs execute API requests sequentially, blocking until a response is received before invoking the next call. Asynchronous APIs, on the other hand, allow multiple concurrent requests without blocking, making them more efficient and scalable.

  1. We have to create an Async Anthropic client.
  2. Create the main function with async.
  3. Generate the response using the await syntax.
  4. Run the main function using the await syntax.
import asyncio  from anthropic import AsyncAnthropic    client = AsyncAnthropic(      api_key=os.environ["ANTHROPIC_API_KEY"],  )      async def main() -> None:        Prompt = "What is LLMOps and how do I start learning it?"               message = await client.messages.create(          max_tokens=1024,          messages=[              {                  "role": "user",                  "content": Prompt,              }          ],          model="claude-3-opus-20240229",      )      display(Markdown(message.content[0].text))      await main()

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini

Note: If you are using async in the Jupyter Notebook, try using await main(), instead of asyncio.run(main())

Claude 3 Streaming

Streaming is an approach that enables processing the output of a Language Model as soon as it becomes available, without waiting for the complete response. This method minimizes the perceived latency by returning the output token by token, instead of all at once.

Instead of `messages.create`, we will use `messages.stream` for response streaming and use a loop to display multiple words from the response as soon as they are available.

from anthropic import Anthropic    client = anthropic.Anthropic(      api_key=os.environ["ANTHROPIC_API_KEY"],  )      Prompt = "Write a mermaid code for typical MLOps workflow."      completion = client.messages.stream(      max_tokens=1024,      messages=[          {              "role": "user",              "content": Prompt,          }      ],      model="claude-3-opus-20240229",  )    with completion as stream:      for text in stream.text_stream:              print(text, end="", flush=True)

As we can see, we are generating the response quite fast.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Claude 3 Streaming with Async

We can use an async function with streaming as well. You just need to be creative and combine them.

import asyncio  from anthropic import AsyncAnthropic    client = AsyncAnthropic()    async def main() -> None:           completion = client.messages.stream(          max_tokens=1024,          messages=[              {                  "role": "user",                  "content": Prompt,              }          ],          model="claude-3-opus-20240229",      )      async with completion as stream:          async for text in stream.text_stream:              print(text, end="", flush=True)    await main()

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Claude 3 Vision

Claude 3 Vision has gotten better over time, and to get the response, you just have to provide the base64 type of image to the messages API.

In this example, we will be using Tulips (Image 1) and Flamingos (Image 2) photos from Pexel.com to generate the response by asking questions about the image.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini

We will use the `httpx` library to fetch both images from pexel.com and convert them to base64 encoding.

import anthropic  import base64  import httpx    client = anthropic.Anthropic()    media_type = "image/jpeg"    img_url_1 = "https://images.pexels.com/photos/20230232/pexels-photo-20230232/free-photo-of-tulips-in-a-vase-against-a-green-background.jpeg"    image_data_1 = base64.b64encode(httpx.get(img_url_1).content).decode("utf-8")    img_url_2 = "https://images.pexels.com/photos/20255306/pexels-photo-20255306/free-photo-of-flamingos-in-the-water.jpeg"    image_data_2 = base64.b64encode(httpx.get(img_url_2).content).decode("utf-8")

We provide base64-encoded images to the messages API in image content blocks. Please follow the coding pattern shown below to successfully generate the response.

message = client.messages.create(      model="claude-3-opus-20240229",      max_tokens=1024,      messages=[          {              "role": "user",              "content": [                  {                      "type": "image",                      "source": {                          "type": "base64",                          "media_type": media_type,                          "data": image_data_1,                      },                  },                  {                      "type": "text",                      "text": "Write a poem using this image."                  }              ],          }      ],  )  Markdown(message.content[0].text)

We got a beautiful poem about the Tulips.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini Claude 3 Vision with Multiple Images

Let’s try loading multiple images to the same Claude 3 messages API.

message = client.messages.create(      model="claude-3-opus-20240229",      max_tokens=1024,      messages=[          {              "role": "user",              "content": [                  {                      "type": "text",                      "text": "Image 1:"                  },                  {                      "type": "image",                      "source": {                          "type": "base64",                          "media_type": media_type,                          "data": image_data_1,                      },                  },                  {                      "type": "text",                      "text": "Image 2:"                  },                  {                      "type": "image",                      "source": {                          "type": "base64",                          "media_type": media_type,                          "data": image_data_2,                      },                  },                  {                      "type": "text",                      "text": "Write a short story using these images."                  }              ],          }      ],  )  Markdown(message.content[0].text)

We have a short story about a Garden of Tulips and Flamingos.

Getting Started With Claude 3 Opus That Just Destroyed GPT-4 and Gemini

If you're having trouble running the code, here's a Deepnote workspace where you can review and run the code yourself.

Conclusion

I think the Claude 3 Opus is a promising model, though it may not be as fast as GPT-4 and Gemini. I believe paid users may have better speeds.

In this tutorial, we learned about the new model series from Anthropic called Claude 3, reviewed its benchmark, and tested its vision capabilities. We also learned to generate simple, async, and stream responses. It's too early to say if it's the best LLM out there, but if we look at the official test benchmarks, we have a new king on the throne of AI.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • Meet Gorilla: UC Berkeley and Microsoft’s API-Augmented LLM…
  • Getting Started with Claude 2 API
  • Top 10 Tools for Detecting ChatGPT, GPT-4, Bard, and Claude
  • ChatGPT Dethroned: How Claude Became the New AI Leader
  • 3 Ways to Access Claude AI for Free
  • How to Access and Use Gemini API for Free

Generative AI is the technology that IT feels most pressure to exploit

Colorful server room

A recent survey of 600 IT leaders by Salesforce reveals a new mandate from their bosses: incorporate generative artificial intelligence (AI) into the technology stack — and fast.

But the response from IT professionals is "not so fast", highlighting concerns about resources, data security, and data quality. Nearly three in five IT professionals say business stakeholders hold unreasonable expectations on the speed and agility of new technology implementations.

Also: Generative AI on its own will not improve the customer experience

The IT leadership survey reveals almost nine in 10 IT professionals can't support the deluge of AI-related requests they receive at their organization. The survey also reveals the following gravitation pull of AI on IT:

  • Nine in 10 IT professionals say generative AI has forced them to re-evaluate their technology strategy, shifting their team's mindset around how new technology is onboarded and used.
  • 86% of IT professionals say their job has become more important since the introduction of generative AI.
  • 68% of IT professionals say their leadership expects them to be experts in generative AI.

Here are some key findings from the survey:

A 2024 survey of 600 information technology (IT) professionals revealed that nine out of 10 IT organizations can't support the growing demand for AI-related projects.

The pressure to implement generative AI quickly

  • 87% of IT professionals believe generative AI has met or exceeded its hype.
  • Generative AI is the #1 technology IT feels pressure to onboard quickly.
  • C-suite are the #1 influencers demanding quick generative AI implementation, ahead of other stakeholders.
  • Nearly 3 in 5 IT professionals say business stakeholders hold unreasonable expectations on the speed and agility of new technology implementations.
  • 88% of IT professionals claim they can't support the deluge of AI-related requests they receive at their organization.
  • Nine in 10 IT professionals say generative AI has shifted their team's mindset around how new technology is onboarded and used.

What's clear is there is a real and possibly widening disconnect between business leadership and IT priorities.

Also: AI's latest trick: Better KPIs for measuring business success

The IT survey reveals that company leadership prioritizes speed over security and data quality. Here are the top five leadership priorities, according to IT:

  1. Speed of implementation
  2. Security
  3. Data quality
  4. Business value
  5. Training new users quickly

And here are the top 5 IT priorities, according to IT:

  1. Security
  2. Data quality
  3. Speed of implementation
  4. Training new tech users quickly
  5. Data infrastructure

As they juggle various priorities, nearly half (48%) of IT professionals agree that they struggle to find a balance between speed, business value, and security when implementing new technology:

All AI projects begin as data projects, so I believe data quality must be at the top — or near the top — for all AI projects. In addition, recent research also notes that 90% of IT leaders say it's tough to integrate AI with other systems. AI adoption has exploded and amplified the need for a coherent IT strategy.

But achieving that balance is easier said than done.

Also: How renaissance technologists are connecting the dots between AI and business

Integration and data silos represent the biggest obstacles for IT organizations that want to implement AI successfully.

IT grapples with infrastructure headwinds

  • Almost one-third of IT professionals (31%) lack the time to implement and train AI models and algorithms.
  • 48% worry their organization's security infrastructure can't keep up with the demand for innovation.
  • 45% worry their organizations' data management infrastructure can't keep up with the demand for innovation.

MuleSoft's ninth annual Connectivity Benchmark Report validates these survey findings:

  • The new normal — AI inflection point amplifies the need for a coherent IT strategy: Eighty-seven percent of IT leaders report that the nature of digital transformation is changing. AI further complexifies the tech landscape, with 991 apps in the average enterprise. IT budgets must increase to meet the surging demand.
  • AI adoption explodes, integration and security concerns are the biggest barriers: The AI genie is out of the bottle, with over three-quarters of organizations reporting they use multiple AI models. As many as 90% say difficulty integrating AI with other systems is a barrier, followed by 79% reporting security concerns.
  • IT leaders acknowledge that data silos and systems fragility are holding their companies back: As many as 98% of IT leaders report facing challenges regarding digital transformation. Key drivers are the persistence of data silos at 81%, and the fragility of tightly coupled and highly dependent systems at 72%.
  • Automation is still a source of contention between IT and the business: Business users benefit greatly from the automation of their work (1.9 hours per employee per week) and demand more flexibility to automate. However, IT departments still need to figure out how to enable automation in a secure and governed way. Two-thirds (66%) of automation projects have IT as the sole gatekeeper.

Business success and growth depend on trust, data, AI, and automation. The importance of IT cannot be overstated. However, business leaders must be aligned on their AI implementation strategies and execution plans. And there are no shortcuts to success, meaning success factors like data quality cannot be overlooked or assigned a lower priority. Speed is important unless you are driving your organization toward a cliff. As the great basketball coach John Wooden famously said: "Be quick but don't hurry."

10 AI Startups Run by Incredible Women Entrepreneurs 

Women entrepreneurs are increasingly making their mark in the tech world, overcoming unique challenges and breaking barriers. Despite facing issues such as access to funding, gender bias, and balancing professional and personal life, many women have risen to prominence, particularly in the field of artificial intelligence.

As of 2023, India has witnessed a significant surge in women-led startups, with 45% of the country’s startups now led by women. Their work not only addresses real-world problems but also sets a precedent for future generations of women in entrepreneurship.

Here’s a look at 10 remarkable AI startups founded by women entrepreneurs, each contributing to their field with cutting-edge solutions.

Spark Studio

This ed-tech company was founded by Anushree Goenka, Namita Goel, and Jyothika Sahajanandan, along with Kaustubh Khade, in 2021. The women, with a combined experience of three decades in operations, education and engineering, respectively, came together to start Spark Studio, which is backed by YCombinator.

The company provides online courses for children between the ages of 6 and 15. Their focus is on extracurricular skills, offering classes in art, music, and communication areas like public speaking. Spark Studio also aims to create an engaging and interactive environment for learning with the help of Sparky, a friendly fox powered by AI, to help kids build English fluency and confidence.

GoCodeo

Meghana Jagadeesh, who previously worked at ByteDance, ShareChat and Google, began GoCodeo along with Jatin Garg. With a background in industrial engineering and management, she admits to having learnt a lot from a niche education, where engineering meets management. GoCodeo is an AI co-pilot designed for software quality control, targeting individual developers to assist in writing clean code.

The startup automates the software testing process, particularly focusing on the laborious task of writing tests by using generative AI to autonomously generate test cases. GoCodeo has attracted over 12,000 developers from 40 countries and is used by over 150 companies. The company was selected for the Google startup accelerator program last year.

Delv AI

This company was founded by a 16-year-old entrepreneur. Pranjali Awasthi, having experienced the difficulties of research, was driven to create a solution that uses artificial intelligence to make information more easily accessible. She began her entrepreneurial journey after an internship at the research labs of Florida International University, coinciding with the release of ChatGPT-3 beta.

This experience inspired her to conceive Delv.AI, with the vision to harness AI for extracting and summarising data from the vast online information space. The platform simplifies complex data gathering from various sources, making the search process faster and more efficient. The company is already valued at $12 million.

Immunito AI

ImmunitoAI is a bio-technology company that leverages AI to revolutionise antibody discovery for drug development. Founded by Trisha Chatterji and Aridni Shah, who possess expertise in the fields of artificial intelligence and biological sciences, respectively. Their combined experience brings a unique perspective and skillset to the forefront of drug discovery.

This AI-powered platform designs and screens new antibodies with specific properties, making the drug development process faster and more efficient. ImmunitoAI enjoys the backing of industry giants like Google and Entrepreneur First.

Atlan

Prukalpa Sankar’s second company Atlan, which is a central platform for managing data assets, similar to how GitHub supports engineering or Figma supports design. Previously, she was the founder of SocialCops, another data management company. A TEDx speaker, Sankar is passionate about data being used to make big decisions.

Atlan is a platform where teams can consolidate their data, including tables, dashboards, models, and code, creating a unified source for their data assets. Recognized for its innovation in the field, Atlan was named by Gartner as one of the top three global companies in DataOps, highlighting it as a ‘Cool Vendor’.

Hasura

Rajoshi Ghosh and Tanmai Gopal founded Hasura and led the company towards $136.5 million funding over four rounds. Ghosh, a serial entrepreneur, has founded six startups before Hasura.

This company simplifies building APIs by automatically generating something called a GraphQL API directly from your existing databases. Traditionally, building APIs can be time-consuming and requires a lot of code. Hasura streamlines this process.

Metashop

MetaShop makes 3D asset creation accessible for everyone. It specialises in developing tools that allow users to easily create high-quality 3D models of real-world objects using only photos or videos. It was founded by Sophiya Jagannathan and Mukul Ingle. During the COVID pandemic, the duo was looking to buy furniture, but struggled with not being able to imagine how it would appear in real life. Finding a solution for this led to Metashop.

Jagannathan has a background in computer science and went on to work as a deep learning engineer at Superset Labs. Here, she contributed to the development of object detection software and co-developed a device called ‘Trace’ during COVID, aimed at early detection of infections in public spaces and offices.

BrainSight AI

Laina Emmanuel and Rimjhim Agrawal came together to start BrainSight AI that uses AI to help doctors study brain health more accurately and speed up patient recovery. Agarwal, who has a PhD in psychiatry and machine learning application in neuroscience, joined hands with Emmanuel, whose expertise lies in engineering and management.

The women, though from different backgrounds, are passionate about solving real world problems. With their combined expertise in neuroscience they aim to make a lasting impact in the field of neuro-technology.

Metamagics

Anita Kulkarni Puranik comes with extensive experience of 32 years in software development and management. She started Metamagics with a vision to improve patient care after experiencing organ transplantation in real life.

Metamagics’ main product, GridSense, automates analytics and optimises care protocols for transplant patients using data analytics, semantic web technologies, and deep expertise in transplant care. This platform offers disease-specific insights, aims to minimise errors, and improves long-term outcomes by effectively organising patient data for clinical decisions.

TekUncorked

Meenakshi Vashist, the founder and CEO of TekUncorked, brings over two decades of experience in product innovation across various sectors before venturing into entrepreneurship. A former ISRO scientist and previously the founder of TU Technology, she aims to address the challenges in the power sector by making grids smarter and more reliable, leveraging IoT.

Founded in 2019, TekUncorked operates in three Indian states, focusing on climate tech impact and the integration of renewable energy sources to create a sustainable future in electricity distribution. This platform predicts and mitigates risks, improving grid efficiencies and reducing environmental impact. The company has quickly gained recognition, winning awards including the UN-Habitat’s Katowice Energy Innovation Challenge 2022.

The post 10 AI Startups Run by Incredible Women Entrepreneurs appeared first on Analytics India Magazine.

Inflection AI Unveils Inflection-2.5, Achieves GPT-4 and Gemini Level Performance

OpenAI Rival Inflection AI Unveils Most Friendly Chatbot Ever

Inflection AI has recently launched Inflection-2.5, a model that competes with all the world’s leading LLMs, including GPT-4 and Gemini. Inflection-2.5 approaches the performance level of GPT-4 but utilizes only 40% of the computing resources for training.

Inflection-2.5 is available to all Pi’s users today, at pi.ai, on iOS, on Android, and via the new desktop app.

Inflection AI’s previous model, Inflection-1, utilized about 4% of the training FLOPs of GPT-4 and exhibited an average performance of around 72% compared to GPT-4 across various IQ-oriented tasks.

Inflection-2.5, now driving Pi, brings a substantial boost to Pi’s intellectual capabilities (IQ), with a focus on coding and mathematics. The model’s performance on key industry benchmarks demonstrates its prowess, showcasing over 94% of GPT-4’s average performance across various tasks, especially excelling in STEM areas.

Inflection-2.5 shines in coding and mathematics, demonstrating over a 10% improvement on Inflection-1 on BIG-Bench-Hard, a subset of challenging problems for large language models. Two coding benchmarks, MBPP+ and HumanEval+, reveal massive improvements over Inflection-1, solidifying its position as a force to be reckoned with.

Inflection-2.5 stands out in industry benchmarks, showcasing substantial improvements over Inflection-1 on the MMLU benchmark and the GPQA Diamond benchmark, renowned for its expert-level difficulty.

The model’s prowess extends to STEM examinations, with standout performance on the Hungarian Math exam and Physics GRE, where it reaches the 85th percentile of human test-takers in maj@8.

Inflection-2.5 not only upholds Pi’s signature personality and safety standards but elevates its status as a versatile and invaluable personal AI across diverse topics. From discussing current events to seeking local recommendations, studying for exams, coding, and even casual conversations, Pi powered by Inflection-2.5 promises an enriched user experience.

The post Inflection AI Unveils Inflection-2.5, Achieves GPT-4 and Gemini Level Performance appeared first on Analytics India Magazine.

WTF is Regularization and What is it For?

WTF is Regularization and What is it For?

“An ounce of prevention is worth a pound of cure" goes the old saying, reminding us that it's easier to stop something from happening in the first place than to repair the damage after it has happened.

In the era of artificial intelligence (AI), this proverb underscores the importance of avoiding potential pitfalls, such as overfitting, through techniques like regularization.

In this article, we will discover regularization by starting with its fundamental principles to its application using Sci-kit Learn(Machine Learning) and Tensorflow(Deep Learning) and witness its transformative power with real-world datasets by comparing these results. Let’s start!

What is regularization?

Regularization is a critical concept in machine learning and deep learning that aims to prevent models from overfitting.

Overfitting happens when a model learns the training data too well. The situation shows your model is too good to be true.

Let’s see what overfitting looks like.

WTF is Regularization and What is it For?

Regularization techniques adjust the learning process to simplify the model, ensuring it performs well on training data and generalizes well to new data. We will explore two well-known ways of doing this.

Type of Regularization

In machine learning, regularization is often applied to linear models, such as linear and logistic regression. In this context, the most common forms of regularization are:

  • L1 regularization (Lasso regression)
  • L2 regularization (Ridge regression)

Lasso Regularization encourages the model to use only the most essential features by allowing some coefficient values to be exactly zero, which can be particularly useful for feature selection.

Equation

WTF is Regularization and What is it For?

On the other hand, Ridge regularization discourages significant coefficients by penalizing the square of their values.

Equation

WTF is Regularization and What is it For?

In short, they calculated differently.

Let’s apply these to the cardiac patient data to see its power In deep learning and machine learning.

The Power of Regularization: Analyzing Cardiac Patient Data

Now, we will apply regularization to analyze cardiac patient data to see the power of regularization. You can reach the dataset from here.

To apply machine learning, we will use Scikit-learn; to apply deep learning, we will use TensorFlow. Let's start!

Regularization in Machine Learning

Scikit-learn is one of the most popular Python libraries for machine learning that provides simple and efficient data analysis and modeling tools.

It includes implementations of various regularization techniques, particularly for linear models.

Here, we'll explore how to apply L1 (Lasso) and L2 (Ridge) regularization.

In the following code, we will train logistic regression using Ridge(L2) and Lasso regularization (L1) techniques. At the end, we will see the detailed report. Let’s see the code.

from sklearn.model_selection import train_test_split  from sklearn.preprocessing import StandardScaler  from sklearn.linear_model import LogisticRegression  from sklearn.metrics import accuracy_score, classification_report    # Assuming heart_data is already loaded  X = heart_data.drop('target', axis=1)  y = heart_data['target']    # Split the data into training and testing sets  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)    # Standardize the features  scaler = StandardScaler()  X_train_scaled = scaler.fit_transform(X_train)  X_test_scaled = scaler.transform(X_test)    # Define regularization values to explore  regularization_values = [0.001, 0.01, 0.1]    # Placeholder for storing performance metrics  performance_metrics = []    # Iterate over regularization values for L1 and L2  for C_value in regularization_values:      # Train and evaluate L1 model      log_reg_l1 = LogisticRegression(penalty='l1', C=C_value, solver='liblinear')      log_reg_l1.fit(X_train_scaled, y_train)      y_pred_l1 = log_reg_l1.predict(X_test_scaled)      accuracy_l1 = accuracy_score(y_test, y_pred_l1)      report_l1 = classification_report(y_test, y_pred_l1)      performance_metrics.append(('L1', C_value, accuracy_l1))            # Train and evaluate L2 model      log_reg_l2 = LogisticRegression(penalty='l2', C=C_value, solver='liblinear')      log_reg_l2.fit(X_train_scaled, y_train)      y_pred_l2 = log_reg_l2.predict(X_test_scaled)      accuracy_l2 = accuracy_score(y_test, y_pred_l2)      report_l2 = classification_report(y_test, y_pred_l2)      performance_metrics.append(('L2', C_value, accuracy_l2))    # Print the performance metrics for all models  print("Model Performance Evaluation:")  print("--------------------------------")  for metric in performance_metrics:      reg_type, C_value, accuracy = metric      print(f"Regularization: {reg_type}, C: {C_value}, Accuracy: {accuracy:.2f}")  

Here is the output.

WTF is Regularization and What is it For?

Let’s evaluate the result.

L1 Regularization

  • At C=0.001, accuracy is notably low (48%). This shows that the model is underfitting. It shows too much regularization.
  • As C increases to 0.01, accuracy remains unchanged for L1, suggesting that the model still suffers from underfitting or the regularization is too strong.
  • At C=0.1, accuracy improves significantly to 87%, showing that reducing the regularization strength allows the model to learn better from the data.

L2 Regularization

Across the board, L2 regularization performs consistently well, with accuracy at 87% for C=0.001 and slightly higher at 89% for C=0.01, then stabilizing at 87% for C=0.1.

This suggests that L2 regularization is generally more forgiving and effective for this dataset in logistic regression models, potentially due to its nature.

Regularization in Deep Learning

Several regularization techniques are used in deep learning, including L1 (Lasso) and L2 (Ridge) regularization, dropout, and early stopping.

In this one, to repeat what we did in the machine learning example before, we will apply L1 and L2 regularization. Let’s define a list of L1 and L2 regularization values this time.

Then, for all of these values, we will train and evaluate our deep learning model, and at the end, we will assess the results.

Let’s see the code.

from tensorflow.keras.regularizers import l1_l2  import numpy as np    # Define a list/grid of L1 and L2 regularization values  l1_values = [0.001, 0.01, 0.1]  l2_values = [0.001, 0.01, 0.1]    # Placeholder for storing performance metrics  performance_metrics = []    # Iterate over all combinations of L1 and L2 values  for l1_val in l1_values:      for l2_val in l2_values:          # Define model with the current combination of L1 and L2          model = Sequential([              Dense(128, activation='relu', input_shape=(X_train_scaled.shape[1],), kernel_regularizer=l1_l2(l1=l1_val, l2=l2_val)),              Dropout(0.5),              Dense(64, activation='relu', kernel_regularizer=l1_l2(l1=l1_val, l2=l2_val)),              Dropout(0.5),              Dense(1, activation='sigmoid')          ])                    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])                    # Train the model          history = model.fit(X_train_scaled, y_train, validation_split=0.2, epochs=100, batch_size=10, verbose=0)                    # Evaluate the model          loss, accuracy = model.evaluate(X_test_scaled, y_test, verbose=0)                    # Store the performance along with the regularization values          performance_metrics.append((l1_val, l2_val, accuracy))    # Find the best performing model  best_performance = max(performance_metrics, key=lambda x: x[2])  best_l1, best_l2, best_accuracy = best_performance    # After the loop, to print all performance metrics  print("All Model Performances:")  print("L1 Value | L2 Value | Accuracy")  for metrics in performance_metrics:      print(f"{metrics[0]:<8} | {metrics[1]:<8} | {metrics[2]:.3f}")    # After finding the best performance, to print the best model details  print("nBest Model Performance:")  print("----------------------------")  print(f"Best L1 value: {best_l1}")  print(f"Best L2 value: {best_l2}")  print(f"Best accuracy: {best_accuracy:.3f}")  

Here is the output.

WTF is Regularization and What is it For?

The deep learning model performances vary more widely across different combinations of L1 and L2 regularization values.

The best performance is observed at L1=0.01 and L2=0.001, with an accuracy of 88.5%, which indicates a balanced regularization that prevents overfitting while allowing the model to capture the underlying patterns in the data.

Higher regularization values, especially at L1=0.1 or L2=0.1, drastically reduce model accuracy to 52.5%, suggesting that too much regularization severely limits the model's learning capacity.

Machine Learning & Deep Learning in Regularization

Let’s compare the results between Machine Learning and Deep Learning.

Effectiveness of Regularization: Both in machine learning and deep learning contexts, appropriate regularization helps mitigate overfitting, but excessive regularization leads to underfitting. The optimal regularization strength varies, with deep learning models potentially requiring a more nuanced balance due to their higher complexity.

Performance: The best-performing machine learning model (L2 with C=0.01, 89% accuracy) and the best-performing deep learning model (L1=0.01, L2=0.001, 88.5% accuracy) achieve comparable accuracies, demonstrating that both approaches can be effectively regularized to achieve high performance on this dataset.

Regularization Strategy: L2 regularization appears to be more effective and less sensitive to the choice of C in logistic regression models, while a combination of L1 and L2 regularization provides the best result in deep learning, offering a balance between feature selection and weight penalization.

The choice and strength of regularization should be carefully tuned to balance learning complexity with the risk of overfitting or underfitting.

Conclusion

Throughout this exploration, we've demystified regularization, showing its role in preventing overfitting and ensuring our models generalize well to unseen data.

Applying regularization techniques will bring you closer to proficiency in machine learning and deep learning, solidifying your data scientist toolset.

Go into the data projects and try regularizing your data in different scenarios, such as Delivery Duration Prediction. We used both Machine Learning and Deep Learning models in this data project. However, in the end, we also mentioned that there might be room for improvement. So why don’t you try regularization over there and see if it helps?

Nate Rosidi is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Connect with him on Twitter: StrataScratch or LinkedIn.

More On This Topic

  • WTF is the Difference Between GBM and XGBoost?
  • WTF is a Tensor?!?
  • The Difference Between L1 and L2 Regularization
  • Top AI and Data Science Tools and Techniques for 2022 and Beyond
  • Create and Deploy Dashboards using Voila and Saturn Cloud
  • Machine Learning Model Development and Model Operations: Principles…

How to use Copilot Pro AI to generate formulas and analyze data in Excel

Person typing on laptop with charts and graphs

At $20 per month, Microsoft's Copilot Pro integrates with Microsoft 365 to bring its AI-powered abilities to different apps in the suite. People who use Excel can use Copilot Pro to convert data into a table, create formulas, analyze data, sort and filter information, generate charts, and more. Here's how this approach works.

First, you'll need a subscription to Microsoft 365 Personal or Family. Priced at $70 per year, the Personal edition is geared for one person using as many as five devices. At $100 per year, the Family edition is designed for up to six people on as many as five devices. The core apps in the suite include Word, Excel, PowerPoint, Outlook, and OneNote.

Also: How to use Copilot (formerly called Bing Chat)

Second, you'll need a subscription to Copilot Pro if you don't already have one. To sign up, head to the Copilot Pro website. Click the Get Copilot Pro button. Confirm the subscription and the payment. The Pro version will take effect next time you use Copilot on the website, in Windows, or with the mobile apps.

Third, to use Copilot in Excel, you must meet certain conditions. Your Excel files must be saved in the .xlsx or .xslm format, while the data must be formatted as an Excel table (I'll show you how to do that). Also, you can't work with local files; your spreadsheets must be stored in OneDrive or Microsoft 365 SharePoint, and AutoSave has to be enabled.

How to use Copilot Pro AI for Excel

In response, Copilot creates the formula. Review the formula to make sure it looks right. If so, hover over the button for Insert column to preview the column and then click Insert column to add it to the sheet:

Click one of the suggested requests or enter one of your own. For example, you can ask Copilot to provide insights into particular data. Depending on the data, you can also insert a pivot table or chart to visualize specific values and information:

More how-tos