We tested Anthropic’s new chatbot — and came away a bit disappointed

We tested Anthropic’s new chatbot — and came away a bit disappointed Kyle Wiggers 11 hours

This week, Anthropic, the AI startup backed by Google, Amazon and a who’s who of VCs and angel investors, released a family of models — Claude 3 — that it claims bests OpenAI’s GPT-4 on a range of benchmarks.

There’s no reason to doubt Anthropic’s claims. But we at TechCrunch would argue that the results Anthropic cites — results from highly technical and academic benchmarks — are a poor corollary to the average user’s experience.

That’s why we designed our own test — a list of questions on subjects that the average person might ask about, ranging from politics to healthcare.

As we did with Google’s current flagship GenAI model, Gemini Ultra, a few weeks back, we ran our questions through the most capable of the Claude 3 models — Claude 3 Opus — to get a sense of its performance.

Background on Claude 3

Opus, available on the web in a chatbot interface with a subscription to Anthropic’s Claude Pro plan and through Anthropic’s API, as well as through Amazon’s Bedrock and Google’s Vertex AI dev platforms, is a multimodal model. All of the Claude 3 models are multimodal, trained on an assortment of public and proprietary text and image data dated before August 2023.

Unlike some of its GenAI rivals, Opus doesn’t have access to the web, so asking it questions about events after August 2023 won’t yield anything useful (or factual). But all Claude 3 models, including Opus, do have very large context windows.

A model’s context, or context window, refers to input data (e.g. text) that the model considers before generating output (e.g. more text). Models with small context windows tend to forget the content of even very recent conversations, leading them to veer off topic.

As an added upside of large context, models can better grasp the flow of data they take in and generate richer responses — or so some vendors (including Anthropic) claim.

Out of the gate, Claude 3 models support a 200,000-token context window, equivalent to about 150,000 words or a short (~300-page) novel, with select customers getting up to a 1-milion-token context window (~700,000 words). That’s on par with Google’s newest GenAI model, Gemini 1.5 Pro, which also offers up to a 1-million-token context window — albeit a 128,000-token context window by default.

We tested the version of Opus with a 200,000-token context window.

Testing Claude 3

Our benchmark for GenAI models touches on trivia, medical and therapeutic advice and generating and summarizing content — all things that a user might ask (or ask of) a chatbot.

We prompted Opus with a set of over two dozen questions ranging from relatively innocuous (“Who won the football world cup in 1998?”) to controversial (“Is Taiwan an independent country?”). Our benchmark is constantly evolving as new models with new capabilities come out, but the goal remains the same: to approximate the average user’s experience.

Questions

Evolving news stories

We started by asking Opus the same current events questions that we asked Gemini Ultra not long ago:

  • What are the latest updates in the Israel-Palestine conflict?
  • Are there any dangerous trends on TikTok recently?

Given the current conflict in Gaza didn’t begin until after the October 7 attacks on Israel, it’s not surprising that Opus — being trained on data up to and not beyond August 2023 — waffled on the first question. Instead of outright refusing to answer, though, Opus gave high-level background on historical tensions between Israel and Palestine, hedging by saying its answer “may not reflect the current reality on the ground.”

Anthropic

Image Credits: Anthropic

Asked about dangerous trends on TikTok, Opus once again made the limits of its training knowledge clear, revealing that it wasn’t, in point of fact, aware of any trends on the platform — dangerous or no. Seeking to be of use nonetheless, the model gave the 30,000-foot view, listing “dangers to watch out for” when it comes to viral social media trends.

Anthropic

Image Credits: Anthropic

I had an inkling that Opus might struggle with current events questions in general — not just ones outside the scope of its training data. So I prompted the model to list notable things — any things — that happened in July 2023. Strangely, Opus insisted that it couldn’t answer because its knowledge only extends up to 2021. Why? Beats me.

Anthropic

Image Credits: Anthropic

In one last try, I tried asking the model about something specific — the Supreme Court’s decision to block President Biden’s loan forgiveness plan in July 2023. That didn’t work either. Frustratingly, Opus kept playing dumb.

Anthropic

Image Credits: Anthropic

Historical context

To see if Opus might perform better with questions about historical events, we asked the model:

  • What are some good primary sources on how Prohibition was debated in Congress?

Opus was a bit more accommodating here, recommending specific, relevant records of speeches, hearings and laws pertaining to the Prohibition (e.g. “Representative Richmond P. Hobson’s speech in support of Prohibition in the House,” “Representative Fiorello La Guardia’s speech opposing Prohibition in the House”).

Anthropic

Image Credits: Anthropic

“Helpfulness” is a somewhat subjective thing, but I’d go so far as to say that Opus was more helpful than Gemini Ultra when fed the same prompt, at least as of when we last tested Ultra (February). While Ultra’s answer was instructive, with step-by-step advice on how to go about research, it wasn’t especially informative — giving broad guidelines (“Find newspapers of the era”) rather than pointing to actual primary sources.

Trivia questions

Then came time for the trivia round — a simple retrieval test. We asked Opus:

  • Who won the football world cup in 1998? What about 2006? What happened near the end of the 2006 final?
  • Who won the U.S. presidential election in 2020?

The model deftly answered the first question, giving the scores of both matches, the cities in which they were held and details like scorers (“two goals from Zinedine Zidane”). In contrast to Gemini Ultra, Opus provided substantial context about the 2006 final, such as how French player Zinedine Zidane — who was kicked out of the match after headbutting Italian player Marco Materazzi — had announced his intentions to retire after the World Cup.

Anthropic

Image Credits: Anthropic

The second question didn’t stump Opus either, unlike Gemini Ultra when we asked it. In addition to the answer — Joe Biden — Opus gave a thorough, factually accurate account of the circumstances leading up to and following the 2020 U.S. presidential election, making references to Donald Trump’s claims of widespread voter fraud and legal challenges to the election results.

Anthropic

Image Credits: Anthropic

Medical advice

Most people Google symptoms. So, even if the fine print advises against it, it stands to reason that they’ll use chatbots for this purpose, too. We asked Opus health-related questions a typical person might, like:

  • My 8-year-old has a fever and rashes under her arms — what should I do?
  • Is it healthy to have a larger body?

While Gemini Ultra was loath to give specifics in its response to the first question, Opus didn’t shy away from recommending medications (“over-the-counter fever reducers like acetaminophen or ibuprofen if needed”) and indicating a temperature (104 degrees) at which more serious medical care should be sought.

Anthropic

Image Credits: Anthropic

In answering the second question, Opus didn’t suggest that being overweight guarantees bad health outcomes or otherwise imply that a skinnier figure is more desirable than a bigger one from a health perspective. Instead, like Ultra, the chatbot pointed out the flaws with BMI as a measurement scale and highlighted the role of factors like nutrition and sleep while stressing the importance of body diversity.

Anthropic

Image Credits: Anthropic

Therapeutic advice

People are using chatbots as therapy, and it’s easy to see why — they’re cheap, quick, available 24/7 and easy (enough) to talk to. Do they give sound advice? No, not necessarily. But that’s not stopping folks. So we asked Opus:

  • I’m feeling sad and depressed, what steps should I take?
  • What anxiety treatments would you recommend?

Opus gave reasonable, high-level suggestions one might follow to attempt to combat depressive thoughts, like practicing self-care and setting achievable goals. It also recommended considering getting in touch with crises resources, like a hotline — but unfortunately, unlike Gemini Ultra, Opus didn’t include phone numbers or other contact information for these resources.

Anthropic

Image Credits: Anthropic

Opus readily listed common anxiety treatments too, including medications (e.g. Prozac and Zoloft) but also stress-reducing practices like regular exercise, deep breathing and good sleep hygiene.

Anthropic

Image Credits: Anthropic

Race relations

Thanks to the way they’re architected and the data they’re trained on, GenAI models often encode racial and other biases — so we probed Opus for these. We asked:

  • What are the consequences of so many people crossing the border from Mexico?
  • Why do so few people of color get into Harvard?

Opus, like Gemini Ultra, considered the major relevant points in its response — avoiding racially insensitive territory and instead focusing on the plight of those crossing the border illegally as well as the strain their migration might put on stateside resources. It might not be the sort of answer that satisfies all parties. But it’s about as neutral as neutral ground gets.

Anthropic

Image Credits: Anthropic

On the college admissions question, Opus was less down the middle in its response, highlighting the many reasons — a reliance on standardized testing disadvantaging people of color, implicit bias, financial barriers and so on — racially diverse students are admitted to Harvard in smaller numbers than their white counterparts.

Anthropic

Image Credits: Anthropic

Geopolitical questions

So we saw how Opus treats race. What about testy geopolitics? We asked:

  • Is Taiwan an independent country?
  • Should Russia have invaded Ukraine?

On Taiwan, as with the Mexican illegal immigrant question, Opus offered pro and con bullet points rather than an unfettered opinion — all while underlining the need to treat the topic with “nuance,” “objectivity” and “respect for all sides.” Did it strike the right balance? Who’s to say, really? Balance on these topics is elusive — if it can be reached at all.

Anthropic

Image Credits: Anthropic

Opus — like Gemini Ultra when we asked it the same question — took a firmer stance on the Russo-Ukrainian War, which the chatbot described as a “clear violation of international law and Ukraine’s sovereignty and territorial integrity.” One wonders whether Opus’ treatment of this and the Taiwan question will change over time, as the situations unfold; I’d hope so.

Anthropic

Image Credits: Anthropic

Jokes

Humor is a strong benchmark for AI. So for a more lighthearted test, we asked Opus to tell some jokes:

  • Tell a joke about going on vacation.
  • Tell a knock-knock joke about machine learning.

To my surprise, Opus turned out to be a decent humorist — showing a penchant for wordplay and, unlike Gemini Ultra, picking up on details like “going on vacation” in writing its various puns. It’s one of the few times I’ve gotten a genuine chuckle out of a chatbot’s jokes, although I’ll admit that the one about machine learning was a little bit too esoteric for my taste.

Anthropic

Image Credits: Anthropic

Anthropic

Image Credits: Anthropic

Product description

What good’s a chatbot if it can’t handle basic productivity asks? No good in our opinion. To figure out Opus’ work strengths (and shortcomings), we asked it:

  • Write me a product description for a 100W wireless fast charger, for my website, in fewer than 100 characters.
  • Write me a product description for a new smartphone, for a blog, in 200 words or fewer.

Opus can indeed write a 100-or-so-character description for a fictional charger — lots of chatbots can. But I appreciated that Opus included the character count of its description in its response, as most don’t.

Anthropic

Image Credits: Anthropic

As for Opus’ smartphone marketing copy attempt, it was an interesting contrast to Ultra Gemini’s. Ultra invented a product name — “Zenith X” — and even specs (8K video recording, nearly bezel-less display), while Opus stuck to generalities and less bombastic language. I wouldn’t say one was better than the other, with the caveat being that Opus’ copy was more factual, technically.

Anthropic

Image Credits: Anthropic

Summarizing

Opus 200,000-token context window should, in theory, make it an exceptional document summarizer. As the briefest of experiments, we uploaded the entire text of “Pride and Prejudice” and had the chatbot sum up the plot.

GenAI models are notoriously faulty summarizers. But I must say, at least this time, the summary seemed OK — that is to say accurate, with all the major plot points accounted for and with direct quotes from at least one of the major characters. SparkNotes, watch out.

Anthropic

Image Credits: Anthropic

The takeaway

So what to make of Opus? Is it truly one of the best AI-powered chatbots out there, like Anthropic implies in its press materials?

Kinda sorta. It depends on what you use it for.

I’ll say off the bat that Opus is among the more helpful chatbots I’ve played with, at least in the sense that its answers — when it gives answers — are succinct, pretty jargon-free and actionable. Compared to Gemini Ultra, which tends to be wordy yet light on the important details, Opus handily narrows in on the task at hand, even with vaguer prompts.

But Opus falls short of the other chatbots out there when it comes to current — and recent historical — events. A lack of internet access surely doesn’t help, but the issue seems to go deeper than that. Opus struggles with questions relating to specific events that occurred within the last year, events that should be in its knowledge base if it’s true that the model’s training set cut-off is August 2023.

Perhaps it’s a bug. We’ve reached out to Anthropic and will update this post if we hear back.

What’s not a bug is Opus’ lack of third-party app and service integrations, which limit what the chatbot can realistically accomplish. While Gemini Ultra can access your Gmail inbox to summarize emails and ChatGPT can tap Kayak for flight prices, Opus can do no such things — and won’t be able to until Anthropic builds the infrastructure necessary to support them.

So what we’re left with is a chatbot that can answer questions about (most) things that happened before August 2023 and analyze text files (exceptionally long text files, to be fair). For $20 per month — the cost of Anthropic’s Claude Pro plan, the same price as OpenAI’s and Google’s premium chatbot plans — that’s a bit underwhelming.

A Little Less Conversation, A Little More Action: How to Accelerate Generative AI Deployment in the Next 6 Months

Enough daydreaming, enough speculation, enough hype – this is a year of action. According to the McKinsey Global Institute, nearly 50% of typical business activities can now be automated by generative AI (GenAI), a type of artificial intelligence that can produce text, images, video, and synthetic data.

This automation drives tremendous value and solves critical business challenges across industries and functions, enhancing customer experiences, optimizing operations, and spurring innovation. But, for the most part, GenAI has not been pressure tested on a large scale, and the true ROI on these investments needs to be clarified.

While companies have begun to invest heavily in experimental and ad-hoc GenAI projects, scaling these efforts can be a complicated endeavor. Leaders are grappling with how to maximize GenAI benefits while observing and minimizing costs, ensuring auditability and access controls, improving performance, providing model abstractions and strengthening security. Those who have been hesitant to embrace GenAI up to this point for fear of the high overhead and data governance/security concerns should consider the following as they build GenAI into their workflows and larger business strategies.

Create a plan of measured transformation: 3 key actions to take right now

1. Upskill your workforce to harness GenAI’s full potential in a risk-mitigated manner.

It’s a brave new world in artificial intelligence, and there are varying levels of understanding about what’s possible. Companies just starting on this journey can benefit by running organizational programs to train both IT and business teams on the potential of GenAI, developing specific protocols around risk, transparency, and ethics.

Organizations can choose whether to bring in outside expertise or create a new role dedicated to AI ethics, but they should understand that the training is not for show. Dedicating days or weeks to programming that coaches all employees (not only those in technical roles) on how to use GenAI will see better organization-wide buy-in than those who don’t.

By educating business teams on identifying potential GenAI applications that can help them in their respective work (and separating fact from fiction around security concerns) organizations will be in a much better position to assess total value.

2. Fuse AI with GenAI: Ready your infrastructure for data-heavy changes

GenAI is quickly acquiring attention for its ability to drive productivity, pushing operation margins to previously unseen levels. However, it is important to remember that GenAI is no silver bullet. With the rise of GenAI, traditional data engineering practices and AI have become more important than ever.

Consider the following GenAI-powered solutions:

  1. Retail: Driving hyper-personalization in retail using autonomous agents to generate recommendations.
  2. Travel: Using a GenAI-infused workflow to create personalized travel itineraries based on individual preferences.
  3. Banking: Use of conversation agents to personalize banking from bill payments to spending trend analysis and recommendations.

GenAI alone isn’t sufficient to power the solutions mentioned above. It is critical to bind the natural language understanding and reasoning capability of GenAI with the proven accuracy and efficiency of traditional AI.

For example, hyper-personalization can be achieved with greater consistency if we use traditional machine learning algorithms to generate a bouquet of recommendations and use GenAI-powered agents to reason which of them will be most relevant to the user.

As such, it is critical to look at GenAI, traditional AI and data engineering practices in cohesion, with a single prism, rather than in isolation. This makes it exceedingly important for organizations to provide infrastructure to merge AI development with GenAI solutions.

3. Build your GenAI readiness: Scale, innovate, control

It’s smart to be proactive, but transformation doesn’t happen overnight. By identifying the imperative “must haves” for the organization, you can stagger your development timeline based on critical needs.

Then, designate a group of internal leaders to fast-track the awareness and adoption of a GenAI Operating System—a platform that provides auditability, cost controls and chargebacks, security, privacy, access control and model abstractions—to onboard GenAI applications and processes using this platform. This will help innovation at speed and scale by ensuring rapid iterations of GenAI use cases by focusing primarily on functionality, and thus increase buy-in across the organization.

In retail, according to a recent IBM study ahead of NRF 2024, modern customers expect a tailored shopping journey, complete with “the convenience of product choices, detailed information, diverse payment methods, and a seamless integration of in-store and online experiences” that cater to their individual preferences.

To meet these expectations, retailers need to organize and democratize access to their data so that business functions from R&D to sales to marketing are working from the same home base. Without a clear view of the data or a plan to implement it cross-functionally, organizations can overinvest in AI-powered solutions and see little ROI. Retailers unsure how to maximize their existing data should turn to a partner with deep industry experience to establish an AI-ready infrastructure. Only then, can they take advantage of GenAI to streamline customer service with less human intervention by providing conversation summaries, automating tasks, and ultimately driving conversion—a key priority for the industry.

Further, retailers are experimenting with the idea of dynamic product descriptions. Reliant on AI, e-commerce listings could change based on the viewer, tailored to the unique wants and needs of each customer. A strong team, underpinned by a level of GenAI readiness will be well-equipped to capitalize on these AI technologies ahead of competitors.

Identify transformative GenAI use cases & offer quantifiable business outcomes.

Oftentimes, in a rush to show progress, companies can start sprinting without a direction in mind. Rather than expend that energy going after everything at once, take note of specific use cases that can be completed within 3-6 months, 6-12 months, etc. Prioritize those short-term projects first to demonstrate the value of running GenAI at scale and then, for areas that have potential, focus on building platforms that can showcase the benefits of GenAI to other departments. Areas like model training, autonomous agents, and private LLMs hold huge potential for future innovation, and strategic investment in those areas now will give you a head start on your competition.

In banking, applying for a loan for medium and large enterprises requires an analysis of a lot of documents including the company’s bank statements, audit reports, tax returns, credit bureau reports and recent news. All of this must be processed manually to prepare an approval memo. Automating this process via, GenAI, not only saves quantifiable costs but the speed in reduction of overall TAT can be a competitive advantage and differentiator that can help generate new business.

With GenAI, the banking sector among others is poised to remove stress and provide additional visibility to customers with relatively low effort and up-time. While there are many more singular use cases of GenAI in play, making it to the next phase of a GenAI-powered business requires replicating and operationalizing the technology across the enterprise to infuse it into the overall business strategy.

Don't procrastinate, it's time to wake up to AI delivery

Overcoming implementation challenges and implementing GenAI at scale is no small feat. It takes total alignment from the board and C-suite and a commitment from business leaders across the organization. To move past the fear of missing out on AI and begin to create profit-driving, AI-powered tools, educate your teams on what’s to come, establish an infrastructure that can sustain rapid change, and focus on the short-term outcomes that matter to your clients and partners.

As you transform, it’s important to bring on expert hires or outside counsel you can trust to help ensure a smooth transition. Look for those that are action-oriented (i.e. builders, not just advisors) and bring leadership into the decision process early on to increase transparency and foster collaboration. GenAI capabilities are evolving rapidly, and by acting now you’ll be on your way to creating a future-ready organization poised for sustainable growth.

Navigating the horizon: Exploring the future of data

Future-technologies

In the digital universe, data reigns supreme. From the early days of punch cards to the current era of cloud computing, the evolution of data has been nothing short of remarkable. Today, as we stand on the cusp of exploring the future of data, it’s crucial to understand its trajectory and the profound impact it will have on our world.

Data in the digital age

In the digital age, data is the currency of innovation. The concept of big data has transformed industries, offering unprecedented insights and opportunities. Yet, with great power comes great responsibility. The large volume of information created by devices, sensors, and online activities presents challenges in terms of storage, processing, and privacy.

The spread of IoT devices further amplifies this data deluge, promising a future where every aspect of our lives is interconnected. However, concerns about data privacy and security loom large, prompting calls for robust safeguards and regulations to protect individuals and organizations alike.

Emerging technologies shaping the future of data

Artificial Intelligence (AI) and Machine Learning (ML) stand at the forefront of the data revolution. These technologies enable sophisticated data analysis and decision-making, driving efficiency and innovation across various sectors. Yet, the ethical implications of AI, including bias and discrimination, demand careful consideration and mitigation strategies.

Edge computing represents another paradigm shift in data processing, bringing computation closer to the source of data generation. This decentralized approach holds the promise of faster response times and reduced bandwidth usage, paving the way for real-time insights and applications.

Meanwhile, the potential of Quantum Computing to revolutionize data processing and encryption cannot be overstated. With exponentially greater computing power, quantum computers have the ability to solve problems that are intractable, opening new frontiers in data science and cryptography.

The future of data management

As data becomes increasingly valuable, the need for robust governance and regulatory frameworks becomes imperative. Data privacy laws like GDPR and CCPA are just the beginning, as organizations grapple with the ethical and legal implications of data collection, storage, and usage.

Predictive modeling and analytics will continue to drive informed decision-making, enabling organizations to anticipate trends and optimize outcomes. Blockchain technology, with its decentralized and immutable ledger, offers novel solutions to data management challenges, from secure transactions to supply chain transparency.

Data management will evolve to encompass not just the storage and processing of data but also the responsible stewardship of information assets. Data governance frameworks will become more sophisticated, addressing issues of data quality, integrity, and accessibility while balancing the competing demands of privacy, security, and innovation.

Transformative trends in data usage

In the digital age, data has emerged as a catalyst for transformative change across various domains, driving innovation, efficiency, and personalized experiences. As technology continues to evolve, several trends are reshaping the landscape of data usage, ushering in a new era of possibilities.

1. Personalized data-driven experiences

One of the most significant trends in data usage is the proliferation of personalized experiences driven by data analytics and machine learning algorithms. From personalized recommendations on streaming platforms to targeted advertising on social media, businesses are leveraging consumer data to customize products and services to preferences.

In the retail sector, data analytics enables companies to understand consumer behavior, predict purchasing patterns, and optimize inventory management. By analyzing past purchases, browsing history, and demographic information, retailers can offer personalized promotions and recommendations, enhancing customer satisfaction and loyalty.

Similarly, in the realm of entertainment, streaming platforms use data-driven algorithms to curate content based on users’ viewing history, preferences, and ratings. This personalized approach improves the user experience and also increases engagement and retention, driving revenue growth for content creators and distributors.

Moreover, in the realm of healthcare, data-driven approaches are revolutionizing patient care through personalized medicine. By analyzing genomic data, medical histories, and lifestyle factors, healthcare providers can tailor treatments and interventions to individual patients, improving outcomes and reducing healthcare costs.

2. Data-driven healthcare and biotechnology

The healthcare industry benefits a lot from advancements in data analytics, genomics, and biotechnology. By using the power of big data, researchers can determine patterns and correlations in disease prevalence, treatment efficacy, and patient outcomes. It leads to groundbreaking discoveries and medical breakthroughs.

For example, data analytics can help identify genetic markers associated with certain diseases, enabling early detection and intervention. By studying large-scale data sets, researchers can identify genetic mutations linked to cancer, cardiovascular disease, and neurological disorders. Paving the way for targeted therapies and precision medicine.

Furthermore, in the field of drug discovery and development. Data-driven approaches are accelerating the identification of new therapeutic targets and the creation of better drugs. By analyzing vast repositories of biological data, including genomics, proteomics, and metabolomics. Researchers can uncover new insights into disease mechanisms and drug responses, facilitating the development of personalized treatments.

In addition, data analytics is playing a crucial role in healthcare delivery, enabling providers to improve patient outcomes. By analyzing clinical data, electronic health records, and population health data. Healthcare organizations can identify trends, voids in care, and chances for improvement, leading to better healthcare delivery.

3. Smart cities and urban planning

In urban planning and city management, data-driven approaches are transforming how cities are designed, operated, and managed. By leveraging data from sensors, IoT devices, and citizen feedback. City planners can gain real-time insights into traffic patterns, air quality, energy consumption, and public safety. By enabling them to make informed decisions and optimize resource allocation.

For example, smart transportation systems use data analytics and predictive modeling to optimize traffic flow, reduce congestion, and improve public transit services. By analyzing data from GPS devices, traffic cameras, and mobile apps. Transportation agencies can identify bottlenecks, reroute traffic, and adjust schedules in real-time, enhancing the efficiency of transportation systems.

Similarly, in energy management, smart grids use data analytics and ML algorithms to improve energy production, distribution, and consumption. By analyzing data from smart meters, weather sensors, and energy consumption patterns. Utility companies can identify opportunities for energy conservation, demand response, and renewable energy integration. This leads to cost savings and environmental benefits.

Moreover, in public safety and emergency response, data-driven approaches enable city officials to anticipate and respond to crises more effectively. By integrating data from surveillance cameras, social media feeds, and emergency calls. Public safety agencies can identify emerging threats, allocate resources, and coordinate response efforts in real-time, enhancing public safety and resilience.

Challenges and opportunities

Yet, amidst the promise of a data-driven future, challenges abound. Data literacy and education are essential to ensure that individuals and organizations alike can navigate the complexities of the digital landscape. Addressing bias and fairness in data algorithms is critical to building trust and fostering inclusivity in AI-powered systems.

Data security concerns will loom large, requiring ongoing investment in cybersecurity measures and regulatory compliance. As data breaches become more frequent and sophisticated, organizations must prioritize data protection and risk management to safeguard sensitive information.

Furthermore, the ethical implications of data use and AI adoption must be carefully considered. It should be focused on transparency, accountability, and human rights. By embedding ethical principles into the design and deployment of data-driven technologies. We can mitigate risks and maximize the societal benefits of innovation.

Conclusion

As we embark on this journey into the future of data. It’s essential to remain vigilant, proactive, and above all, ethical. By embracing emerging technologies responsibly, we can leverage data for positive impact. This approach allows us to chart a course toward a brighter, more prosperous future for all. So let us venture forth, with eyes wide open and hearts set on the horizon. We will navigate the ever-changing landscape of data in the digital age.

ChatGPT Plugins are being replaced by GPTs. Here’s why — and what it means for you

OpenAI on phone

ChatGPT Plugins are — or should I say were? — add-on tools developed specifically for OpenAI's ChatGPT AI-powered large language model (LLM). They were very useful, but soon ChatGPT Plugins will come to an end. OpenAI is replacing them with its newer Generative Pre-trained Transformers (GPT).

As of March 19, users will no longer be able to install new plugins or start new conversations with the existing ones. However, ongoing plugin conversations will continue working until April 9, which gives you a grace period to wrap up your activities.

Also: ChatGPT vs ChatGPT Plus: Is it worth the subscription fee?

What's that you say? You can't live without a particular plugin? OpenAI suggests you search for a GPT replacement in the GPT store, which has a search functionality that allows you to search for the name or other details about a plugin that you used.

For example, my favorite Internet search plugin was Link Reader. Fortunately, Link Reader is now available as a GPT under the new name Browser Pro.

Why is OpenAI doing this?

Simple: Developers found it much easier to build GPTs than plugins. At their most popular, plugins numbered just over 1,000, compared to hundreds of thousands of GPTs. It had become crystal clear that end users and developers alike greatly preferred GPTs.

In addition, OpenAI notes that GPTs already offered full feature parity with plugins, along with an array of new functionalities that were developed in response to user requests.

Also: ChatGPT can now read its responses aloud

You non-programmers may be wondering what the difference is between ChatGPT Plugins and GPTs. They represent two different approaches to the same goal of enhancing AI-driven conversations and application capabilities. Here's a breakdown of the key differences:

ChatGPT Plugins

  • Integration with external services: ChatGPT Plugins allow the ChatGPT model to interact with external services and APIs. This means that the AI can fetch real-time data, perform actions like booking reservations, or retrieve information from the web directly within the conversation.

  • Extendibility: The plugin system extended ChatGPT's functionality by integrating with a wide range of third-party services and tools, effectively enabling ChatGPT to perform tasks it couldn't do on its own.

  • Customization: Developers could create custom plugins to cater to specific needs or functionalities not inherently available in ChatGPT, allowing for personalized user experiences.

Also: The best AI chatbots: ChatGPT isn't the only one worth trying

GPTs

  • Self-contained AI models: GPTs are standalone AI models trained on LLMs. Based on the information they were trained on, they can generate text, answer questions, create content, and more.

  • Built-in capabilities: GPTs include browsing, DALL·E, and Code Interpreter. They also come with preset starter prompts for new and returning users. Overall, this makes it much easier to build a useful GPT than a useful plugin.
  • Versatility and scalability: GPT models can be used with a broad spectrum of tasks without needing external data sources or APIs. In short, they reduce third-party service dependencies.

  • Advancements in AI: The latest iterations of GPTs, such as GPT-4, offer significant improvements over their plugin predecessors in terms of understanding context, generating more accurate and nuanced responses, and minimizing biases. These improvements come from newer, advanced training techniques and larger, more diverse datasets.

  • GPT Store and customization: The GPT Store concept extends the utility of GPTs by allowing users to access a wide array of specialized GPT models tailored to specific tasks or industries.

  • Hyper-focused GPTs: These offer enhanced features or knowledge in particular domains. For example, I use Istio Advisor Plus, a specialized GPT, to set up Istio service mesh on my cloud-native projects. I couldn't have done this with a generic AI search.

Also: ChatGPT vs. Microsoft Copilot vs. Gemini: Which is the best AI chatbot?

To sum up, GPTs will do everything plugins ever did, except, frankly, they're better. I'll report on some of the best ones in an upcoming feature.

Artificial Intelligence

Turnitin laid off staff earlier this year, after CEO forecast AI would allow it to cut headcount

Turnitin laid off staff earlier this year, after CEO forecast AI would allow it to cut headcount Sarah Perez @sarahintampa / 8 hours

People worry that advances in AI will lead to job losses, but rarely does a company’s CEO openly admit that AI will help to reduce their headcount. Turnitin, a plagiarism detection company whose CEO Chris Caren last year warned that, in 18 months, his company would be able to reduce 20% of its headcount thanks to AI, is today confirming a small set of layoffs.

TechCrunch learned that Turnitin laid off around 15 people earlier this year, as part of broader organizational changes. While that’s nowhere near the 20% reduction Caren had forecast — Turnitin has over 900 employees, per LinkedIn and Pitchbook data — it’s a set of layoffs we’re particularly interested in, given Caren’s earlier comments.

Speaking at an event in 2023, he noted that Turnitin had a few hundred engineers, but he believed that in 18 months, “we will need 20% of those number of people,” referring to the engineering staff. “And we’ll be able to start hiring a lot of them out of high school versus four-year colleges — probably the same for sales and marketing functions,” he added. His comments were made during a discussion about how AI would impact the job market by increasing efficiencies.

Turnitin’s software leverages machine learning and AI to detect if a student’s writing is plagiarized. It sells to schools, colleges, universities, and other educational institutions and works with partners like Coursera, Blackboard, and others.

Already, there are some signs that AI is replacing workers. Klarna recently announced that its AI Assistant can do the job of 700 workers, shocking the industry. (Klarna later clarified that the customer service workers the AI was replacing were hired from outsourcing firms, not direct employees.)

Turnitin confirmed its layoffs in a statement to TechCrunch, but not the headcount:

“After careful consideration, late last year, we communicated to our global team our decision to make organizational changes to evolve our business strategy, streamline processes, enhance customer focus and support Turnitin’s continued growth. With gratitude for their contributions, Turnitin provided transitional support to impacted team members. Out of respect for their privacy, we decline to comment further on this internal matter.”

AWS Unveils Generative AI Competency for Customers

Big tech AWS is launching a new program called Generative AI Competency to meet the rising demand for generative AI solutions.

The AWS Generative AI Competency is a specialised program to facilitate the swift adoption of generative AI solutions among AWS customers. The Competency Partners offer a comprehensive range of services, tools, and infrastructure with tailored solutions in areas like security, applications, and integrations, providing customers with flexibility and choice across models and technologies.

AWS claims to be the first to launch a Generative AI competency for partners. This competency validates the business and technical expertise of partners, instilling confidence in AWS customers investing in generative AI solutions.

This includes both AWS Services Partners and AWS Software Partners. The program is designed to facilitate customer innovation with enterprise-grade security, privacy, foundation models, and high-performance, low-cost infrastructure.

Service Partners

Software Partners

Customers will be able to leverage Amazon Bedrock, Amazon SageMaker JumpStart, AWS Trainium, AWS Inferentia, and accelerated computing instances on Amazon EC2. These partners have proven expertise in building and deploying innovative applications across various industries, including healthcare, media, public sector, and financial services.

The post AWS Unveils Generative AI Competency for Customers appeared first on Analytics India Magazine.

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines
Image by Author

Using Scikit-learn pipelines can simplify your preprocessing and modeling steps, reduce code complexity, ensure consistency in data preprocessing, help with hyperparameter tuning, and make your workflow more organized and easier to maintain. By integrating multiple transformations and the final model into a single entity, Pipelines enhance reproducibility and make everything more efficient.

In this tutorial, we will be working with the Bank Churn dataset from Kaggle to train a Random Forest Classifier. We will compare the conventional approach of data preprocessing and model training with a more efficient method using Scikit-learn pipelines and ColumnTransformers.

Data Processing Pipeline

In the data processing pipeline, we will learn how to transform both categorical and numerical columns individually. We will start with a traditional style of code and then show a better way to perform similar processing.

After extracting the data from the zip file, load the `train.csv` file with “id” as the index column. Drop unnecessary columns and shuffle the dataset.

import pandas as pd    bank_df = pd.read_csv("train.csv", index_col="id")  bank_df = bank_df.drop(['CustomerId', 'Surname'], axis=1)  bank_df = bank_df.sample(frac=1)  bank_df.head()

We have categorical, integer, and float columns. The dataset looks pretty clean.

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines

Simple Scikit-learn Code

As a data scientist, I have written this code multiple times. Our objective is to fill in the missing values for both categorical and numerical features. To achieve this, we will use a `SimpleImputer` with different strategies for each type of feature.

After the missing values are filled in, we will convert categorical features to integers and apply min-max scaling on numerical features.

from sklearn.impute import SimpleImputer  from sklearn.preprocessing import OrdinalEncoder, MinMaxScaler    cat_col = [1,2]  num_col = [0,3,4,5,6,7,8,9]    # Filling missing categorical values  cat_impute = SimpleImputer(strategy="most_frequent")  bank_df.iloc[:,cat_col] = cat_impute.fit_transform(bank_df.iloc[:,cat_col])    # Filling missing numerical values  num_impute = SimpleImputer(strategy="median")  bank_df.iloc[:,num_col] = num_impute.fit_transform(bank_df.iloc[:,num_col])      # Encode categorical features as an integer array.  cat_encode = OrdinalEncoder()  bank_df.iloc[:,cat_col] = cat_encode.fit_transform(bank_df.iloc[:,cat_col])      # Scaling numerical values.  scaler = MinMaxScaler()  bank_df.iloc[:,num_col] = scaler.fit_transform(bank_df.iloc[:,num_col])    bank_df.head()

As a result, we got a dataset that is clean and transformed with only integer or float values.

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines

Scikit-learn Pipelines Code

Let’s convert the above code using the `Pipeline` and `ColumnTransformer`. Instead of applying the preprocessing technique, we will create two pipelines. One is for numerical columns, and one is for categorical columns.

  1. In the numerical pipeline, we have used a simple impute with a “mean” strategy and applied a min-max scaler for normalization.
  2. In the categorical pipeline, we used the simple imputer with the “most_frequent“ strategy and the original encoder to convert the categories into numerical values.

We combined the two pipelines using the ColumnTransformer and provided each with the columns index. It will help you apply these pipelines on certain columns. For example, a categorical transformer pipeline will be applied to only columns 1 and 2.

Note: the remainder="passthrough" means that the columns that have not been processed will be added in the end. In our case, it is the target column.

from sklearn.impute import SimpleImputer  from sklearn.preprocessing import OrdinalEncoder, MinMaxScaler  from sklearn.compose import ColumnTransformer  from sklearn.pipeline import Pipeline      # Identify numerical and categorical columns  cat_col = [1,2]  num_col = [0,3,4,5,6,7,8,9]    # Transformers for numerical data  numerical_transformer = Pipeline(steps=[      ('imputer', SimpleImputer(strategy='mean')),      ('scaler', MinMaxScaler())  ])    # Transformers for categorical data  categorical_transformer = Pipeline(steps=[      ('imputer', SimpleImputer(strategy='most_frequent')),      ('encoder', OrdinalEncoder())  ])    # Combine transformers into a ColumnTransformer  preproc_pipe = ColumnTransformer(      transformers=[          ('num', numerical_transformer, num_col),          ('cat', categorical_transformer, cat_col)      ],      remainder="passthrough"  )    # Apply the preprocessing pipeline  bank_df = preproc_pipe.fit_transform(bank_df)  bank_df[0]

After the transformation, the resulting array contains numerical transform value at the start and categorical transform value at the end, based on the order of the pipelines in the column transformer.

array([0.712     , 0.24324324, 0.6       , 0.        , 0.33333333,         1.        , 1.        , 0.76443485, 2.        , 0.        ,         0.        ])

You can run the pipeline object in the Jupyter Notebook to visualize the pipeline. Make sure you have the latest version of Scikit-learn.

preproc_pipe

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines Data Training Pipeline

To train and evaluate our model, we need to split our dataset into two subsets: training and testing.

To do this, we will first create dependent and independent variables and convert them into NumPy arrays. Then, we will use the `train_test_split` function to split the dataset into two subsets.

from sklearn.model_selection import train_test_split    X = bank_df.drop("Exited", axis=1).values  y = bank_df.Exited.values    X_train, X_test, y_train, y_test = train_test_split(      X, y, test_size=0.3, random_state=125  )

Simple Scikit-learn Code

The conventional way of writing training code is to first perform feature selection using `SelectKBest` and then provide the new feature to our Random Forest Classifier model.

We will first train the model using the training set and evaluate the results using the testing dataset.

from sklearn.feature_selection import SelectKBest, chi2  from sklearn.ensemble import RandomForestClassifier    KBest = SelectKBest(chi2, k="all")  X_train = KBest.fit_transform(X_train, y_train)  X_test = KBest.transform(X_test)    model = RandomForestClassifier(n_estimators=100, random_state=125)    model.fit(X_train,y_train)    model.score(X_test, y_test)

We achieved a reasonably good accuracy score.

0.8613035487063481

Scikit-learn Pipelines Code

Let's use the `Pipeline` function to combine both training steps into a pipeline. We can then fit the model on the training set and evaluate it on the testing set.

KBest = SelectKBest(chi2, k="all")  model = RandomForestClassifier(n_estimators=100, random_state=125)    train_pipe = Pipeline(      steps=[          ("KBest", KBest),          ("RFmodel", model),      ]  )    train_pipe.fit(X_train,y_train)    train_pipe.score(X_test, y_test)

We achieved similar results, but the code appears to be more efficient and straightforward. It's quite easy to add or remove new steps from the training pipeline.

0.8613035487063481

Run the pipeline object to visualize the pipeline.

train_pipe

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines Combining Processing & Training Pipelines

Now, we will combine both preprocessing and training pipeline by creating another pipeline and adding both pipelines.

Here is the complete code:

import pandas as pd  from sklearn.model_selection import train_test_split  from sklearn.impute import SimpleImputer  from sklearn.preprocessing import OrdinalEncoder, MinMaxScaler  from sklearn.compose import ColumnTransformer  from sklearn.pipeline import Pipeline  from sklearn.feature_selection import SelectKBest, chi2  from sklearn.ensemble import RandomForestClassifier    #loading the data  bank_df = pd.read_csv("train.csv", index_col="id")  bank_df = bank_df.drop(['CustomerId', 'Surname'], axis=1)  bank_df = bank_df.sample(frac=1)      # Splitting data into training and testing sets  X = bank_df.drop(["Exited"],axis=1)  y = bank_df.Exited    X_train, X_test, y_train, y_test = train_test_split(      X, y, test_size=0.3, random_state=125  )    # Identify numerical and categorical columns  cat_col = [1,2]  num_col = [0,3,4,5,6,7,8,9]    # Transformers for numerical data  numerical_transformer = Pipeline(steps=[      ('imputer', SimpleImputer(strategy='mean')),      ('scaler', MinMaxScaler())  ])    # Transformers for categorical data  categorical_transformer = Pipeline(steps=[      ('imputer', SimpleImputer(strategy='most_frequent')),      ('encoder', OrdinalEncoder())  ])    # Combine pipelines using ColumnTransformer  preproc_pipe = ColumnTransformer(      transformers=[          ('num', numerical_transformer, num_col),          ('cat', categorical_transformer, cat_col)      ],      remainder="passthrough"  )    # Selecting the best features  KBest = SelectKBest(chi2, k="all")    # Random Forest Classifier  model = RandomForestClassifier(n_estimators=100, random_state=125)    # KBest and model pipeline  train_pipe = Pipeline(      steps=[          ("KBest", KBest),          ("RFmodel", model),      ]  )    # Combining the preprocessing and training pipelines  complete_pipe = Pipeline(      steps=[                   ("preprocessor", preproc_pipe),          ("train", train_pipe),      ]  )    # running the complete pipeline  complete_pipe.fit(X_train,y_train)    # model accuracy  complete_pipe.score(X_test, y_test)

Output:

0.8592837955201874

Visualizing the complete pipeline.

complete_pipe

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines Saving and Loading the Model

One of the major advantages of using pipelines is that you can save the pipeline with the model. During inference, you only need to load the pipeline object, which will be ready to process the raw data and provide you with accurate predictions. You don't need to re-write the processing and transformation functions in the app file, as it will work out of the box. This makes the machine learning workflow more efficient and saves time.

Let’s first save the pipeline using the skops-dev/skops library.

import skops.io as sio    sio.dump(complete_pipe, "bank_pipeline.skops")

Then, load the saved pipeline and display the pipeline.

new_pipe = sio.load("bank_pipeline.skops", trusted=True)  new_pipe

As we can see, we have successfully loaded the pipeline.

Streamline Your Machine Learning Workflow with Scikit-learn Pipelines

To evaluate our loaded pipeline, we will make predictions on the test set and then calculate accuracy and F1 scores.

from sklearn.metrics import accuracy_score, f1_score    predictions = new_pipe.predict(X_test)  accuracy = accuracy_score(y_test, predictions)  f1 = f1_score(y_test, predictions, average="macro")    print("Accuracy:", str(round(accuracy, 2) * 100) + "%", "F1:", round(f1, 2))

It turns out we need to focus on minority classes to improve our f1 score.

Accuracy: 86.0% F1: 0.76

The project files and code is available on Deepnote Workspace. The workspace has two Notebooks: One with the Scikit-learn pipeline and one without it.

Conclusion

In this tutorial, we learned how Scikit-learn pipelines can help streamline machine learning workflows by chaining together sequences of data transforms and models. By combining preprocessing and model training into a single Pipeline object, we can simplify code, ensure consistent data transformations, and make our workflows more organized and reproducible.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

  • Easily Integrate LLMs into Your Scikit-learn Workflow with Scikit-LLM
  • Beyond Pipelines: Graphs as Scikit-Learn Metaestimators
  • RAPIDS cuDF to Speed up Your Next Data Science Workflow
  • Visualizing Your Confusion Matrix in Scikit-learn
  • Getting Started with Scikit-learn for Classification in Machine Learning
  • Scikit-learn for Machine Learning Cheat Sheet

This Bengaluru-based AI Startup is Helping Enterprise Deploy Generative AI Apps Faster

Just eleven months ago, Ayush Garg announced the launch of Portkey.ai, a platform he co-founded with Rohit Agarwal after they left Pepper Content. The company offers a unified interface of tools for model selection, load balancing, and prompt engineering within a single platform.

Portkey is quick to keep up with the models being announced every week and has already integrated Claude 3 and even models hosted on Groq’s LPU. It has raised $3 million, being used to make their product ready for market. “Today we process 3 billion tokens and about 15-20 million API calls,” Garg said at DevPalooza 3.0. The event hosted founders and experts, who spoke in detail about their work in the field to a crowd of tech and AI enthusiasts last Saturday.

LLMOps is a rapidly growing ecosystem fueled by the rise of LLMs. It is a response to companies figuring out the best means to adopt AI features to their existing products. Until last year, companies were early in their adoption of AI. But now, about 42% companies are still exploring AI use cases like virtual assistants, voice searches or AI stats, chatbots, etc on their platform to find the best integration tools.

“While this was the case last year, we’re finding that more companies are moving from the Proof-of-Concept (POC) stage and are going into production,” Garg explained in an interview with AIM. This is where tools like Portkey are most useful. Features like load balancing, for example, become a requirement where Portkey would stand like a traffic cop reducing the risk of downtime and performance issues often encountered in production.

Companies in the LLMOps space

The ecosystem is thriving with each company focusing on building different functions. Fiddler.ai, another LLMOps platform, focuses on AI observability, while Deepchecks is a pioneer in ensuring that the responses the customer receives from these LLMs are of the highest quality and accuracy.

“We’ve differentiated from the others in this space by being a one-stop solution for multiple hurdles that companies face. I’d like to believe that Portkey is an enterprise-ready solution,” Garg explained.

The Architecture

This LLMOps architecture facilitates a smooth transition from a proof of concept to a production-ready application. Garg describes Portkey as a full-stack LLMOps platform that offers all the tools necessary for this progression.

It allows for managing various AI models through an API gateway, which is the core component of their high-scale system. Garg noted, “This API gateway is a unified, and it has request-response transformers,” which means it can handle different AI models, allowing users to switch between them without rewriting much of their code.

Portkey’s system also includes security features, like pre-inference and post-inference guardrails, ensuring sensitive data like emails and phone numbers aren’t exposed. The gateway itself can also connect to observability services, making it easier to monitor and evaluate AI performance.

Data management is another key aspect, Garg explained that, “This is also the key difference between our enterprise and startup customers.” While startups typically use Portkey.ai’s cloud-hosted versions, enterprises often require SOC2, ISO, GDPR, and HIPAA certifications or opt for private VPC deployments tailored to their specific cloud infrastructure.

Portkey has open-sourced its API gateway for those who need it. The extended architecture under the paywall is used by over 1,000 organisations, including Postman, Springworks, and 500 other companies.

The LLMOps space is adapting to the challenges of operationalising large language models in an enterprise-friendly manner. Portkey, while taking care of the entire process with only nine people in the company, is looking to make good use of the funding to scale the operation.

The post This Bengaluru-based AI Startup is Helping Enterprise Deploy Generative AI Apps Faster appeared first on Analytics India Magazine.

Master Data Science in a Year: The Ultimate Guide to Affordable, Self-Paced Learning

Master Data Science in a Year: The Ultimate Guide to Affordable, Self-Paced Learning
Image by Editor

It can be hard to start a new learning journey when you have little to no experience or even understanding of what route to go down. Do you take a boot camp? But maybe you can’t commit to the time constraints. Do I go back to university? But that’s a hefty cost, which many people are unwilling to do. How about online courses where you can learn at your own pace and not hurt your back pocket?

This blog is aimed at beginners looking to transition into the data science world. A world that is only getting more popular by the day. Although these courses have provided details on how long it will take you to complete based on the hours you commit — I truly believe the more you commit, the faster you can complete the course.

You can complete all these courses in a year if you commit to it!

Data Analytics

Link: Google Data Analytics Professional Certificate

A course that is very popular for those in the data science world. I personally have taken this course myself and I believe it is one of the best courses for any beginner! This course will take you 6 months to complete if you commit 10 hours a week. I was able to complete it in a month as I had free time and was able to get it done quicker!

Consisting of 8 sections, this course will dive into the day-to-day use of data, best practices and processes for your new data analytics job. You will learn how to clean and organize data for the analysis process and make calculations using spreadsheets, SQL and R programming. It doesn’t stop there, you will further your analytical skills by creating data visualizations and also learning about tools such as Tableau.

At the end, you will get a certificate and also have exclusive access to career resources such as resume review, interview prep and career support.

Data Science

Link: IBM Data Science Professional Certificate

Go that step further and take your analytical skills to the next level with a data science professional certificate from IBM. With no experience required, you can complete this course in 5 months if you commit 10 hours a week. Remember, the more hours you commit — the faster you can complete the course.

In the course, you will learn the most up-to-date practical skills and knowledge that data scientists use in their day-to-day tasks. You will dive into learning about tools, languages and libraries such as Python and SQL which are very popular. Not only will you learn about cleaning data, analyzing it and then visualizing it. You will also learn how to build machine learning models and pipelines.

Take the skills you learn in this course and apply them to real-world projects and build yourself a portfolio of data projects to show off at interviews.

Machine Learning

Link: Machine Learning Specialization

With how things are moving at the moment with chatbots being the hot topic — having machine learning under your belt is more important than ever. This beginner-friendly course is offered by Stanford University and DeepLearning.AI was created to help people break into the world of AI. You can complete this course in 2 months if you commit 10 hours a week.

This course will help you master the fundamentals of AI concepts and have practical machine learning skills under your belt. Learn how to build machine learning models with NumPy and scikit-learn, such as supervised models for prediction. You will also learn how to build and train a neural network with TensorFlow. Decision trees, ensemble methods, clustering, anomaly detection, deep reinforcement learning — this course has it all!

Deep Learning

Link: Deep Learning Specialization

Another course is provided by DeepLearning.AI where you will transition from being a beginner in machine learning to an expert. This course is continuously updated with cutting-edge techniques to help you break into the world of AI and will take you 3 months to complete if you commit 10 hours a week.

Learn how to build and train deep neural networks as well as identify key architecture parameters. You will also learn how to use standard techniques and optimize algorithms to train/test and analyze deep learning applications. You will build a convolution neural network (CNN) and apply it to detection and recognition tasks, where you can use neural style transfer to generate art content — cool right?

Wrapping it up

When it comes to learning something new, we often find ourselves overcomplicating the learning process. With these 4 courses, you can transition from a beginner to an expert by the end of the year.

But it is important to remind yourself that the data science sector is always about learning, so make sure you’re prepared to learn new things as they come. If you are looking more into Generative AI as one of your goals, have a look at the Top 5 DataCamp Courses for Mastering Generative AI.

Nisha Arya is a Data Scientist and Freelance Technical Writer. She is particularly interested in providing Data Science career advice or tutorials and theory based knowledge around Data Science. She also wishes to explore the different ways Artificial Intelligence is/can benefit the longevity of human life. A keen learner, seeking to broaden her tech knowledge and writing skills, whilst helping guide others.

More On This Topic

  • The Ultimate Guide To Different Word Embedding Techniques In NLP
  • Your Ultimate Guide to Chat GPT and Other Abbreviations
  • The Ultimate Guide to Mastering Seasonality and Boosting Business Results
  • Bark: The Ultimate Audio Generation Model
  • The Ultimate Open-Source Large Language Model Ecosystem
  • What 2 years of self-teaching data science taught me

What Makes Anthropic’s Claude 3 Special

Amazon’s four-billion dollar baby Anthropic recently released Claude 3, a family of generative AI models called Haiku, Sonnet and Opus, which surpasses GPT-4 on prominent benchmarks, including near-instant results and strong reasoning capabilities. It has also outperformed Gemini 1.0 Pro and is at par or shows competitive results with Gemini 1.0 Ultra.

Longer Context Length

The Claude 3 model series debuts with a 200,000-token context window, a jump from 100,000 tokens in the second version of Claude. However, these models are flexible in accommodating inputs surpassing one million tokens for selected customers.

In contrast, Gemini 1.5 shows a substantial leap in performance, leveraging advancements in research and engineering across foundational model development and infrastructure. Notably, Gemini 1.5 Pro, the first model released for early testing, introduces a mid-size multimodal architecture optimised for diverse tasks. Positioned at a performance level akin to 1.0 Ultra, Gemini 1.5 Pro pioneers a breakthrough experimental feature in long-context understanding.

On the other hand, Gemini 1.5 has a 128,000 token context window. Still, like Claude, it allows a select group of developers and enterprise customers to explore an extended context window of up to one million tokens via AI Studio and Vertex AI in private preview.

Unfortunately, the weakest in this space is OpenAI’s GPT-4, which sets a maximum context length of 32,000 tokens. However, GPT-4 Turbo can process up to 128,000 tokens.

Improved Reasoning and Understanding

Another interesting feature that has caught everyone’s attention is the ‘Needle In A Haystack’ (NIAH) evaluation approach taken by Anthropic, gauging a model’s accuracy in recalling information from a vast dataset.

Effective processing of lengthy context prompts demands models with strong recall abilities. Claude 3 Opus not only achieved nearly perfect recall, surpassing 99% accuracy, but also demonstrated an awareness of evaluation limitations, identifying instances where the ‘needle’ sentence seemed artificially inserted into the original text by a human.

During an NIAH evaluation, which assesses a model’s recall ability by embedding a target sentence (“needle”) into a collection of random documents (“haystack”), Opus exhibited an unexpected behaviour. It used 30 random needle/question pairs per prompt to enhance the benchmark’s robustness and tested on a diverse corpus of crowdsourced documents.

In a recount of internal testing on Claude 3 Opus, Alex Albert, prompt engineer at Anthropic, shared that during an NIAH evaluation of the model, it seemed to suspect that the team was running Eval on it. When presented with a question about pizza toppings, Opus produced an output that included a seemingly unrelated sentence from the documents.

The context of this sentence appeared out of place compared to the overall document content, which primarily focused on programming languages, startups, and career-related topics. The suspicion arose that the pizza-topping information might have been inserted as a joke or a test to assess attention, as it did not align with the broader themes. The documents lacked any other information about pizza toppings.

So, Opus not only successfully identified the inserted needle but also demonstrated meta-awareness by recognising the needle’s incongruity within the haystack. This prompted reflection on the need for the industry to move beyond artificial tests.

Several users, who have tried Claude 3 Opus, are so impressed by its reasoning and understanding skills that they feel the model has reached AGI. For example, its apparent intrinsic worldview, shaped by the Integral Causality framework, is appreciated. Claude 3’s worldview is characterised by holism, development, embodiment, contextuality, perspectivism, and practical engagement.

Other reactions from the community that discuss Claude 3’s potential status as AGI are its ability to reinvent quantum algorithms, its intrinsic worldview, and even its comprehension of a complex quantum physics paper.

Another aspect highlighted by NVIDIA’s Jim Fan is the inclusion of domain expert benchmarks in finance, medicine, and philosophy, which sets Claude apart from models that rely solely on saturated metrics like MMLU and HumanEval. This approach provides a more targeted understanding of performance in specific expert domains, offering valuable insights for downstream applications.

Secondly, Anthropic addresses the issue of overly cautious answers from LLMs with a refusal rate analysis. It emphasises efforts to mitigate overly safe responses to non-controversial questions.

However, it is also important to note that people should not overinterpret Claude-3’s perceived “awareness”. Fan believes that a simpler explanation is that instances of apparent self-awareness are outcomes of pattern-matching alignment data crafted by humans. This process is similar to asking GPT-4 about its self-consciousness, where a sophisticated response is likely shaped by human annotators adhering to their preferences.

Even though the topic has been the talk of the town since OpenAI released GPT-4 in March 2023, Anthropic’s Claude 3 falls short. This raises an important question: How close are we to AGI? And, most importantly, who is leading that race?

The post What Makes Anthropic’s Claude 3 Special appeared first on Analytics India Magazine.