Рубрика: AI

With the advancements Large Language Models have made in recent years, it's unsurprising why these LLM frameworks excel as semantic planners for sequential high-level decision-making tasks. However, developers still find it challenging to utilize the full potential of LLM frameworks for learning complex low-level manipulation tasks. Despite their efficiency, today's Large Language Models require considerable domain and subject expertise to learn even simple skills or construct textual prompts, creating a significant gap between their performance and human-level dexterity.

To bridge this gap, developers from Nvidia, CalTech, UPenn, and others have introduced EUREKA, an LLM-powered human-level design algorithm. EUREKA aims to harness various capabilities of LLM frameworks, including code-writing, in-context improvement, and zero-shot content generation, to perform unprecedented optimization of reward codes. These reward codes, combined with reinforcement learning, enable the frameworks to learn complex skills or perform manipulation tasks.

In this article, we will examine the EUREKA framework from a development perspective, exploring its framework, workings, and the results it achieves in generating reward functions. These functions, as claimed by the developers, outperform those generated by humans. We will also delve into how the EUREKA framework paves the way for a new approach to RLHF (Reinforcement Learning using Human Feedback) by enabling gradient-free in-context learning. Let's get started.

EUREKA : An Introduction

Today, state of the art LLM frameworks like GPT-3, and GPT-4 deliver outstanding results when serving as semantic planners for sequential high-level decision making tasks, but developers are still looking for ways to enhance their performance when it comes to learning low-level manipulation tasks like pen spinning dexterity. Furthermore, developers have observed that reinforcement learning can be used to achieve sustainable results in dexterous conditions, and other domains provided the reward functions are constructed carefully by human designers, and these reward functions are capable of providing the learning signals for favorable behaviors. When compared to real-world reinforcement learning tasks that accept sparse rewards makes it difficult for the model to learn the patterns, shaping these rewards provides the necessary incremental learning signals. Furthermore, rewards functions, despite their importance, are extremely challenging to design, and sub-optimal designs of these functions often lead to unintended behaviors.

To tackle these challenges and maximize the efficiency of these reward tokens, the EUREKA or Evolution-driven Universal REward Kit for Agent aims to make the following contributions.

1. Achieving human-level performance for designing Reward Functions.
2. Effectively solve manipulation tasks without using manual reward engineering.
3. Generate more human-aligned and more performant reward functions by introducing a new gradient-free in-context learning approach instead of traditional RLHF or Reinforcement Learning from Human Feedback method.

There are three key algorithmic design choices that the developers have opted for to enhance EUREKA’s generality: evolutionary search, environment as context, and reward reflection. First, the EUREKA framework takes the environment source code as context to generate executable reward functions in a zero-shot setting. Following this, the framework performs an evolutionary search to improve the quality of its rewards substantially, proposes batches of reward candidates with every iteration or epoch, and refines the ones that it finds to be the most promising. In the third and the final stage, the framework uses the reward reflection approach to make the in-context improvement of rewards more effective, a process that ultimately helps the framework enable targeted and automated reward editing by using a textual summary of the quality of these rewards on the basis of policy training statistics. The following figure gives you a brief overview of how the EUREKA framework works, and in the upcoming section, we will be talking about the architecture and working in greater detail.

EUREKA : Model Architecture, and Problem Setting

The primary aim of reward shaping is to return a shaped or curated reward function for a ground-truth reward function, which might pose difficulties when being directly optimized like sparse rewards. Furthermore, designers can only use queries to access these ground-truth reward functions which is the reason why the EUREKA framework opts for reward generation, a program synthesis setting based on RDP or the Reward Design Problem.

The Reward Design Problem or RDP is a tuple that contains a world model with a state space, space for reward functions, a transition function, and an action space. A learning algorithm then optimizes rewards by generating a policy that results in a MDP or Markov Design Process, that produces the scalar evolution of any policy, and can only be accessed using policy queries. The primary goal of the RDP is to output a reward function in a way such that the policy is capable of achieving the maximum fitness score. In EUREKA’s problem setting, the developers have specified every component in the Reward Design Problem using code. Furthermore, for a given string that specifies the details of the task, the primary objective of the reward generation problem is to generate a reward function code to maximize the fitness score.

Moving along, at its core, there are three fundamental algorithmic components in the EUREKA framework. Evolutionary search(proposing and rewarding refining candidates iteratively), environment as context(generating executable rewards in zero-shot setting), and reward reflection(to enable fine-grained improvement of rewards). The pseudo code for the algorithm is illustrated in the following image.

Environment as Context

Currently, LLM frameworks need environment specifications as inputs for designing rewards whereas the EUREKA framework proposes to feed the raw environment code directly as context, without the reward code allowing the LLM frameworks to take the world model as context. The approach followed by EUREKA has two major benefits. First, LLM frameworks for coding purposes are trained on native code sets that are written in existing programming languages like C, C++, Python, Java, and more, which is the fundamental reason why they are better at producing code outputs when they are directly allowed to compose code in the syntax and style that they have originally trained on. Second, using the environment source code usually reveals the environments involved semantically, and the variables that are fit or ideal for use in an attempt to output a reward function in accordance with the specified task. On the basis of these insights, the EUREKA framework instructs the LLM to return a more executable Python code directly with the help of only formatting tips, and generic reward designs.

Evolutionary Search

The inclusion of evolutionary search in the EUREKA framework aims to present a natural solution to the sub-optimality challenges, and errors occurred during execution as mentioned before. With each iteration or epoch, the framework various independent outputs from the Large Language Model, and provided the generations are all i.i.d, it exponentially reduces the probability of reward functions during the iterations being buggy given the number of samples are increasing with every epoch.

In the next step, the EUREKA framework uses the executable rewards functions from previous iteration the perform an in-context reward mutation, and then proposes a new and improved reward function on the basis of textual feedback. The EUREKA framework when combined with the in-context improvement, and instruction-following capabilities of Large Language Models is able to specify the mutation operator as a text prompt, and suggests a method to use the textual summary of policy training to modify existing reward codes.

Reward Reflection

To ground in-context reward mutations, it is essential to assess the quality of the generated rewards, and more importantly, put them into words, and the EUREKA framework tackles it by using the simple strategy of providing the numerical scores as reward evaluation. When the task fitness function serves as a holistic metric for ground-truth, it lacks credit assignment, and is unable to provide any valuable information as to why the reward function works, or why it does not work. So, in an attempt to provide a more targeted and intricate reward diagnosis, the framework proposes to use automated feedbacks to summarize the policy training dynamics in texts. Furthermore, in the reward program, the reward functions in the EUREKA framework are asked to expose their components individually allowing the framework to track the scalar values of every unique reward component at policy checkpoints during the entire training phase.

Although the reward function procedure followed by the EUREKA framework is simple to construct, it is essential thanks to the algorithmic-dependent nature of optimizing rewards. It means that the effectiveness of a reward function is directly influenced by the choice of a Reinforcement Learning algorithm, and with a change in hyperparameters, the reward may perform differently even with the same optimizer. Thus, the EUREKA framework is able to edit the records more effectively & selectively while synthesizing reward functions that are in enhanced synergy with the Reinforcement Learning algorithm.

Training and Baseline

There are two major training components of the EUREKA framework: Policy Learning and Reward Evaluation Metrics.

Policy Learning

The final reward functions for every individual task is optimized with the help of the same reinforcement learning algorithm using the same set of hyperparameters that are fine-tuned to make the human-engineered rewards function well.

Reward Evaluation Metrics

As the task metric varies in terms of scale & semantic meaning with every task, the EUREKA framework reports the human normalized score, a metric that provides a holistic measure for the framework to compare how it performs against the expert human-generated rewards in accordance with the ground-truth metrics.

Moving along, there are three primary baselines: L2R, Human, and Sparse.

L2R

L2R is a dual-stage Large Language Model prompting solution that helps in generating templated rewards. First, a LLM framework fills in a natural language template for environment and task specified in natural language, and then a second LLM framework converts this “motion description” into a code that writes a reward function by calling a set of manually written reward API primitives.

Human

The Human baseline are the original reward functions written by reinforcement learning researchers, thus representing the outcomes of human reward engineering at an unprecedented level.

Sparse

The Sparse baseline resembles the fitness functions, and they are used to evaluate the quality of the rewards the framework generates.

Results and Outcomes

To analyze the performance of the EUREKA framework, we will evaluate it on different parameters including its performance against human rewards, improvement in results over time, generating novel rewards, enabling targeted improvement, and working with human feedback.

EUREKA Outperforms Human Rewards

The following figure illustrates the aggregate results over different benchmarks, and as it can be clearly observed, the EUREKA framework either outperforms or performs on par to human-level rewards on both Dexterity and Issac tasks. In comparison, the L2R baseline delivers similar performance on low-dimensional tasks, but when it comes to high-dimensional tasks, the gap in the performance is quite substantial.

Consistently Improving Over Time

One of the major highlights of the EUREKA framework is its ability to constantly improve and enhance its performance over time with each iteration, and the results are demonstrated in the figure below.

As it can be clearly seen, the framework constantly generates better rewards with each iteration, and it also improves & eventually surpasses the performance of human rewards, thanks to its use of in-context evolutionary reward search approach.

Generating Novel Rewards

The novelty of the rewards of the EUREKA framework can be assessed by calculating the correlation between human and EUREKA rewards on the entirety of Issac tasks. These correlations are then plotted on a scatter-plot or map against the human normalized scores, with each point on the plot representing an individual EUREKA reward for every individual task. As it can be clearly seen, the EUREKA framework predominantly generates weak correlated reward functions outperforming the human reward functions.

Enabling Targeted Improvement

To evaluate the importance of adding reward reflection in reward feedback, developers evaluated an ablation, a EUREKA framework with no reward reflection that reduces the feedback prompts to consist only of snapshot values. When running Issac tasks, developers observed that without reward reflection, the EUREKA framework witnessed a drop of about 29% in the average normalized score.

Working with Human Feedbacks

To readily incorporate a wide array of inputs to generate human-aligned and more performant reward functions, the EUREKA framework in addition to automated reward designs also introduces a new gradient-free in-context learning approach to Reinforcement Learning from Human Feedback, and there were two significant observations.

1. EUREKA can benefit and improve from human-reward functions.
2. Using human feedback for reward reflections induces aligned behavior.

The above figure demonstrates how the EUREKA framework demonstrates a substantial boost in performance, and efficiency using human reward initialization regardless of the quality of the human rewards suggesting the quality of the base rewards does not have a significant impact on the in-context reward improvement abilities of the framework.

The above figure illustrates how the EUREKA framework can not only induce more human-aligned policies, but also modify rewards by incorporating human feedback.

Final Thoughts

In this article, we have talked about EUREKA, a LLM-powered human-level design algorithm, that attempts to harness various capabilities of LLM frameworks including code-writing, in-context improvement capabilities, and zero-shot content generation to perform unprecedented optimization of reward codes. The reward code along with reinforcement learning can then be used by these frameworks to learn complex skills, or perform manipulation tasks. Without human intervention or task-specific prompt engineering, the framework delivers human-level reward generation capabilities on a wide array of tasks, and its major strength lies in learning complex tasks with a curriculum learning approach.

Overall, the substantial performance and versatility of the EUREKA framework indicates the potential of combining evolutionary algorithms with large language models might result in a scalable and general approach to design rewards, and this insight might be applicable to other open-ended search problems.

Anthropic’s Claude just got a massive upgrade. Claude 2.1, the latest iteration of its AI language model, is now available through API, revolutionising the claude.ai chat experience. Bringing forth key enhancements for enterprises, Claude 2.1 introduces a remarkable 200K token context window, substantial reductions in model hallucination rates, and a beta feature called tool use.

This update also includes a pricing overhaul aimed at improving cost efficiency for diverse customer base.

Responding to user feedback, Claude 2.1 doubles the allowable token limit, now enabling a context window of 200,000 tokens, compared to 128,000 of GPT-4 Turbo announced at OpenAI DevDay. This corresponds to approximately 150,000 words or over 500 pages of material. Users can leverage this extended capacity to upload comprehensive documents such as codebases, financial statements, or lengthy literary works.

Claude’s capabilities now extend to summarisation, Q&A, trend forecasting, document comparison, and more, all with the ability to process complex tasks in a matter of minutes.

Addressing user demands, a new beta feature, tool use, has been introduced, allowing Claude to seamlessly integrate with existing processes, products, and APIs. This expanded interoperability enhances Claude’s utility in day-to-day operations, enabling it to orchestrate across developer-defined functions, search web sources, retrieve information from private knowledge bases, and perform various actions on behalf of users.

Read: Meet Silicon Valley’s Generative AI Darling

Furthermore, Claude 2.1 achieves a significant milestone with a 2x decrease in false statements compared to its predecessor, Claude 2.0. This boost in honesty empowers enterprises to deploy AI applications with greater trust and reliability across their operations. Rigorous testing revealed Claude 2.1’s increased likelihood to demur rather than provide incorrect information, bolstering its credibility in handling complex, factual questions.

Claude 2.1 has made significant gains in honesty, with a 2x decrease in false statements compared to Claude 2.0.
This enables enterprises to build high-performing applications that solve business problems with accuracy and reliability. pic.twitter.com/Km5UuYHR3M

— Anthropic (@AnthropicAI) November 21, 2023

Simplifying the developer Console experience, Anthropic introduced Workbench, a tool that enables developers to iterate on prompts in a playground-style environment. This facilitates faster learning and optimization of Claude’s behaviour. System prompts have also been introduced, allowing users to provide custom instructions to enhance Claude’s performance, aligning responses with specific personalities or roles.

Claude 2.1 is now available in the API and powers the chat interface at claude.ai for both free and Pro tiers. The usage of the 200K token context window is exclusive to Claude Pro users, who can now upload larger files than ever before.

The post Anthropic Launches Claude 2.1, Surpasses GPT-4 Turbo in Context Length appeared first on Analytics India Magazine.

Sponsored Content

The ability to use algorithms to solve real-world problems is a must-have skill for any developer or programmer. But a major issue for them is to dive into a big pool of algorithms and find the most relevant ones.

This book (50 Algorithms Every Programmer Should Know) will help you not only to develop the skills to select and use an algorithm to tackle problems in the real world but also to understand how it works.

You'll start with an introduction to algorithms and discover various algorithm design techniques before exploring how to implement different types of algorithms, with the help of practical examples. As you advance, you'll learn about linear programming, page ranking, and graphs, and will then work with machine learning algorithms to understand the math and logic behind them. Additionally, the book will delve into modern deep learning techniques, including Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Recurrent Neural Networks (RNNs), providing insights into their applications. The expansive realm of Generative AI and Large Language Models (LLMs) such as ChatGPT will also be explored, unraveling the algorithms, methodologies, and architectures that drive their implementation.

Case studies will show you how to apply these algorithms optimally before you focus on deep learning algorithms and learn about different types of deep learning models along with their practical use. Finally, you'll become well-versed in techniques that enable parallel processing, giving you the ability to use these algorithms for compute-intensive tasks.

By the end of this programming book, you'll have become adept at solving real-world computational problems by using a wide range of algorithms, including modern deep learning techniques.

Hurry Up, grab your copy from: https://packt.link/wAk8W

More On This Topic

• How our Obsession with Algorithms Broke Computer Vision: And how…
• Start a career in Computer Science with Penn’s Master in Computer…
• How To Tackle 3 Common Machine Learning Challenges
• Math 2.0: The Fundamental Importance of Machine Learning
• Machine Learning Is Not Like Your Brain Part 3: Fundamental Architecture
• Using Data Science to Predict and Prevent Real World Problems

Image by Editor

Large Language Models (LLMs) like OpenAI’s GPT-3, Google’s BERT, and Meta’s LLaMA are revolutionizing various sectors with their ability to generate a wide array of text?—?from marketing copy and data science scripts to poetry.

Even though ChatGPT’s intuitive interface has managed to be in most people's devices today, there’s still a vast landscape of untapped potential for using LLMs in diverse software integrations.

The main problem?

Most applications require more fluid and native communication with LLMs.

And this is precisely where LangChain kicks in!

If you are interested in Generative AI and LLMs, this tutorial is tailor-made for you.

So… let’s start!

What are LLMs?

Just in case you have been living within a cave and haven’t gotten any news lately, I’ll briefly explain Large Language Models or LLMs.

An LLM is a sophisticated artificial intelligence system built to mimic human-like textual understanding and generation. By training on enormous data sets, these models discern intricate patterns, grasp linguistic subtleties, and produce coherent outputs.

If you wonder how to interact with these AI-powered models, there are two main ways to do so:

1. The most common and direct way is talking or chatting with the model. It involves crafting a prompt, sending it to the AI-powered model, and getting a text-based output as a response.
2. Another method is converting text into numerical arrays. This process involves composing a prompt for the AI and receiving a numerical array in return. What is commonly known as an “embedding”. It has experienced a recent surge in Vector Databases and semantic search.

And it is precisely these two main problems that LangChain tries to address. If you are interested in the main problems of interacting with LLMs, you can check this article here.

LangChain and its basics

LangChain is an open-source framework built around LLMs. It brings to the table an arsenal of tools, components, and interfaces that streamline the architecture of LLM-driven applications.

With LangChain, engaging with language models, interlinking diverse components, and incorporating assets like APIs and databases become a breeze. This intuitive framework substantially simplifies the LLM application development journey.

The core idea of Long Chain is that we can connect together different components or modules, also known as chains, to create more sophisticated LLM-powered solutions.

Here are some standout features of LangChain:

1. Customizable prompt templates to standardize our interactions.
2. Chain link components tailored for sophisticated use cases.
3. Seamless integration with leading language models, including OpenAI’s GPTs and those on HuggingFace Hub.
4. Modular components for a mix-and-match approach to assess any specific problem or task.

Image by Author

LangChain is distinguished by its focus on adaptability and modular design.

The main idea behind LangChain is breaking down the natural language processing sequence into individual parts, allowing developers to customize workflows based on their requirements.

Such versatility positions LangChain as a prime choice for building AI solutions in different situations and industries.

Some of its most important components are…

Image by Author

1. LLMs

LLMs are fundamental components that leverage vast amounts of training data to understand and generate human-like text. They are at the core of many operations within LangChain, providing the necessary language processing capabilities to analyze, interpret, and respond to text input.

Usage: Powering chatbots, generating human-like text for various applications, aiding in information retrieval, and performing other language processing

2. Prompt templates

Prompts are fundamental for interacting with LLM, and when working on specific tasks, their structure tends to be similar. Prompt templates, which are preset prompts usable across chains, allow standardization of “prompts” by adding specific values. This enhances the adaptability and customization of any LLM.

Usage: Standardizing the process of interacting with LLMs.

3. Output Parsers

Output parsers are components that take the raw output from a preceding stage in the chain and convert it into a structured format. This structured data can then be used more effectively in subsequent stages or delivered as a response to the end user.

Usage: For instance, in a chatbot, an output parser might take the raw text response from a language model, extract key pieces of information, and format them into a structured reply.

4. Components and chains

In LangChain, each component acts as a module responsible for a particular task in the language processing sequence. These components can be connected to form chains for customized workflows.

Usage: Generating sentiment detection and response generator chains in a specific chatbot.

5. Memory

Memory in LangChain refers to a component that provides a storage and retrieval mechanism for information within a workflow. This component allows for the temporary or persistent storage of data that can be accessed and manipulated by other components during the interaction with the LLM.

Usage: This is useful in scenarios where data needs to be retained across different stages of processing, for example, storing conversation history in a chatbot to provide context-aware responses.

6. Agents

Agents are autonomous components capable of taking actions based on the data they process. They can interact with other components, external systems, or users, to perform specific tasks within a LangChain workflow.

Usage: For instance, an agent might handle user interactions, process incoming requests, and coordinate the flow of data through the chain to generate appropriate responses.

7. Indexes and Retrievers

Indexes and Retrievers play a crucial role in managing and accessing data efficiently. Indexes are data structures holding information and metadata from the model’s training data. On the other hand, retrievers are mechanisms that interact with these indexes to fetch relevant data based on specified criteria and allow the model to reply better by supplying relevant context.

Usage: They are instrumental in quickly fetching relevant data or documents from a large dataset, which is essential for tasks like information retrieval or question answering.

8. Document Transformers

In LangChain, Document Transformers are specialized components designed to process and transform documents in a way that makes them suitable for further analysis or processing. These transformations may include tasks such as text normalization, feature extraction, or the conversion of text into a different format.

Usage: Preparing text data for subsequent processing stages, such as analysis by machine learning models or indexing for efficient retrieval.

9. Embedding Models

They are used to convert text data into numerical vectors in a high-dimensional space. These models capture semantic relationships between words and phrases, enabling a machine-readable representation. They form the foundation for various downstream Natural Language Processing (NLP) tasks within the LangChain ecosystem.

Usage: Facilitating semantic searches, similarity comparisons, and other machine-learning tasks by providing a numerical representation of text.

10. Vector stores

Type of database system that specializes to store and search information via embeddings, essentially analyzing numerical representations of text-like data. VectorStore serves as a storage facility for these embeddings.

Usage: Allowing efficient search based on semantic similarity.

Setting it up and first examples

Installing it using PIP

The first thing we have to do is make sure we have LangChain installed in our environment.

pip install langchain

Environment setup

Utilizing LangChain typically means integrating with diverse model providers, data stores, APIs, among other components. And as you already know, like any integration, supplying the relevant and correct API keys is crucial for LangChain's operation.

Imagine we want to use our OpenAI API. We can easily accomplish this in two ways:

1. Setting up key as an environment variable

OPENAI_API_KEY="..."

or

import os    os.environ['OPENAI_API_KEY'] = “...”

If you choose not to establish an environment variable, you have the option to provide the key directly through the openai_api_key named parameter when initiating the OpenAI LLM class:

1. Directly set up the key in the relevant class.

from langchain.llms import OpenAI    llm = OpenAI(openai_api_key="...")

LangChain in action

Switching between LLMs becomes straightforward

LangChain provides an LLM class that allows us to interact with different language model providers, such as OpenAI and Hugging Face.

It is quite easy to get started with any LLM, as the most basic and easiest-to-implement functionality of any LLM is just generating text.

However, asking the very same prompt to different LLMs at once is not so easy.

This is where LangChain kicks in…

Getting back to the easiest functionality of any LLM, we can easily build an application with LangChain that gets a string prompt and returns the output of our designated LLM..

Code by Author

We can simply use the same prompt and get the response of two different models within few lines of code!

Code by Author

Impressive… right?

Giving structure to our prompts with prompt templates

A common issue with Language Models (LLMs) is their inability to escalate complex applications. LangChain addresses this by offering a solution to streamline the process of creating prompts, which is often more intricate than just defining a task as it requires outlining the AI's persona and ensuring factual accuracy. A significant part of this involves repetitive boilerplate text. LangChain alleviates this by offering prompt templates, which auto-include boilerplate text in new prompts, thus simplifying prompt creation and ensuring consistency across different tasks.

Code by Author

Getting structured responses with output parsers

In chat-based interactions, the model's output is merely text. Yet, within software applications, having a structured output is preferable as it allows for further programming actions. For instance, when generating a dataset, receiving the response in a specific format such as CSV or JSON is desired. Assuming a prompt can be crafted to elicit a consistent and suitably formatted response from the AI, there's a need for tools to manage this output. LangChain caters to this requirement by offering output parser tools to handle and utilize the structured output effectively.

Code by Author

You can go check the whole code on my GitHub.

Conclusions

Not long ago, the advanced capabilities of ChatGPT left us in awe. Yet, the technological environment is ever-changing, and now tools like LangChain are at our fingertips, allowing us to craft outstanding prototypes from our personal computers in just a few hours.

LangChain, a freely available Python platform, provides a means for users to develop applications anchored by LLMs (Language Model Models). This platform delivers a flexible interface to a variety of foundational models, streamlining prompt handling and acting as a nexus for elements like prompt templates, more LLMs, external information, and other resources via agents, as of the current documentation.

Imagine chatbots, digital assistants, language translation tools, and sentiment analysis utilities; all these LLM-enabled applications come to life with LangChain. Developers utilize this platform to craft custom-tailored language model solutions addressing distinct requirements.

As the horizon of natural language processing expands, and its adoption deepens, the realm of its applications seems boundless.

Josep Ferrer is an analytics engineer from Barcelona. He graduated in physics engineering and is currently working in the Data Science field applied to human mobility. He is a part-time content creator focused on data science and technology. You can contact him on LinkedIn, Twitter or Medium.

More On This Topic

• Top Open Source Large Language Models
• More Free Courses on Large Language Models
• Learn About Large Language Models
• Introducing Healthcare-Specific Large Language Models from John Snow Labs
• What are Large Language Models and How Do They Work?
• AI: Large Language & Visual Models

In an era marked by rapid technological evolution, the landscape of artificial intelligence is undergoing a monumental shift, spearheaded by the advent and integration of generative AI. O'Reilly, a leading beacon in technology and business learning, has unveiled its 2023 Generative AI in the Enterprise Report, offering a comprehensive global survey that illuminates the current state of generative AI in the business world.

This report, compiled from the responses of over 2,800 technology professionals, delves into the burgeoning adoption of generative AI, elucidating the trends, challenges, and opportunities it presents within the enterprise sector.

Unprecedented Adoption of Generative AI in Enterprises

The O'Reilly 2023 report reveals a significant milestone in AI's journey within the enterprise sector: a 67% adoption rate of generative AI technologies. This figure is not just impressive; it represents the fastest adoption of a technological innovation in recent history. What makes this adoption rate even more remarkable is that 38% of these enterprises have been using AI for less than a year, suggesting a rapidly growing interest and confidence in AI capabilities.

This surge in adoption can be attributed to several factors. Firstly, the evolution of generative AI technologies has made them more accessible and easier to implement. Training models have become more user-friendly, and the rise of open-source models has reduced resource requirements. Secondly, the development of tools that simplify AI interactions, such as automated prompt generation and vector databases for document retrieval, has made AI more approachable for a broader range of organizations.

In essence, the rapid integration of generative AI into enterprises signals a transformative phase in the business world. Companies are not just experimenting with AI; they are actively incorporating it into their core operations, driving growth, and enhancing their competitive edge.

Image: O'Reilly

Emerging Trends in AI Use

The O'Reilly report sheds light on how enterprises are currently leveraging generative AI, revealing key trends in its application. A substantial majority, 77%, are using AI for programming tasks, indicating a significant shift towards automation in software development. Tools like GitHub Copilot and ChatGPT are becoming increasingly popular, enhancing productivity and efficiency in coding.

Data analysis emerges as the second most common use case, with 70% of enterprises employing AI for this purpose. The ability of AI to process and analyze large datasets is proving invaluable, enabling businesses to gain deeper insights and make more informed decisions.

Customer-facing applications are also a major area of focus, with 65% of enterprises using generative AI to enhance customer experiences. This includes chatbots, personalized recommendations, and automated customer support, all aimed at providing more engaging and responsive interactions.

Interestingly, the survey also highlights generative AI’s role in content creation. About 47% of enterprises use AI for marketing copy, and 56% for other forms of copy, showcasing AI’s growing influence in creative domains.

These trends reflect a broader shift in enterprise strategy. Generative AI is no longer just a tool for efficiency; it’s becoming a core component in driving business innovation. By automating routine tasks, providing insights through data analysis, and enhancing customer engagement, AI is enabling businesses to explore new opportunities and redefine their operational models. This utilization of AI across various functions underlines its transformative impact and versatility in the enterprise sector.

Generative AI Challenges and Barriers

Despite the rapid adoption of generative AI in enterprises, the O'Reilly report identifies significant challenges and barriers. The foremost obstacle, as cited by 53% of respondents, is identifying appropriate use cases for AI implementation. This challenge underscores a gap in understanding how best to leverage AI technologies effectively within specific business contexts.

The second major barrier involves legal, risk, and compliance issues, mentioned by 38% of respondents. As AI technology advances, enterprises are grappling with the complexities of integrating these systems while adhering to legal standards and mitigating risks, particularly in areas like data privacy and ethical AI use.

These findings highlight the need for a more nuanced approach to AI integration. Enterprises must not only be technologically ready but also strategically prepared to identify the right applications and navigate the complex legal landscape surrounding AI.

Demand for AI Skills and Risk Management

The accelerated integration of generative AI has created a significant demand for skilled technology workers. Skills in AI programming are most sought after (66%), followed closely by data analysis (59%) and operations for AI/ML (54%). This demand reflects the growing complexity and sophistication of AI systems and the need for specialized expertise to develop and manage these technologies.

In terms of risk management, enterprises are primarily concerned with unexpected outcomes (49%), security vulnerabilities (48%), and issues related to safety, reliability, fairness, bias, ethics, and privacy (each cited by 46% of respondents). These concerns highlight the need for rigorous testing and validation of AI systems, as well as the development of robust frameworks to address ethical considerations and ensure responsible AI use.

Image: O'Reilly

Reflecting the Early Stages of AI Adoption

While the adoption rate is high, the report reflects that many enterprises are still in the early stages of implementing generative AI. About 34% are at the proof-of-concept stage, exploring the capabilities and potential applications of AI. Another 14% are in the product development phase, and 10% are in the process of building models. Notably, 18% have advanced to having AI applications in production, indicating a swift movement from theoretical exploration to practical application.

Among respondents, a significant 64% have transitioned from using prepackaged AI solutions to developing custom applications. This shift represents a considerable advancement, signaling that enterprises are not just adopting AI but are also innovating and creating bespoke AI solutions tailored to their specific needs.

The report also highlights a diverse AI ecosystem beyond the well-known GPT models. For instance, 16% of companies are building on open-source models, showcasing an active community engaged in developing and sharing AI technologies. The use of less common models like LLaMA and Google Bard, though still in the minority, indicates an openness to a wide range of AI technologies, fostering a dynamic and innovative AI landscape.

These findings point to a rapidly evolving AI environment in enterprises, marked by a shift from experimentation to practical application and innovation. The diversity in AI model usage and the move towards custom solutions underscore the dynamic nature of the field and the eagerness of enterprises to explore and harness the full potential of AI technologies.

The O'Reilly report not only highlights the current state of generative AI in enterprises but also serves as a call to action. It urges businesses to actively participate in shaping the future of AI, fostering an environment where technology serves as a catalyst for growth, innovation, and ethical progress.

You can download the full report here.

Microsoft’s Satya Nadella is surely a tech coach of the AI world. While the OpenAI drama was unfolding with Microsoft shares on the slide, he somehow managed to catch up with the Cricket World Cup finals, and put a stop to all the chaos.

In a recent conversation with Kara Swisher, Nadella discusses the OpenAI soap opera, talking about India-Australia Men’s ODI Cricket World Cup finals, and if he would be interested in buying the Australian cricket team.

“That would be like buying OpenAI,” quipped Nadella, saying that none of these things can be accomplished. “But we can partner well with OpenAI and watch Aussies play cricket.” That explains why he hired five of the best players from OpenAI, instead of acquiring the company. Does that make Microsoft the new OpenAI?

Just when you thought that OpenAI drama was all over, things are taking a very interesting turn. A few hours after Sam Altman joined Microsoft to be the CEO of their new AI research team, OpenAI wants him back. But all of that is in the dust again, as Nadella confirmed that Altman is going to lead the new AI division within Microsoft. Nadella is reinforcing this with multiple interviews, clearing the air once and for all.

Altman has confirmed this.

satya and my top priority remains to ensure openai continues to thrive
we are committed to fully providing continuity of operations to our partners and customers
the openai/microsoft partnership makes this very doable

— Sam Altman (@sama) November 20, 2023

OpenAI is nothing without its people

Interestingly, almost all the employees are deserting OpenAI, mostly to join Microsoft. As of now, out of the 770 employees at the company, 715 have left, some with tenuous visa situations as well.

“It’s silly to think that one company with little technology is going to make the difference. We love the partnership, but we are not single threaded,” said Nadella.

Microsoft’s entire Azure infrastructure, and all of its Copilot products and services are built around this “little tech company‘s” AI models. Moreover, Altman has repeatedly said that he loves the OpenAI team and appreciates the work that the team is doing.

the openai leadership team, particularly mira brad and jason but really all of them, have been doing an incredible job through this that will be in the history books.
incredibly proud of them.

— Sam Altman (@sama) November 20, 2023

It is funny to see how just a few days back at Microsoft Ignite and even the OpenAI DevDay, Nadella had highlighted how much he loves OpenAI, and wishes to continue working with them. But now that the board has decided to flip around so much with its employees, Nadella just wants to keep the talent directly under Microsoft, almost discarding OpenAI.

It all started with Mira Murati’s post on X, “OpenAI is nothing without its people.” Just a few minutes later, the feed was filled with people writing the same thing on their timelines. To Murati’s post, Altman replied with a heart. This took a turn to a mass exodus of employees.

Nadella highlighted that Altman was the only one he was in close contact with when he was at OpenAI. Now, he wants Altman to lead the new AI research lab at Microsoft, along with Greg Brockman, which is going to be similar to the GitHub, LinkedIn, or Mojang Studios team, as he highlighted.

Interestingly, Nadella wants Altman to be a full-time dedicated employee of the company, which means a lot of his other ventures such as WorldCoin would be at stake, but the story is still developing.

On the other hand, OpenAI is left with nothing but just the board, which is also expected to leave, along with Ilya Sutskever, who is regretting all the actions of the board, given that all of the employees are leaving. Though Microsoft continually wants to work with OpenAI, it would not be possible if the team does not exist at the non-profit company. But for Microsoft, it now has all the best human capital in the world to build the next AGI.

I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.

— Ilya Sutskever (@ilyasut) November 20, 2023

“I have not known him before, I met him for the first time last night,” Nadella said about the new interim CEO of OpenAI Emmett Shear. But for Altman, he said, “Sam chose Microsoft once, Sam chose Microsoft twice,” he said. Nadella was hell bent on not losing out on the key employees of OpenAI, as they are the leaders of the AI world.

Microsoft is now about AGI

“We were very happy on Friday morning, and we will go back to operating how we were from Monday,” said Nadella about whatever drama unfolded over the weekend. “We have learnt a lesson or two, and we will double down on it.”

Even though a lot of investors, along with the team, want Altman back as the CEO of OpenAI, Altman is ready to work with Microsoft. For how long, that is still a question. The best part about all this is that the company wants to keep delivering what it was delivering all this while, but now with OpenAI on board, a lot of that would be directed towards developing AGI.

pic.twitter.com/aeg2gV04Aw

— Elon Musk (@elonmusk) November 20, 2023

“If everything disappeared, we will still have everything we need in order to just continue innovating,” expressed Nadella. Given the compute capabilities that Microsoft already has, probably the largest one in the world, the OpenAI team might be best to work directly with Microsoft now, without the shackles of being stuck with a non-profit organisation.

“We cannot let our customers feel like there are going to be any more surprises,” highlighted Nadella. Moreover, Nadella is extremely happy with Altman in his company. When asked if there is going to be a clash between Altman’s and his vision in the company, Nadella said, “Lets just say it is fantastic to have a lot of capable people who can become the CEO of Microsoft.”

The post Satya Calls OpenAI a ‘Little Tech Company’ appeared first on Analytics India Magazine.

Hansa Cequity has once again been certified as the Best Firm for Data Scientists to work for by Analytics India Magazine (AIM) through its workplace recognition programme.

The Best Firm For Data Scientists certification surveys a company’s data scientists and analytics employees to identify and recognise organisations with great company cultures. AIM analyses the survey data to gauge the employees’ approval ratings and uncover actionable insights.

“We are delighted to be chosen as the Best Firm For Data Scientists by Analytics India Magazine. At Hansa Cequity, we are integrating the power of data and AI to solve strategic marketing problems in a holistic manner with extreme passion and customer-centricity. We are continuously innovating with cutting-edge AI algorithms & cloud with a sharp focus to bring a much-needed paradigm shift in the industry. We will continue to provide the data-rich insights to the CXO community that enable the positive impact to their business topline” said Prasad Kothari, Head – Data Science & AI at Hansa Cequity.

The analytics industry currently faces a talent crunch, and attracting good employees is one of the most pressing challenges that enterprises are facing.

The certification by Analytics India Magazine is considered a gold standard in identifying the best data science workplaces and companies participate in the programme to increase brand awareness and attract talent.

Best Firms for Data Scientists is the biggest data science workplace recognition programme in India. To nominate your organisation for the certification, please fill out the form here.

The post Hansa Cequity Certified as Best Firm for Data Scientists for the 2nd Time appeared first on Analytics India Magazine.

By Andrea De Mauro and Mahantesh Pattadkal

As we pick up from where we left off in Part 1 of the blog series "Job Trends in Data Analytics", our journey through the world of data analytics job trends and the role of Natural Language Processing (NLP) continues.

In Part 1, we introduced the "Data Analytics Job Trends" application, which is all about gathering data and applying NLP to analyze it, powered by KNIME Analytics Platform. We discussed the web scraping phase used to collect live data regarding the data analytics job market, followed by the process of cleaning up the data using NLP techniques. We then introduced a topic model that revealed seven homogeneous skillsets within job postings. Such skillsets represent the competencies and activities employers across various industries seek in data analytics professionals.

In the second part of the blog series, we will describe the identified skillsets and make some data-backed considerations on the evolving landscape of professional careers in Data Science.

The Topics and Their Descriptions

To label the skillsets, we use the most frequent terms and weights identified through the LDA algorithm that was previously applied to the job postings. We further analyze the job descriptions in each topic to highlight the key activities, essential skills, and industries where they are most commonly found. Understanding these topics can help job seekers align their skillsets with the market demands and increase their chances of securing a suitable position in the field of Data Analytics. In the following paragraphs, you will find a brief description of each skillset.

Topic 0: Research and Data Analysis

The following table shows the top five terms and their weights for topic 0. The weights refer to the significance of the term in defining that particular topic. Considering these terms and the documents labeled as topic 0, we interpret this skillset to be “Research and Data Analysis”.

Term	Weight
Research	4510
Position	4195
Information	4112
Health	3404
University	2118

Table 0: Term-Weights for Topic 0

This skillset encompasses activities such as conducting research, analyzing data, and providing insights that drive decision-making in various sectors. As a cornerstone of data analytics, this skillset facilitates the extraction of valuable insights from data, trend identification, and informed decision-making.
From what we gathered within the corpus of job posts, the fundamental competency requirements connected with this skillset are:

• Strong analytical and problem-solving abilities
• Expertise in statistical software (R, Python)
• Experience with data visualization tools
• Effective communication and documentation skills
• A background in relevant field (mathematics, statistics, or data science)

Topic 1: Administration and Customer Support

By looking at the terms and weights from Table 1 and at the documents associated with Topic 1, we decided to label it as “Administration and Customer Support”. This skillset entails managing customer interactions, providing administrative support, and coordinating logistics or procurement processes.

Term	Weight
Support	2321
Management	2307
Information	2134
Position	2126
Customer	1909

Table 1: Term-Weights for Topic 1

In our opinion, the fundamental competencies necessary to succeed in jobs requiring this skillset are:

• Strong organizational and time management abilities
• Attention to detail
• Proficiency in office software and communication tools
• Excellent interpersonal and problem-solving skills

Topic 2: Marketing and Product Management

Based on the terms shown in Table 2, we interpret this to be the “ Marketing and Product Management ” skillset.

Term	Weight
Business	8487
Team	8021
Product	6825
Customer	3923
Marketing	3740

Table 2: Term-Weights for Topic 2

This skillset revolves around developing marketing strategies, managing product lifecycles, and driving market growth. It's vital in data analytics-focused jobs, as it allows professionals to use data-driven insights to make informed decisions regarding market trends, customer preferences, and product performance.

The essential competencies required within the Marketing and Product Management skillset are:

• Strong analytical and strategic thinking abilities
• Expertise in market research and competitive intelligence
• Experience with marketing tools and platforms
• Excellent communication and leadership skills
• A background in business, marketing, or a related field

Topic 3: Business Management, Data Governance, and Compliance

Based on the terms shown in Table 2, we concluded that it referred to the “Business Management, Data Governance, and Compliance ” skillset.

This skillset encompasses overseeing business operations, ensuring data quality and security, and managing risk and regulatory requirements. In data analytics-intensive jobs, this skillset enables maintaining data integrity, compliance monitoring, risk identification, and business process optimization using data-driven insights.

Term	Weight
Business	14046
Management	10531
Team	5835
Analysis	5672
Project	4309

Table 3: Term-Weights for Topic 3

According to our findings, the required competencies within this skillset are:

• Strong organizational and leadership abilities
• Expertise in data management, data governance and risk assessment
• Experience with regulatory frameworks and industry standards
• Effective communication and problem-solving skills
• A background in business, finance, or a related field

Topic 4: Business Intelligence and Data Visualization

Looking at the terms we found within Topic 4, we call it the “Business Intelligence and Data Visualization” skillset.

This skillset involves designing ever-present BI solutions such as dashboards and reports, creating insightful visualizations, and analyzing data for informed decision-making. It's pivotal in jobs leveraging data analytics, transforming raw data into actionable insights that drive strategic decisions.

Term	Weight
Business	19372
Analysis	7687
Power bi	7359
intelligence	7040
Sql	5836

Table 4: Term-Weights for Topic 4

In our opinion, the fundamental competency requirements within BI and Data Visualization are:

• Strong analytical and problem-solving abilities
• Expertise in BI tools (like Power BI, Tableau, SQL)
• Experience with data visualization techniques
• Effective communication and storytelling skills

Topic 5: Data Warehouse and Cloud Infrastructure

Based on the terms shown in Table 5, we interpret this to be the “Data Warehouse and Cloud Infrastructure ” skillset.

Job posts requiring a cloud and big data engineering skillset are typically connected with activities such as designing and implementing cloud-based solutions, managing large-scale data processing, and developing software applications. It's vital in data analytics-focused jobs, enabling efficient processing and analysis of large data volumes for valuable insights.

Term	Weight
Development	4525
Cloud	3998
Engineering	3692
Software	3510
Design	3494

Table 5: Term-Weights for Topic 5

In our opinion, the fundamental competency requirements related to skillset are

• Strong programming and problem-solving abilities
• Expertise in cloud platforms (like AWS, Azure, and Google Cloud)
• Experience with big data technologies (like Hadoop, Spark, and NoSQL databases)
• Knowledge of Information Security policies and related processes

Topic 6: Machine Learning

Based on the terms shown in Table 6, we interpret this to be the “Machine Learning ” skillset, which revolves around designing AI models, researching cutting-edge ML techniques, and developing intelligent software solutions. In data analytics-intensive jobs, it forms the basis for AI model training and performance optimization.

Term	Weight
Machine	9782
Science	8861
Research	4686
Computer	4209
Python	4053

Table 6: Term-Weights for Topic 6

According to our findings, the fundamental competencies required in machine learning today are

• Strong programming and mathematical abilities
• Expertise in machine learning frameworks (like TensorFlow, PyTorch)
• Experience with advanced AI techniques (like deep learning, and natural language processing)
• Effective communication and collaboration skills

The Skillsets and the Professional Profiles

In this installment, our focus turns to the intricate analysis of skillset associations as revealed through topic modeling across three distinct professional profiles: Data Engineer, Data Analyst, and Data Scientist. To align these professional profiles with job postings, we leveraged a rule-based classifier. This classifier managed to determine the profile designation of a job listing based on keywords found within the job title. For instance, a job post titled "Data Architect" would be categorized as a Data Engineer role, while a posting titled "Machine Learning Engineer" would be attributed to the Data Scientist category.

Using Latent Dirichlet Allocation (LDA) topic modeling furnishes us with topic weights for each job posting, spanning seven distinct skillsets. By calculating the mean weight of each skillset across all professional profiles, we arrive at the average skillset weight specific to each role. Notably, these weights are then normalized and represented as percentages.

As Illustrated in Figure 1, we present an insightful visualization of the interplay between professional designations and corresponding skillsets. This visual encapsulates the collective anticipation of employers concerning the fundamental proficiencies crucial for Data Engineers, Data Analysts, and Data Scientists.

As anticipated, the role of Data Engineer prominently necessitates mastery in the "Data Warehouse & Cloud Infrastructure" skillset. Moreover, a supplementary grasp of Visualization and Machine Learning is imperative. This emphasis on skill diversity can be attributed to the anticipation that Data Engineers will be integral in supporting both Data Analysts and Data Scientists.

Conversely, the paramount expertise projected for Data Scientists lies in "Machine Learning," closely followed by a proficiency in "Research" methodologies. Notably, a hybrid skillset encompassing "Business Management" and "Product Management" also ranks high in significance. This encapsulates the intricate array of competencies sought by the job market for aspiring Data Scientists.

Turning our attention to the Data Analyst domain, a pivotal requirement emerges for proficiency in "BI and Visualization." Given their role in generating business reports, driving dashboards, and monitoring business vitality, this comes as no surprise. The parallel demand for "Business Management" as a secondary key skill mirrors the strategic acumen expected from this role. Moreover, akin to the Data Scientist role, there's a parallel requirement for "Product Management" and "Research" proficiencies within the Data Analyst spectrum.

In summation, this exploration underscores the nuanced landscape of skillset prerequisites across various Data Analytics roles. It portrays employers' multifaceted expectations for candidates aspiring to excel in the capacities of Data Engineers, Data Analysts, and Data Scientists.

Figure 1: The Radar Plot displays the association between professional profiles plotted against the skillsets shown in dimensions (click to enlarge).
Conclusion

Our analysis of job postings in the expanding field of Data Analytics aims to categorize jobs based on distinct skillsets and clarify the diverse range of abilities required in each category. With exponential growth in this domain and the critical nature of decisions made based on data, the process of collecting, storing, and analyzing data has seen remarkable advances, leading to an insatiable demand for professionals skilled in data analytics.

Through the classification of job postings into seven notable skillset topics, we shed light on the necessity for both specialized and multifaceted skills in this rapidly changing field. The topics ranged from data analysis and business intelligence to machine learning and artificial intelligence, underscoring the surging demand for individuals adept at harnessing data, technology, and cross-functional teamwork.

Notwithstanding, this study has several limitations. The dynamic nature of the job market and the emergence of novel technologies and methodologies require continuous updating of our analysis versus a static “snapshot” view as we did here. Furthermore, our approach may not have captured every nuance of the diverse job roles and skills in the Data Analytics arena, given the reliance on available job postings at the time of research.

All our work is freely available at KNIME Community Hub Public Space — “Job Competency Application”. You can download and play with the workflows to try out and discover for yourself and extend or improve.

What Next?

Looking ahead, we see the potential for considerable expansion of this study. This includes the development of KNIME components to implement the 'Stop Phrases removal' method, described in Part 1, and a human-in-the-loop interactive visualization framework in KNIME. Such a framework would simplify the process of human judgment in selecting the most coherent topic model for a given corpus, enhancing the scaling of our work. We also envision the application of LLM-aided mechanisms to support and simplify the topic modeling phase: this scenario certainly leaves room for further experimentation and research.

Professionals in the Data Analytics field must remain informed and adaptable in the face of emerging technologies. This ensures that their skillsets stay relevant and valuable in the ever-changing landscape of data-driven decision-making. By recognizing and cultivating the skills related to the identified topics, job seekers can gain a competitive edge in this vibrant market. To protect their relevance in the field, Data Analytics professionals must remain curious throughout their careers and continue to learn continuously.

Mahantesh Pattadkal brings more than 6 years of experience in consulting on data science projects and products. With a Master's Degree in Data Science, his expertise shines in Deep Learning, Natural Language Processing, and Explainable Machine Learning. Additionally, he actively engages with the KNIME Community for collaboration on data science-based projects.

Andrea De Mauro has over 15 years of experience building business analytics and data science teams at multinational companies such as P&G and Vodafone. Apart from his corporate role, he enjoys teaching Marketing Analytics and Applied Machine Learning at several universities in Italy and Switzerland. Through his research and writing, he has explored the business and societal impact of Data and AI, convinced that a broader analytics literacy will make the world better. His latest book is 'Data Analytics Made Easy', published by Packt. He appeared in CDO magazine's 2022 global 'Forty Under 40' list.

More On This Topic

• Job Trends in Data Analytics: NLP for Job Trend Analysis
• 5 Key Data Science Trends & Analytics Trends
• Are you satisfied in your job? Take our Data Community Job…
• Top Data and Analytics Trends
• AI, Analytics, Machine Learning, Data Science, Deep Learning…
• Data Science & Analytics Industry Main Developments in 2021 and Key…