Рубрика: AI

The GLM-130B framework is a bilingual pre-trained large language model with over 130 billion parameters capable of generating text outputs in both English and Chinese. The GLM-130B framework is an attempt to open source a language model at a scale of over 100B parameters, and discuss how frameworks of such a large scale can be pre-trained because currently, training a model of such a large scale is often rattled with issues like divergence & loss spikes.

In this article, we will be talking about the GLM-130B framework, which attempts to devise a method to effectively pre-train large language models with hundreds of billions of parameters. We will take a deeper dive into the working & architecture of the GLM-130B framework along with the training process & design choices that not only helps in increasing the efficiency, but also the stability. Initial experiments carried out to test the working of the GLM-130B framework on a wide array of English benchmarks resulted in the GLM-130B model outperforming the current state of the art GPT-3 framework by a considerable margin. So let’s begin, and explore how the GLM-130B framework delivers such consistent, accurate, and stable results.

An Introduction to the GLM-130B Framework

Large Language Models capable of operating in few-shot & zero-shot settings, especially those with over 100 billion parameters present attractive scaling laws, out of which, the GPT-3 framework is one of the best performing frameworks that delivers considerable performance upgrades over its predecessor, the BERT framework. However, despite the popularity of the GPT-3 framework, and its widespread applications, the training process, and in some ways, the GPT-3 framework in itself has been non transparent to the public. Furthermore, empirically enumerating all the possible designs for training LLMs over 100B parameters is computationally unaffordable which makes it even more critical to come up with a pre-training method for large scale LLM frameworks.

The above point makes sharing the working, and the training process of high-quality large-scale LLM frameworks like GPT-3 is of critical value, and with the ethical concerns kept in mind, the GLM-130B framework is an attempt to pre-train an accurate, and open-source LLM with over 100B parameters. During the course of their attempt, the GLM-130B development team observed that pre-training a large scale LLM framework is often accompanied with a wide array of engineering & technical challenges in terms of pre-training stability, efficiency, and convergence.

To be more specific, the GLM-130B is a bidirectional, and bilingual dense framework consisting over 130B parameters, pre-trained over 400B tokens on a cluster of 96 NVIDIA DGX-A100 GPU nodes over a span of nearly two months. Furthermore, instead of opting for the GPT-style architecture, the GLM-130B framework uses the GLM or General Language Model algorithm in an attempt to leverage its autoregressive blank infilling objectives, and the bidirectional attention advantage. The following table compares the GLM-130B framework with other models with over 100B parameters including GPT, BLOOM-176B, and OPT-175B.

The engineering and development concepts involved in the GLM-130B framework outperforms almost every large-scale LLM framework including GPT-3, and PaLM 540B with over 500B parameters in a lot of cases, and across a wide array of benchmarks. The following figure compares the performance of the GLM-130B framework with models with over 100B+ parameters, and as it be seen, the GLM-130B framework has significantly less generation toxicity, and bias than its counterparts.

Finally, the GLM-130B has been designed in a way to allow as many developers to conduct studies on frameworks with over 100B parameters, and there are two ways in which the GLM-130B framework achieves this. Firstly, instead of using over 175B parameters like BLOOM & OPT, the GLM-130B framework uses 130B parameters, because the size of the model supports interference even on a lone A100 server. Secondly, the GPU requirements to run the GLM-130B framework is less when compared to other LLM frameworks, and the GLM-130B framework achieves this by quantizing the original framework into INT4 precision. The INT4 quantization used by the GLM-130B framework enhances the performance while maintaining negligible performance degradation.

GLM-130B : Architecture

The inductive bias of a machine learning model is described by its architecture, and it doesn’t come as a surprise when developers cannot explore various architectural designs for large language models given the computational affordability, and viability. With that being said, let’s have a look at GLM-130B’s architecture.

Large-scale LLM frameworks like PaLM, GPT, and more have over 100B parameters, and they are built on the conventional decoder-only GPT-style architecture for autoregressive language modeling. On the other hand, the GLM-130B framework explores the possibility of using a bidirectional General Language Model or GLM, a transformer-based language model that aims to leverage autoregressive blank filling as the training objective, as its foundation. Briefly, for a given text sequence the GLM framework samples the text spans that are then replaced with a single mask token.

The bidirectional attention of the General Language Model over uncorrupted or unmasked contexts is what separates the GLM-130B framework from the GPT-style approach that makes use of a unidirectional approach. Furthermore, to support both generation & understanding of data, the GLM framework amalgamates two corruption strategies, each of which is indicated with a special & unique mask token.

• [MASK] : [MASK] is a corruption strategy that uses short blanks in sentences, the lengths of which add up to a certain percentage of the input.
• [gMASK] : [gMASK] is a corruption strategy that makes use of random-length blanks towards the end of the sentence with the prefix contexts.

The approach followed by the GLM framework is what allows the framework to record an accuracy score of over 80% on zero-shot LAMBADA language modeling, and outperforms both the PaLM 540B, and the GPT-3 framework.

Layer Normalization

One of the major challenges faced by developers when training a LLM framework is the training instability, and using an appropriate LN(Layer Normalization) might help with the training of LLMs. The GLM-130B framework uses a Post-LN approach thanks to its performance on downstream tasks.

FFNs and Positional Encoding

Feedforward Neural Networks or FFNs and positional encoding are two approaches adopted by the GLM-130B framework to introduce high-end downstream performance & training stability.

Pre-Training Setup

The pre-training objectives of the GLM-130B framework not only includes multi-task learning for a small number of tokens, but also includes the self-supervised GLM for autoregressive filling of the blanks, with the expectation that this approach will help the GLM-130B framework in downstream tasks. With that being said, the pre-training setup of the GLM-130B framework looks like the following.

Self-Supervised Blank Filling

As already mentioned, the GLM-130B framework uses two corruption strategies namely the [MASK] and [gMASK], and one of these strategies is independently applied to every individual training sequence, one at a time. For infilling the blanks, the [MASK] strategy masks consecutive spans in 30% of the training sequence, where the lengths of the spans add to up to 15% of the input, and follows a Poisson distribution. For the remaining 70% of the sequence, the prefix of every sequence is kept as a context, and the [gMASK] strategy helps in masking the rest of it, and the masked length is then sampled using the Uniform distribution.

Multi-Task Instructions Pre-Training

It has been indicated that following a multi-task learning approach for pre-training the models can deliver better results than fine-tuning, to improve task transfers in a zero-shot setting. Subsequently, the GLM-130B framework proposes to use an array of instruction prompted datasets including language generation, understanding, and information extraction during pre-training.

When compared to other approaches for zero-shot task transfer that make use of multi-task prompted fine-tuning, the Multi-Task Instructions Pre-Training approach followed by the GLM-130B framework accounts only for 5% of the total tokens, and it is set during the pre-training phase in an attempt to prevent spoiling other abilities of the LLM framework or in other words, unconditional free generation.

3D Parallel Strategy

There are two de facto practices for training large scale models with billions of parameters, the tensor model parallelism and the data parallelism. In an attempt to minimize the GPU utilization, and to handle immense GPU requirements, the GLM-130B framework implements a 3D parallel strategy that combines the pipeline model parallelism strategy with the tensor model parallelism and the data parallelism strategies.

GLM-130B : Training Stability

Training stability is an important factor when determining a LLM’s quality, and the training stability is influenced heavily depending on the number of tokens it passes through. Furthermore, it is vital to establish a trade-off between stability and efficiency with regards to floating point formats given the computing restraints. For example, low precision floating point formats boost the computing efficiency, but they often result in training collapses given they are prone to underflow and overflow errors.

Mixed Precision

In an attempt to boost training accuracy and reduce memory usage, the GLM-130B framework follows the common practice of using mixed precisions i.e FP16 for both forward & backwards, and FP32 for both master weights and optimizer states. Just like other popular LLM frameworks including BLOOM-176B and OPT-175B, the training phase of the GLM-130B framework using the mixed precision strategy faces frequent loss spikes, and the frequency of these spike losses tend to increase as the model continues to train. Furthermore, there are major issues that developers face when they are scaling up the transformers.

First, the value scale of the main branch of the transformer can be vast in the deeper layers when using Pre-LN, and in the GLM-130B framework, it is addressed by using a DeepNorm based Pre-LN, which ensures that the value scale remains bounded at all times. Second, as the model scales up, the attention scores grow to a point where they exceed FP16’s range.

Embedding-Layer Gradient Shrink or EGS

Developers working on the GLM-130B framework identified that the gradient norm can act as an informative indicator for training collapses, and a training collapse usually lags behind a spike in the gradient norm. The cause for these spikes is the abnormal gradients of the embedding layer, and developers observed that when compared to the gradient norm of other layers, the gradient norm of the embedding layers is larger by several magnitudes, and it also tends to fluctuate dramatically during the early training of the framework. Vision models also face this issue, and it is handled by freezing the patch projection layer. However, the same approach cannot be applied to LLMs as in language models, you cannot freeze the projection layers.

GLM-130B : Results and Performance

To evaluate GLM-130B’s performance for English tasks, it implements the same settings followed by common LLM frameworks including PaLM and GPT-3, and as the GLM-130B is a bilingual framework, it is also evaluated across several Chinese benchmarks. The GLM-130B framework’s performance will be measured across multiple benchmarks including Language Modelling, MMLU or Massive Multitask Language Understanding, BIG-Bench or Beyond the Imitation Game Benchmark, and CLUE or Chinese Language Understanding Evaluation. So let’s get started.

Language Modeling

The Language Modeling benchmark test on the GLM-130B framework is performed across two datasets: LAMBADA, and Pile.

The LAMBADA dataset is used to test the last word modeling capabilities of LLMs, and the GLM-130B framework achieves a zero-shot accuracy score of 80.2 in a bilingual setting, and in route, set a new benchmark record on the LAMBADA dataset.

On the other hand, Pile is a test set that comprises a series of benchmarks for language models. On average, in comparison to the GPT-3 and Jurassic-1, the GLM-130B framework delivers its best performance on 18 shared test sets in terms of weighted BPBs. The results demonstrate the strong language capabilities of the GLM-130B framework, and the results are included in the table below.

MMLU or Massive Multitask Language Understanding

MMLU or Massive Multitask Language Understanding is a diverse benchmark that comprises over 50 multiple-choice question answering tasks concerning human intelligence & knowledge, ranging from high-school to expert levels, and it is released after the crawling of the Pile test set, and thus, it serves as an ideal test-best to evaluate the few-shot learning capabilities of a LLM.

As it can be seen, in a few shot settings(5-shot), the performance of the GLM-130B framework approaches the performance of the GPT-3 model after viewing close to 300B tokens. The performance continues to boost as the training proceeds further, and when the training ends, the framework achieves an accuracy score of 44.8 after viewing a total of 400B tokens.

BIG-Bench or Beyond the Imitation Game Benchmark

BIG-Bench or Beyond the Imitation Game Benchmarks challenging tasks tests a model’s ability on knowledge, reasoning, and commonsense. As demonstrated in the following figures, in zero-shot setting, the GLM-130B framework outperforms both PaLM 540B and GPT-3 175B frameworks which might be because of MIP and the bidirectional context attention to boost the GLM-130B’s performance in unseen tasks in zero-shot setting. Furthermore, as the number of shots increases, the performance of the GLM-130B framework also improves, outperforming the GPT-3 framework consistently.

CLUE or Chinese Language Understanding Evaluation

GLM-130B’s Chinese zero-shot performance is evaluated on established NLP benchmark tasks including CLUE and FewCLUE, and is compared against 260B ERNIE Titan 3.0, the largest existing Chinese language model. As it can be observed, the GLM-130B framework constantly outperforms the 260B ERNIE Titan 3.0 framework across 12 different tasks, and performs nearly 260% better than the ERNIE framework on two abstractive MRC datasets.

Conclusion

In this article, we have talked about GLM-130B, a bilingual pre-trained large language model that aims to promote inclusive LLM research. The architecture, engineering, and technical undertakings aims to provide the AI community with a better insight into the architecture of LLM frameworks, training efficiency & stability, pre-training objectives, and affordable interference.

Image by Author

Data analysis is a crucial part of making data-driven decisions in business, research, and everyday life. It involves extracting insights and patterns from data to gain a deeper understanding of the underlying information. With the introduction of ChatGPT's new Vision capability, data analysis has taken a significant leap forward. ChatGPT Vision allows users to interpret images, equations, graphs, and charts, opening up a wide range of possibilities for extracting insights from visual data.

In this article, we will explore 5 key ways ChatGPT Vision can be used for data analysis tasks.

1. SQL Table

You can now simply take the screenshot of the dataset and ask ChatGPT to write a SQL query for you.

Demo Database from programiz.com

Prompt:

I have uploaded three tables. Please write an SQL query to determine whether John has received his keyboard.

As we can see, the SQL query ran perfectly and I got my answer (Pending) too.

SELECT s.status  FROM Customers c  JOIN Orders o ON c.customer_id = o.customer_id  JOIN Shippings s ON o.order_id = s.shipping_id  WHERE c.first_name = 'John' AND o.item = 'Keyboard';

Result:

The ChatGPT Vision functionality enables non-technical managers to run queries on multiple relational tables and complex questions.

2. Chart Analysis

Using ChatGPT Vision for chart analysis is the best way to understand what each graph is telling you about.

In our case, we have provided multiple images of data analysis plots from the Evolution of Digital Learning During COVID19 notebook and asked ChatGPT to write us the detailed report.

Prompt:

As a data scientist, it can often take a full day to create a proper data analysis report. However, with ChatGPT, we were able to produce the report within a minute. It was able to identify hidden patterns that I had missed during my initial analysis. Once again, ChatGPT Vision proved to be a valuable and reliable assistant.

3. Dashboard

Next, we will provide a more complex image of the Super Sample Superstore dashboard to help understand each component and its meaning.

Prompt:

Can you please explain each dashboard section in detail? 

ChatGPT did a great job by providing a detailed explanation of the dashboard from a simple prompt. Additionally, it explained the numbers and charts on the dashboard, such as KPIs, trends, and regional comparisons.

4. Evaluation

I often struggle with evaluating results and making sense of them. For instance, when I am trying to determine the best number of clusters for the KMeans algorithm using the Alcoholic Drinks Promotion in Russia dataset. So, instead of checking for multiple clusters, I will provide an Elbow plot to ChatGPT Vision and ask it to pick a number for me.

Prompt:

I have uploaded the elbow plot to figure out the optimal number of clusters for the KMeans algorithm. Please pick the number for me. 

You can also use this new functionality to better understand the machine learning results. For example, to understand the classification report of the Mobile Price classification model.

Prompt:

I have uploaded the classification report of Mobile Price Classification. Can you please explain the result? 

Using these results, I can easily explain our initial results to my non-technical manager and stakeholders. It just made my life easy.

5. Equations

The best use of ChatGPT Vision is to use it to understand various math equations on research papers, websites, videos, and blogs. You can just take a screenshot of the equation and ask ChatGPt to explain it for you in simple words. Like we asked it to explain the equation of singular value decomposition.

Prompt:

Can you explain the singular value decomposition using the uploaded image of equations

In another example, we will be asking ChatGPT to convert the reward function from Fine-Tuning Language Models from Human Preferences research paper into Latex.

Prompt:

Can you convert the reward function equation into latex?

As you can see, the generated Latex code works perfectly.

Screenshot from Codecogs Final Thoughts

With its visual interpretation skills, ChatGPT has become an invaluable assistant for data scientists, analysts, researchers and even non-technical professionals looking to work with data. It eliminates the need for manual analysis and speeds up the process of going from data to insights. As the capabilities continue to improve, ChatGPT Vision promises to revolutionize the way we work with and understand data.

Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master's degree in Technology Management and a bachelor's degree in Telecommunication Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.

More On This Topic

• 5 Ways You Can Use ChatGPT's Code Interpreter For Data Science
• If You Can Write Functions, You Can Use Dask
• How our Obsession with Algorithms Broke Computer Vision: And how…
• How You Can Use Machine Learning to Automatically Label Data
• 7 Ways ChatGPT Makes You Code Better and Faster
• eBook: 101 Ways to Use Third-Party Data to Make Smarter Decisions

We are seeing a bunch of AI regulatory announcements from both sides of the Atlantic

President Biden with his executive order on safe, secure and trustworthy AI

and Prime Minister Rishi Sunak on his AI safety summit (which is a closed door event)

Both these initiatives are dominated by a specific view i.e. safety security and risk

Against this, Andrew Ng stands as one of the few voices of reason who says big tech is lying about AI extinction danger

I like Andrew Ng views and I think we should elaborate on these ideas more

So much so, I jokingly said “Andrew Ng for President”

But on a more serious note – this is a very serious issue and a lot is at stake.

Lets break down key points from Andrew Ng:

1) AI extinction of humanity is a bad idea.

2) Big tech is promoting this idea to create heavy regulation which will cripple competition – especially from open source.

3) The combination of existential threats and licensing is an even worse idea.

4) These could crush innovation and reward the big players.
5) The White House’s use of the Defense Production Act—typically reserved for war or national emergencies—distorts AI through the lens of security, for example with phrases like “companies developing any foundation model that poses a serious risk to national security.”

6) like electricity and encryption, AI is a dual-use technology ie can be used for civilian or military purposes, but conflating AI safety for civilian use cases and military applications is a mistake.
7) It’s also a mistake to set reporting requirements based on a computation threshold for model training. Today’s supercomputer is tomorrow’s pocket watch. So as AI progresses, more players — including small companies will run into this threshold.

8) Over time, governments’ reporting requirements tend to become more burdensome.

9) The right place to regulate AI is at the application layer ex healthcare, self-driving cars

10) Adding burdens to foundation model development unnecessarily slows down AI’s progress.

I agree with all these points

Image source: Andrew Ng

Alibaba Cloud Unveils AI Innovations To Challenge Amazon And Microsoft November 7, 2023 by Ali Azhar

At its annual flagship tech event Apsara Conference, Chinese tech giant Alibaba unveiled its upgraded artificial intelligence (AI) model as part of its ongoing commitment to fuel AI advancement in the country.

The upgraded AI model, named Tongyi Qianwen 2.0, is trained on vast amounts of data and is optimized for generative AI (GenAI) applications. The upgrade is designed to enable customers, developers, partners, and startups to fully harness the transformative potential of GenAI.

The new AI model includes the latest large language model (LLM). Alibaba called it a “substantial upgrade from its predecessor” which was launched earlier this year. Tongyi Qianwen 2.0 “demonstrates remarkable capabilities in understanding complex instructions, copywriting, reasoning, memorizing, and preventing hallucinations,” Alibaba said in a press release.

Alibaba is China’s biggest e-commerce and cloud computing player. It provides cloud computing for 80 percent of China’s tech companies, including a majority of the country’s businesses that are working on AI and LLMs. This deep integration with China’s tech landscape underscores the crucial role of Alibaba in the country’s AI advancement and strategy.

Several major cloud players in the market are offering tools for organizations to build their own GenAI applications using their own data. With the launch of the new AI model, Alibaba has positioned itself as a major global competitor to Microsoft’s Azure OpenAI Studio and Amazon Website Service’s Bedrock.

“We hope that through this cloud, it will become easier and affordable for everyone to develop and use AI, so we can help, especially small and medium-sized enterprises, to turn AI into huge productivity,” said Joe Tsai, Chairman of Alibaba Group.

As part of the new release, the Tongue Qianwen 2.0 features industry-specific models for enhanced business performance. These industries include healthcare, finance, legal, customer support, and code development. "Large language models hold immense potential to revolutionize industries. We're committed to using cutting-edge technologies, including generative AI, to help our customers capture the growth momentum forward," said Jingren Zhou, CTO of Alibaba Cloud.

To support the rising demand for cloud computing and data processing in GenAI development, Alibaba also announced a series of innovative cloud products This includes an upgrade to the Platform for Artificial Intelligence (PAI), which now features enhanced computing power, storage, model training, and interface capabilities. To help expedite the process of building customer GenAI applications, Alibaba Cloud has enhanced its database solutions including the cloud-native warehouse AnalyticsDB and cloud-native database PolarDB.

To enable an efficient and scalable cloud infrastructure for application development, Alibaba introduced the Alibaba Cloud Contained Compute Service (ACS). This serverless solution provides users with an option to integrate container services with cloud computing resources for enhanced efficiency and scalability.

Alibaba Cloud also announced its plan to open source its 72 billion-parameters version of Tongui Qianwen later this year. In addition, the tech giant announced the Alibaba Cloud Startup Catalytic Program to provide cloud credit support of up to $120,000, networking opportunities, and online learning memberships to promising global AI startups.

As the biggest tech player in China Alibaba’s success in its AI initiatives and programs will be critical to the overall growth and market penetration of AI technology in the country. With its focus now on AI-powered solutions, Alibaba is set to invest significant resources in achieving this goal. However, only time will tell whether Alibaba will catch up to the likes of Amazon and Microsoft in the field of AI.

Related Items

AI Safety Summit 2023 – Understanding Global AI Risks

Survey: Majority of US Workers Are Already Using Generative AI Tools, But Company Policies Trail Behind

AI’s Unstoppable Momentum Leaves Some Enterprise IT Teams Scrambling: AMD

Related

We are seeing a flurry of regulation

But we should ask ourselves if we are seeing regulatory capture — ie letting corporations write lax rules that lead to public harm.

Andrew Ng points out some contradictions:

“It’s also a mistake to set reporting requirements based on a computation threshold for model training. This will stifle open source and innovation: (i) Today’s supercomputer is tomorrow’s pocket watch. So as AI progresses, more players — including small companies without the compliance capabilities of big tech — will run into this threshold. (ii) Over time, governments’ reporting requirements tend to become more burdensome. (Ask yourself: Has the tax code become more, or less, complicated over time?)

The right place to regulate AI is at the application layer. Requiring AI applications such as underwriting software, healthcare applications, self-driving, chat applications, etc. to meet stringent requirements, and even pass audits, can ensure safety. But adding burdens to foundation model development unnecessarily slows down AI’s progress.”

source: https://lnkd.in/eWv89Pi2

But more to the point: “Regulation favours the incumbent”

The phrase “regulation favours the incumbent” refers to the idea that regulatory policies or legal frameworks often benefit existing, established companies (the incumbents) at the expense of new entrants or startups.

Here are some reasons why:

Barriers to Entry: Regulations can create high barriers to entry. For instance, new regulations might require businesses to get certain licenses, meet specific standards, or maintain particular records. While large, established companies might have the resources to meet these requirements, smaller startups or new entrants might struggle, making it difficult for them to compete.

Costs: Compliance with regulations often comes with costs – be it in the form of equipment, software, staff, or training. Established businesses, with their larger revenue streams and capital, are generally better equipped to absorb these costs than new entrants.

Economies of Scale: Incumbents often benefit from economies of scale. This means the more they produce, the lower the cost per unit. So, if both an incumbent and a new entrant have to invest in new machinery to meet a regulation, the cost per product or service unit might be much lower for the incumbent due to their larger scale.

Influence on Regulation: Established companies, especially in certain influential industries, may have more sway over policymakers and regulators. This means they can potentially shape regulations in ways that favor their operations or make it harder for new competitors to emerge.

Experience with Bureaucracy: Incumbents have often navigated regulatory environments for years or decades. They might have dedicated legal and compliance teams, and they know the ins and outs of the regulatory landscape. New entrants lack this experience and might find it challenging to navigate complex regulatory requirements.

Risk Aversion: Established companies have more to lose, and as a result, they might be more risk-averse. Regulations can serve to legitimize their cautious approaches, while startups, which often rely on disruptive and innovative models, might find those models hampered by regulations designed for older business paradigms.

Certainty and Stability: Incumbents often prefer a stable business environment. While they might grumble about regulations, the certainty they provide can be preferable to a “wild west” scenario where anything goes. New entrants, however, might thrive in less regulated environments where they can innovate and disrupt without many constraints.

How much of this applies to AI?

Image source: https://pixabay.com/photos/cowboy-rodeo-western-horse-animal-4995302/

During its inaugural Developer Day, AI startup OpenAI released a series of open-source models. The slew of products included an upgraded version of its open-source automatic speech recognition model, Whisper large-v3. The company’s future plans involve making the model’s API accessible to users.

The models for English-only applications tend to perform better, especially for the `tiny.en` and `base.en` models as per the official page. The model’s performance varies widely depending on the language.

Initially focused on English, the neural net model was released in September last year. Then it got an upgraded version 2 in December which was enhanced to support multiple languages, although specific languages were not explicitly mentioned.

Accessible on GitHub under a permissive license, Whisper large-v3 effortlessly transcribes various content for users and has been called the best transcription tool out there. The model features a unique timestamp section that facilitates its application as subtitles on platforms such as YouTube.

The tool initiates the process by segmenting audio into 30-second clips, converting them, and subsequently passing them through an encoder and decoder, which predict the corresponding text caption. Technical intricacies also involve language identification, facilitating multilingual speech transcription, and translation to English.

The model was initially expected to be integrated with ChatGPT, to let the users converse directly with the chatbot through speech. But OpenAI then decided to release the model to the public directly. Interestingly, Whisper is not aimed at the end users as of now but rather at researchers.

The reason for open-sourcing as per OpenAI was to “serve as a foundation for building useful applications and for further research on robust speech processing“. OpenAI’s AI tool was honed using an extensive dataset of 680,000 hours of meticulously supervised data sourced from the internet, with one third portion originating from non-English sources.

The post OpenAI Silently Unveils Whisper 3, A New Generation Open Source ASR Model appeared first on Analytics India Magazine.

Image generated by DALL-E

In the digital age, the marvels of artificial intelligence have transformed the way we interact, work, and even think.

From voice assistants that curate our playlists to predictive algorithms that forecast market trends, AI has seamlessly integrated into our daily lives.

But as with any technological advancement, it’s not without its twists.

A large language model or LLM is a trained machine learning model that generates text based on the prompt you provided. In order to generate good responses, the models take advantage of all the knowledge retained during its training phase.

Recently, LLMs have shown impressive and increasing capabilities, including generating convincing responses to any type of user prompts.

However, even though LLMs have an incredible ability to generate text, it is hard to tell if this generation is accurate or not.

And this is precisely what is commonly known as hallucinations.

But what are these hallucinations, and how do they impact the reliability and utility of AI?

The Enigma of LLM Hallucinations

LLMs are masterminds when it comes to text generation, translations, creative content, and more.

Despite being potent tools, LLM do present some significant shortcomings:

1. The decoding techniques employed can yield outputs that are either uninspiring, lacking coherence, or prone to falling into monotonous repetitions.
2. Their knowledge foundation is “static” in nature, presenting challenges in seamless updates.
3. A common issue is the generation of text that is either nonsensical or inaccurate.

The last point is referred to as hallucination, which is an AI-extended concept from humans.

For humans, hallucinations represent experiences perceived as real despite being imaginary. This concept extends to AI models, where the hallucinated text appears accurate even though it's false.

In the context of LLMs, “hallucination” refers to a phenomenon where the model generates text that is incorrect, nonsensical, or not real.

Image by Dall-E

LLMs are not designed like databases or search engines, so they don’t reference specific sources or knowledge in their answers.

I bet most of you might be wondering… How can it be possible?

Well… these models produce text by building upon the given prompt. The generated response isn’t always directly backed by specific training data, but is crafted to align with the context of the prompt.

In simpler terms:

They can confidently spew out information that’s factually incorrect or simply doesn’t make sense.

Deciphering the Types of Hallucinations

Identifying hallucinations in humans has always posed a significant challenge. This task becomes even more complex given our limited ability to access a reliable baseline for comparison.

While detailed insights like output probability distributions from Large Language Models can aid in this process, such data is not always available, adding another layer of complexity.

The issue of hallucination detection remains unsolved and is a subject of ongoing research.

1. The Blatant Untruths: LLMs might conjure up events or figures that never existed.
2. The Overly Accurate: They might overshare, potentially leading to the spread of sensitive information.
3. The Nonsensical: Sometimes, the output might just be pure gibberish.

   Why Do These Hallucinations Occur?

Why Do These Hallucinations Occur?

The root cause lies in the training data. LLMs learn from vast datasets, which can sometimes be incomplete, outdated, or even contradictory. This ambiguity can lead them astray, making them associate certain words or phrases with inaccurate concepts.

Moreover, the sheer volume of data means that LLMs might not have a clear “source of truth” to verify the information they generate.

Using Hallucinations to Your Advantage

Interestingly, these hallucinations can be a boon in disguise. If you’re seeking creativity, you’d want LLMs like ChatGPT to hallucinate.

Image generated by DALL-E

Imagine asking for a unique fantasy story plot, you’d want a fresh narrative, not a replica of an existing one.

Similarly, when brainstorming, hallucinations can offer a plethora of diverse ideas.

Mitigating the Mirage

Awareness is the first step towards addressing these hallucinations. Here are some strategies to keep them in check:

• Consistency Checks: Generate multiple responses to the same prompt and compare.
• Semantic Similarity Checks: Use tools like BERTScore to measure the semantic similarity between generated texts.
• Training on Updated Data: Regularly update the training data to ensure relevancy. You can even fine-tune the GPT model to improve its performance in some specific fields.
• User Awareness: Educate users about potential hallucinations and the importance of cross-referencing information.

And the final one, but not least… EXPLORE!

This article has laid the groundwork regarding LLM hallucinations, yet the implications for you and your application might diverge considerably.

Moreover, your interpretation of these phenomena may not precisely correspond with actuality. The key to fully grasping and valuing the impact of LLM hallucinations on your endeavors is through an in-depth exploration of LLMs.

In Conclusion

The journey of AI, especially LLMs, is akin to sailing in uncharted waters. While the vast ocean of possibilities is exciting, it’s essential to be wary of the mirages that might lead us astray.

By understanding the nature of these hallucinations and implementing strategies to mitigate them, we can continue to harness the transformative power of AI, ensuring its accuracy and reliability in our ever-evolving digital landscape.

Josep Ferrer is an analytics engineer from Barcelona. He graduated in physics engineering and is currently working in the Data Science field applied to human mobility. He is a part-time content creator focused on data science and technology. You can contact him on LinkedIn, Twitter or Medium.

More On This Topic

• What’s Your Passion? Make It a Reality During Our Challenge
• Data Science Career: 7 Expectations vs Reality
• Data Science: Reality vs Expectations
• The Difference Between Data Scientists and ML Engineers
• 7 Differences Between a Data Analyst and a Data Scientist
• What Is The Real Difference Between Data Engineers and Data Scientists?

A podcast with Fabrizio del Maffeo of Axelera AI

As CEO and Co-founder of Axelera AI, Fabrizio del Maffeo is focused on harnessing the potential of the fifth generation of RISC (reduced instruction set computing) platforms in edge AI applications. Axelera announced early availability of its in-memory Metis AI platform in September 2023.

Edge AI is the use of artificial intelligence in an edge computing environment. Edge computing for its part moves the responsibility for processing critical workloads closer to where the remote demand for information is, such as in a mobile transportation, warehouse, or factory environment. Proximity to the end user means more efficient use of energy and network bandwidth.

Users of the 50 billion + devices that are currently internet connected are underserved, del Maffeo points out. Much of edge computing’s promise is still unrealized, with data taking far too many round trips back to central data centers. The result is massive inefficiency and a poor user experience due to latency that could be substantially reduced with the help of better design.

The issue isn’t just putting more compute and storage closer to the demand, del Maffeo says. GPUs crafted for gaming purposes aren’t designed for AI, much less AI at the edge. The Nvidia H100 GPUs in demand by major public cloud providers for their data centers draw 700 watts per unit, with each unit weighing as much as three kilograms. The challenge of edge AI is handling high volumes of table multiplications with limited power and network bandwidth.

To tackle this challenge, Axelera took advantage of memory potential in the RISC-V architecture to merge memory and processing functions, as well as parallelize processing at the edge in new ways. The ultimate goal of platforms such as Axelera’s Metis AI is ambitious: large language model (LLM) processing at the edge.

Del Maffeo in this podcast explores a wide range of edge use cases, from smart cities, to automotive, to retail. Hope you find the discussion as illuminating as I have.