Cycode acquires Bearer to accelerate its move into AI-enhanced security remediation

Cycode acquires Bearer to accelerate its move into AI-enhanced security remediation Frederic Lardinois @fredericl / 16 hours

Cycode is a well-funded startup that offers an end-to-end application security posture management platform — that is, a tool that continuously scans code (and the libraries it relies on) for potential security vulnerabilities throughout the software development life cycle and then helps remediate those issues. Today, the company announced that it has acquired Bearer, a static application security testing (SAST) startup that focuses on making the developer experience as smooth as possible while still providing them with essential security feedback. With its recently launched AI assistant, Bearer also bet on generative AI to suggest code fixes and explain vulnerabilities.

Bearer raised an $8 million seed round led by Alven in 2022. Cycode has raised over $81 million in funding so far, including a $56 million Series B round in 2021. That’s what put the company in a position to make today’s acquisition, though the two companies did not disclose the price.

As Cycode co-founder and CEO Lior Levy told me, this acquisition now provides the company with all of the capabilities it needs to become a full-fledged application security platform. Like so many startups, that’s not necessarily what the team was focused on when Cycode first launched. At the time, Cycode was one of the earlier players in the software supply chain space. Today, the company addresses a far wider range of attack vectors.

“Day one, we were focused on what we wanted to build around software supply chain security,” he told me. “But then, as time progressed, we realized that there was additional value that we needed to capture. It’s the right approach to become a platform and this is what we’ve heard from customers. So we adopted and addressed those needs.”

The Cycode and Bearer teams first started talking last August and stayed in touch after that. According to Levy, it was Dor Atias, Cycode’s co-founder and VP of R&D, who realized that Bearer’s technology would nicely complement — and complete — the larger startup’s existing solution. “Our mission is to be a complete platform,” Atias said. “The missing part was a SAST tool that can be fast and connected easily to the Cycode platform. And the Bearer team invested a lot in the brain of the SAST engine — not only the rules and stuff like that — but the engine itself. I tested it a few times and saw that it could be integrated easily.”

And that’s what the Cycode team has already done, even though the deal only closed last week. Now, Atias said, the team is looking at how it can bring that Bearer engine to other parts of the platform, too. That includes some of Bearer’s AI solutions as well, including its remediation capabilities. Levy believes that using AI to fix issues before they even go into the source control is something akin to the “secret sauce for security” and will help reduce the burden on developers and security teams.

Cycode co-founder Ronen Slavin, the company’s CTO, also noted that this acquisition fits in well with the company’s focus on both putting security and the developer experience first. “Developers were not hired to fix security issues,” he said. “They get frustrated and they end up with false positives. Bearer has the highest rate of precision in terms of the rate of false positives. In terms of GenAI, one of the components is the context for the remediation that developers get that they wouldn’t get otherwise, which is also tied to the improved experience.”

While Bearer still exists as a stand-alone product for now, Cycode plans to move its customers over to its platform over time.

“We are thrilled to be joining forces with Cycode, a company that shares our vision for making developer security a team sport,” said Bearer CEO Guillaume Montard. “This union marks a critical milestone in our journey, amplifying our reach and impact to the world’s best security and development teams. Together, we’re set to continue redefining the standards of the complete approach to application security posture management.”

Cycode raises $20M to secure DevOps pipelines

Cycode raises $56M Series B to help secure software supply chains

DSC Weekly 5 March 2024

Announcements

  • Data loss is an inescapable reality in the digital business world. Data backup techniques are conceptually simple, but implementing a robust disaster recovery plan that both protects data from cyberattacks and maintains business continuity during a breach can be difficult. Register for the free Advancing Data Backup Tools and Techniques summit to gain insights from leading experts on the future of data backup solutions to help design a disaster recovery plan that protects your most critical data and mitigates the effects of attacks when they do occur.
  • The cloud ecosystem is larger, more complex and interdependent than ever. As organizations increasingly migrate their data and assets to the cloud, attacks that attempt to break through its growing attack surface are getting more sophisticated. Cloud security strategies are necessary for proper user and device authentication, resource access control and data privacy to ensure that the cloud system remains impenetrable to cyberattacks. Attend the Securing the Cloud Ecosystem summit to hear leading experts discuss effective strategies to secure identity and access management as well as the leading tools and approaches that best secure the extensive cloud ecosystem from insidious attacks.

Top Stories

  • Scaling understanding with the help of feedback loops, knowledge graphs and NLP
    March 4, 2024
    by Alan Morrison
    Assisted annotation and other automation methods that can be used for knowledge graph creation and natural language understanding should be more flexible and iterative than they often are. Much depends on the architecture and tooling choices your organization makes.
  • New Book: Practical GenAI and Machine Learning – Projects and Datasets
    February 29, 2024
    by Vincent Granville
    This book is for participants in my AI and machine learning certification program. However, it is now free and available to everyone. With tutorials, enterprise-grade projects and solutions, it covers state-of-the-art material on topics such as generative adversarial networks (GAN), specialized LLM, data synthetization, as well as classical machine learning.
  • GenAI: The User Interface to Artificial Intelligence (AI)?
    March 3, 2024
    by Bill Schmarzo
    Generative AI (GenAI) Chatbots like Microsoft Copilot (formerly Bing AI), Google’s Gemini (formerly Google Bard), and OpenAI ChatGPT (still OpenAI ChatGPT) are driving extraordinary productivity improvements by assisting knowledge workers in providing highly relevant information, answering questions, and engaging in wide-ranging exploratory conversations.
Education_DSC_160x600-2

In-Depth

  • Future of LLM application development – impact of Gemini 1.5 Pro with a 1M context window
    March 5, 2024
    by ajitjaokar
    Current LLM applications are mostly based on langchain or llamaindex. langChain and LlamaIndex are frameworks designed for LLM development. They each cater to different use cases with unique features. LangChain is a framework ideal for creating data-aware and agent-based applications.
  • Open source LLMs – no more than a marketing term?
    March 5, 2024
    by ajitjaokar
    This week, the tech community has been abuzz with the announcement of the latest model from Mistral being closed source. This revelation confirms a suspicion held by many: the concept of open-source Large Language Models (LLMs) today is more a marketing term than a substantive promise.
  • Data Cleaning in Python vs. Data Quality Tools
    March 5, 2024
    by Ovais Naseem
    In the era of big data, the importance of clean and high-quality data cannot be overstated. Data cleaning is a crucial step in the data preprocessing pipeline, aimed at identifying and rectifying errors, inconsistencies, and inaccuracies in datasets. Traditionally, data cleaning has been performed using Python and other programming languages.
  • Leveraging machine learning for predictive analysis in customer satisfaction surveys
    March 1, 2024
    by John Lee
    The dynamic landscape of customer-centric businesses requires understanding and improving customer satisfaction. Traditional survey analysis rarely yields real-time actionable insights. However, machine learning (ML) predictive analysis allows organizations to use advanced algorithms to transform customer satisfaction surveys. ML predictive analysis is changing how businesses measure and improve customer satisfaction.
  • Mobile drivers’ licenses: A humbler take on self-sovereign identity and personal data protection
    March 1, 2024
    by Alan Morrison
    I’ve been interested in self-sovereign identity for a number of years now, ever since I interviewed Phil Windley, a founder of the Internet Identity Workshop (IIW) and then chair of the Sovrin Foundation, in 2018. In a self-sovereign identity (SSI) scenario, the users themselves control the sensitive information previously stored by a third party.
  • Optimizing logistics operations with technology implementation
    February 29, 2024
    by Manoj Kumar
    In today’s fast-paced business terrain, optimizing logistics operations has become more pivotal than ever. With increasing competition and rising client prospects, companies are under pressure to deliver products directly and quickly. The capability to effectively manage and streamline the movement of goods and products has a direct impact on client satisfaction, cost-effectiveness, and overall functional success.
  • DSC Weekly 27 February 2024
    February 27, 2024
    by Scott Thompson
    Read more of the top articles from the Data Science Central community.

Numbers Station lets business users chat with their data

Numbers Station lets business users chat with their data Frederic Lardinois @fredericl / 10 hours

Numbers Station, a startup that is using large language models (LLMs) to power its data analytics platform, is launching its first cloud-based product today: the aptly named Numbers Station Cloud, which is now in early access. With this service, virtually any user in an enterprise can analyze their internal data using Numbers Station’s chat interface.

Several similar tools focus on translating natural language queries into a database language like SQL. However, the Numbers Station team argues that this approach has limitations, in part because the generic LLM doesn’t have an understanding of how a given company operates, how it structures its data and how people inside the company refer to specific data objects.

Image Credits: Numbers Station

As Numbers Station co-founder and CEO Chris Aberger told me, he’s somewhat tired of talking about how the service allows users to “chat with their data,” because there is so much noise around that. “But the higher level thing of business executives, non-technical users, having questions they want to ask, and then getting answers over these classic structured data sources, is really where things have led,” he told me. “There’s a lot of data modeling, data plumbing around these foundation models and large language models to make them work.”

For Numbers Station, this has meant spending a lot of engineering resources on building what the company calls its semantic catalog. That catalog is essentially an automatically curated source of a company’s metrics and definitions. That catalog is specific to every company (and not shared between them). Aberger described the catalog as “a beastly thing” that, for example, ensures that the model’s definition of ‘recurring revenue’ is aligned with the company’s usage of that term.

While Numbers Station’s platform sits on top of a set of very specialized LLMs and machine learning models, it’s this catalog that holds everything together. As Numbers Station co-founder and chief scientist Ines Chami told me, the team had originally underestimated the challenge of building out that part of the platform.

“It goes back to classical [machine learning] and classical data engineering: how do I create a representation of knowledge that the model can actually use to answer those questions,” she told me. “Because there’s no way a model is going to understand all those metrics, all those things that business users ask about.” Even humans don’t understand every question immediately, after all, and the model has to turn those vague questions into very concrete queries. Numbers Stations’ research shows that its approach results in significantly improved precision compared to more traditional text-to-SQL pipelines.

Image Credits: Numbers Station

While the company is launching this chat service today, the overall vision is significantly larger.

“What we’re doing, fundamentally, is building an AI platform for analytics,” Aberger said. “This is one of the applications […]. There’s a bigger, broader play that we’re still working on as a company, which is going after a bunch of different data problems that sit on top of here, examples of which are: how do I enrich my data with third-party data sources? How do I do some of these more classic algorithms like fuzzy matching, etc? There’s almost an infinite number of spokes that you can build off of on this platform.”

The company has already signed up several Fortune 500 customers, including the likes of global real estate services firm Jones Lang LaSalle. “Numbers Station is at the cutting edge of enterprise AI for structured data,” said Sharad Rastogi, the CEO of Work Dynamics Technology from Jones Lang LaSalle. “We are impressed by Numbers Station’s trusted and engaging platform. It continuously learns as we use it, enabling our data teams discover and verify hypotheses for driving impactful business outcomes.”

Overcoming LLM Hallucinations Using Retrieval Augmented Generation (RAG)

Featured image

Large Language Models (LLMs) are revolutionizing how we process and generate language, but they're imperfect. Just like humans might see shapes in clouds or faces on the moon, LLMs can also ‘hallucinate,' creating information that isn’t accurate. This phenomenon, known as LLM hallucinations, poses a growing concern as the use of LLMs expands.

Mistakes can confuse users and, in some cases, even lead to legal troubles for companies. For instance, in 2023, an Air Force veteran Jeffery Battle (known as The Aerospace Professor) filed a lawsuit against Microsoft when he found that Microsoft’s ChatGPT-powered Bing search sometimes gives factually inaccurate and damaging information on his name search. The search engine confuses him with a convicted felon Jeffery Leon Battle.

To tackle hallucinations, Retrieval-Augmented Generation (RAG) has emerged as a promising solution. It incorporates knowledge from external databases to enhance the outcome accuracy and credibility of the LLMs. Let’s take a closer look at how RAG makes LLMs more accurate and reliable. We'll also discuss if RAG can effectively counteract the LLM hallucination issue.

Understanding LLM Hallucinations: Causes and Examples

LLMs, including renowned models like ChatGPT, ChatGLM, and Claude, are trained on extensive textual datasets but are not immune to producing factually incorrect outputs, a phenomenon called ‘hallucinations.' Hallucinations occur because LLMs are trained to create meaningful responses based on underlying language rules, regardless of their factual accuracy.

A Tidio study found that while 72% of users believe LLMs are reliable, 75% have received incorrect information from AI at least once. Even the most promising LLM models like GPT-3.5 and GPT-4 can sometimes produce inaccurate or nonsensical content.

Here's a brief overview of common types of LLM hallucinations:

Common AI Hallucination Types:

  1. Source Conflation: This occurs when a model merges details from various sources, leading to contradictions or even fabricated sources.
  2. Factual Errors: LLMs may generate content with inaccurate factual basis, especially given the internet's inherent inaccuracies
  3. Nonsensical Information: LLMs predict the next word based on probability. It can result in grammatically correct but meaningless text, misleading users about the content's authority.

Last year, two lawyers faced possible sanctions for referencing six nonexistent cases in their legal documents, misled by ChatGPT-generated information. This example highlights the importance of approaching LLM-generated content with a critical eye, underscoring the need for verification to ensure reliability. While its creative capacity benefits applications like storytelling, it poses challenges for tasks requiring strict adherence to facts, such as conducting academic research, writing medical and financial analysis reports, and providing legal advice.

Exploring the Solution for LLM Hallucinations: How Retrieval Augmented Generation (RAG) Works

In 2020, LLM researchers introduced a technique called Retrieval Augmented Generation (RAG) to mitigate LLM hallucinations by integrating an external data source. Unlike traditional LLMs that rely solely on their pre-trained knowledge, RAG-based LLM models generate factually accurate responses by dynamically retrieving relevant information from an external database before answering questions or generating text.

RAG Process Breakdown:

Steps of RAG

Steps of RAG Process: Source

Step 1: Retrieval

The system searches a specific knowledge base for information related to the user's query. For instance, if someone asks about the last soccer World Cup winner, it looks for the most relevant soccer information.

Step 2: Augmentation

The original query is then enhanced with the information found. Using the soccer example, the query “Who won the soccer world cup?” is updated with specific details like “Argentina won the soccer world cup.”

Step 3: Generation

With the enriched query, the LLM generates a detailed and accurate response. In our case, it would craft a response based on the augmented information about Argentina winning the World Cup.

This method helps reduce inaccuracies and ensures the LLM's responses are more reliable and grounded in accurate data.

Pros and Cons of RAG in Reducing Hallucinations

RAG has shown promise in reducing hallucinations by fixing the generation process. This mechanism allows RAG models to provide more accurate, up-to-date, and contextually relevant information.

Certainly, discussing Retrieval Augmented Generation (RAG) in a more general sense allows for a broader understanding of its advantages and limitations across various implementations.

Advantages of RAG:

  • Better Information Search: RAG quickly finds accurate information from big data sources.
  • Improved Content: It creates clear, well-matched content for what users need.
  • Flexible Use: Users can adjust RAG to fit their specific requirements, like using their proprietary data sources, boosting effectiveness.

Challenges of RAG:

  • Needs Specific Data: Accurately understanding query context to provide relevant and precise information can be difficult.
  • Scalability: Expanding the model to handle large datasets and queries while maintaining performance is difficult.
  • Continuous Update: Automatically updating the knowledge dataset with the latest information is resource-intensive.

Exploring Alternatives to RAG

Besides RAG, here are a few other promising methods enable LLM researchers to reduce hallucinations:

  • G-EVAL: Cross-verifies generated content's accuracy with a trusted dataset, enhancing reliability.
  • SelfCheckGPT: Automatically checks and fixes its own errors to keep outputs accurate and consistent.
  • Prompt Engineering: Helps users design precise input prompts to guide models towards accurate, relevant responses.
  • Fine-tuning: Adjusts the model to task-specific datasets for improved domain-specific performance.
  • LoRA (Low-Rank Adaptation): This method modifies a small part of the model's parameters for task-specific adaptation, enhancing efficiency.

The exploration of RAG and its alternatives highlights the dynamic and multifaceted approach to improving LLM accuracy and reliability. As we advance, continuous innovation in technologies like RAG is essential for addressing the inherent challenges of LLM hallucinations.

To stay updated with the latest developments in AI and machine learning, including in-depth analyses and news, visit unite.ai.

Best Free Resources to Learn Data Analysis and Data Science

Sponsored Content

Best Free Resources to Learn Data Analysis and Data Science

In my decade of teaching online, the most significant inspiration has been that online learning democratizes access to education globally. Regardless of your ethnic background, income level, and geographical location—as long as you can surf the web—you can find an ocean of free educational content to help you learn new skills.

This article introduces six top-notch, free data science resources ideal for aspiring data analysts, data scientists, or anyone aiming to enhance their analytical skills.

1. 365 Data Science Flashcards

At 365 Data Science, we offer a range of expertly designed flashcard decks to teach fundamental data science concepts—including terms and glossaries for tools like Microsoft Excel, SQL, Python, and ChatGPT. Additionally, our flashcards cover math, statistics, probability, and machine learning—providing an excellent starting point for beginners by familiarizing them with the essential data science language. Free Udemy Courses

2. Udemy

Udemy is the go-to marketplace for online courses. Their content library includes over 100,000 titles on almost every topic—including data analytics and data science. They also offer free courses uploaded by authors eager to share their knowledge with the public at no cost. Use the filtering options when browsing the marketplace to discover unique learning materials.

3. 365 Data Science Statistics Calculators

Recently, 365 Data Science launched a series of statistics calculators designed for university students and practitioners eager to grasp statistical calculations' underlying mechanics and theory—beyond merely achieving results through tools like Excel or Python. These calculators enable users to input problem data for homework, exams, or practice, revealing each step necessary to arrive at the solution rather than only the outcome. Each calculator includes a detailed statistical article to explain the concept—offering an invaluable opportunity to learn through comprehension and application.

Utilize these complimentary statistics calculators to discover how to:

  • Study the measures of central tendency
  • Obtain dispersion measures such as variance, standard deviation, and coefficient of variation
  • Get an idea about the shape of a statistical distribution: skewness and kurtosis
  • Calculate confidence intervals
  • Perform hypothesis testing
  • Obtain p-value and decide on the significance of A/B testing results
  • Calculate the difference in means
  • Run simple linear regression

4. YouTube Tutorials

YouTube's data content creators deliver immense value—offering everything from concise tutorials and job-seeking advice in data science to comprehensive courses. If you're seeking exceptional, free data science learning resources, consider these top recommendations:

  • Programming with Mosh's full course on Python Programming for Beginners
  • freeCodeCamp’s full course on Data Science Fundamentals
  • MySQL Basics for Data Analysts playlist by Alex the Analyst
  • SQL Sundays playlist by Tina Huang
  • Data Science projects playlist by Ken Lee

5. 365 Data Science Course Notes and Career Guides

At 365 Data Science, we offer a wealth of free resources to our students. For those keen on exploring data science without financial constraints, download our complimentary course notes and career guides. You can access various topics at no cost, including Intro to Data Science, Statistics, Probability, Python, Machine Learning, Data Strategy, and others.

In addition to course notes, you can download 365’s free Data Analyst Career Guide and Data Scientist Career Guide.

6. Free Big Tech Courses

Google, Microsoft, Amazon, and other Big Tech organizations have shown increasing interest in providing free online courses to individuals worldwide.

For instance, you can sign up at no cost for Google's Data Analytics Professional Certification on Coursera, accessing courses for free until opting to pay a reasonable fee for a certificate of completion.

Conclusion

This comprehensive list of free data science learning resources is a testament to the quality content available online at no cost. The wealth of available educational resources will motivate future data analysts and scientists—enhancing global skillsets and career prospects, regardless of humble origins.

More On This Topic

  • The Best Data Science Resources, Bootcamp, and Courses to Learn…
  • Best Resources to Learn Natural Language Processing in 2021
  • Top 3 Free Resources to Learn Linear Algebra for Machine Learning
  • Top Free Resources To Learn ChatGPT
  • The Best Learning Resources for Data Science in 2022
  • 6 Best Free Online Courses to Learn Python and Boost Your Career

Future of LLM application development – impact of Gemini 1.5 Pro with a 1M context window, 

Gemini-Google

Image source https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/

Current LLM applications are mostly based on langchain or llamaindex. langChain and LlamaIndex are frameworks designed for LLM development. They each cater to different use cases with unique features.

LangChain is a framework ideal for creating data-aware and agent-based applications. It offers high-level APIs for easy integration with various large language model (LLM) providers, supporting a broad range of capabilities and tool integration.
LlamaIndex focuses on indexing and retrieval of data, making it highly suitable for applications that require smart search and deep exploration of data. It features a lightweight interface for data loading and transfer and offers a list index feature for composing an index from other indexes. This functionality is useful for searching and summarizing data from heterogeneous sources, making LlamaIndex as a good choice for projects centered around data retrieval and search capabilities.
The choice between LangChain and LlamaIndex depends on the specific needs of your project. If you require a broader, more versatile framework for developing complex language model applications with multiple tool integrations, LangChain might be the right choice. Conversely, if your application’s core functionality revolves around efficient data search and retrieval, LlamaIndex could offer more targeted benefits.
A typical use case for LangChain involves building intelligent agents capable of integrating with various language models and external data sources. For example, creating an application that uses natural language to interact with databases, such as a chatbot that can query a SQL database in plain English and provide users with answers based on the database’s data.

A typical use case for llamaindex might be a custom knowledge management system where private or domain-specific documents are ingested and indexed, allowing users to perform natural language searches to find precise information within those documents. LlamaIndex, with its focus on efficient indexing and retrieval, is ideal for applications that need smart search capabilities across large volumes of data.

Now, the question is: How does this LLM application development status quo (ex RAG) change with the Gemini 1.5 pro LLM with a 1M token context window?

Google recently released Gemini 1.5 Pro with a 1M context window, The ability to process 1 million tokens in one go implies the capability to process vast amounts of information — including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code or over 700,000 words.

It also means the LLM can reason over such vast amounts of information. For example, when given the 402-page transcripts from Apollo 11’s mission to the moon, it can reason about conversations, events and details found across the document.

Also, 1.5 Pro can perform highly-sophisticated understanding and reasoning tasks for different modalities, including video. The same abilities can also extend to problem-solving with longer blocks of code. In this scenario what happens to RAG?

llamaindex creator Jerry Liu proposes his vision having worked with Gemeni 1.5 Pro. He believes that as tokens get cheaper, we will see a new wave of large context LLMs in future. While long-context LLMs will simplify certain parts of the RAG pipeline (e.g. chunking), new RAG architectures will need to be evolved to cater for the new use cases arising from long context LLMs. These could include QA over semi-structured data, over complex documents, and agentic reasoning in a multi-doc setting.
References:
https://www.llamaindex.ai/blog/towards-long-context-rag
https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/

Extractive Summarization with LLM using BERT

Extractive Summarization with LLM using BERT
Image by Editor
Introduction

In today's fast-paced world, we're bombarded with more information than we can handle. We’re increasingly getting used to receiving more information in less time, leading to frustration when having to read extensive documents or books. That's where extractive summarization steps in. To the heart of a text, the process pulls out key sentences from an article, piece, or page to give us a snapshot of its most important points.

For anyone needing to understand big documents without reading every word, this is a game changer.

In this article, we delve into the fundamentals and applications of extractive summarization. We'll examine the role of Large Language Models, especially BERT (Bidirectional Encoder Representations from Transformers), in enhancing the process. The article will also include a hands-on tutorial on using BERT for extractive summarization, showcasing its practicality in condensing large text volumes into informative summaries.

Understanding Extractive Summarization

Extractive summarization is a prominent technique in the field of natural language processing (NLP) and text analysis. With it, key sentences or phrases are carefully selected from the original text and combined to create a concise and informative summary. This involves meticulously sifting through text to identify the most crucial elements and central ideas or arguments presented in the selected piece.

Where abstractive summarization involves generating entirely new sentences often not present in the source material, extractive summarization sticks to the original text. It doesn’t alter or paraphrase but instead extracts the sentences exactly as they appear, maintaining the original wording and structure. This way the summary stays true to the source material's tone and content. The technique of extractive summarization is extremely beneficial in instances where the accuracy of the information and the preservation of the author's original intent are a priority.

It has many different uses, like summarizing news articles, academic papers, or lengthy reports. The process effectively conveys the original content's message without potential biases or reinterpretations that can occur with paraphrasing.

How Does Extractive Summarization Use LLMs?

1. Text Parsing

This initial step involves breaking down the text into its essential elements, primarily sentences, and phrases. The goal is to identify the basic units (sentences, in this context) the algorithm will later evaluate to include in a summary, like dissecting a text to understand its structure and individual components.

For instance, the model would analyze a four-sentence paragraph by breaking it down into the following four-sentence components.

  1. The Pyramids of Giza, built in ancient Egypt, stood magnificently for millennia.
  2. They were constructed as tombs for pharaohs.
  3. The Great Pyramids are the most famous.
  4. These structures symbolize architectural intelligence.

2. Feature Extraction

In this stage, the algorithm analyzes each sentence to identify characteristics or 'features' that might indicate what their significance is to the overall text. Common features include the frequency and repeated use of keywords and phrases, the length of sentences, their position in the text and its implications, and the presence of specific keywords or phrases that are central to the text's main topic.

Below is an example of how the LLM would do feature extraction for the first sentence “The Pyramids of Giza, built in ancient Egypt, stand magnificently for millennia.”

Attributes Text
Frequency “Pyramids of Giza”, “Ancient Egypt”, “Millennia”
Sentence Length Moderate
Position in Text Introductory, sets the topic
Specific Keywords “Pyramids of Giza”, “ancient Egypt”

3. Scoring Sentences

Each sentence is assigned a score based on its content. This score reflects a sentence’s perceived importance in the context of the entire text. Sentences that score higher are deemed to carry more weight or relevance.

Simply put, this process rates each sentence for its potential significance to a summary of the entire text.

Sentence Score Sentence Explanation
9 The Pyramids of Giza, built in ancient Egypt, stood magnificently for millennia. It's the introductory sentence that sets the topic and context, containing key terms like “Pyramids of Giza” and “Ancient Egypt.
8 They were constructed as tombs for pharaohs. This sentence provides critical historical information about the Pyramids and may identify them as significant for understanding their purpose and significance.
3 The Great Pyramids are the most famous. Although this sentence adds specific information about the Great Pyramids, it is considered less crucial in the broader context of summarizing the overall significance of the Pyramids.
7 These structures symbolize architectural intelligence. This summarizing statement captures the overarching significance of the Pyramids.

4. Selection and Aggregation

The final phase involves selecting the highest-scoring sentences and compiling them into a summary. When carefully done, this ensures the summary remains coherent and an aggregately representative of the main ideas and themes of the original text.

To create an effective summary, the algorithm must balance the need to include important sentences that are concise, avoid redundancy, and ensure that the selected sentences provide a clear and comprehensive overview of the entire original text.

  • The Pyramids of Giza, built in Ancient Egypt, stood magnificently for millennia. They were constructed as tombs for pharaohs. These structures symbolize architectural brilliance.

This example is extremely basic, extracting 3 of the total 4 sentences for the best overall summarization. Reading an extra sentence doesn't hurt, but what happens when the text is longer? Let's say, 3 paragraphs?

How to Run Extractive Summarization with BERT LLMs

Step 1: Installing and Importing Necessary Packages

We will be leveraging the pre-trained BERT model. However, we won't be using just any BERT model; instead, we'll focus on the BERT Extractive Summarizer. This particular model has been finely tuned for specialized tasks in extractive summarization.

!pip install bert-extractive-summarizer  from summarizer import Summarizer

Step 2: Introduce Summarizer Function

The Summarizer() function imported from the summarizer in Python is an extractive text summarization tool. It uses the BERT model to analyze and extract key sentences from a larger text. This function aims to retain the most important information, providing a condensed version of the original content. It's commonly used to summarize lengthy documents efficiently.

model = Summarizer()

Step 3: Importing our Text

Here, we will import any piece of text that we would like to test our model on. To test our extractive summary model we generated text using ChatGPT 3.5 with the prompt: “Provide a 3-paragraph summary of the history of GPUs and how they are used today.”

text = "The history of Graphics Processing Units (GPUs) dates back to the early 1980s when companies like IBM and Texas Instruments developed specialized graphics accelerators for rendering images and improving overall graphical performance. However, it was not until the late 1990s and early 2000s that GPUs gained prominence with the advent of 3D gaming and multimedia applications. NVIDIA's GeForce 256, released in 1999, is often considered the first GPU, as it integrated both 2D and 3D acceleration on a single chip. ATI (later acquired by AMD) also played a significant role in the development of GPUs during this period. The parallel architecture of GPUs, with thousands of cores, allows them to handle multiple computations simultaneously, making them well-suited for tasks that require massive parallelism. Today, GPUs have evolved far beyond their original graphics-centric purpose, now widely used for parallel processing tasks in various fields, such as scientific simulations, artificial intelligence, and machine learning.  Industries like finance, healthcare, and automotive engineering leverage GPUs for complex data analysis, medical imaging, and autonomous vehicle development, showcasing their versatility beyond traditional graphical applications. With advancements in technology, modern GPUs continue to push the boundaries of computational power, enabling breakthroughs in diverse fields through parallel computing. GPUs also remain integral to the gaming industry, providing immersive and realistic graphics for video games where high-performance GPUs enhance visual experiences and support demanding game graphics. As technology progresses, GPUs are expected to play an even more critical role in shaping the future of computing."  

Step 4: Performing Extractive Summarization

Finally, we'll execute our summarization function. This function requires two inputs: the text to be summarized and the desired number of sentences for the summary. After processing, it will generate an extractive summary, which we will then display.

# Specifying the number of sentences in the summary  summary = model(text, num_sentences=4)   print(summary)

Extractive Summary Output:

The history of Graphics Processing Units (GPUs) dates back to the early 1980s when companies like IBM and Texas Instruments developed specialized graphics accelerators for rendering images and improving overall graphical performance. NVIDIA's GeForce 256, released in 1999, is often considered the first GPU, as it integrated both 2D and 3D acceleration on a single chip. Today, GPUs have evolved far beyond their original graphics-centric purpose, now widely used for parallel processing tasks in various fields, such as scientific simulations, artificial intelligence, and machine learning. As technology progresses, GPUs are expected to play an even more critical role in shaping the future of computing.

Our model pulled the 4 most important sentences from our large corpus of text to generate this summary!

Challenges of Extractive Summarization Using LLMs

  1. Contextual Understanding Limitations
    1. While LLMs are proficient in processing and generating language, their understanding of context, especially in longer texts, is limited. LLMs can miss subtle nuances or fail to recognize critical aspects of the text leading to less accurate or relevant summaries. The more advanced the language model the better the summary will be.
  2. Bias in Training Data
    1. LLMs learn from vast datasets compiled from various sources, including the internet. These datasets can contain biases, which the models might inadvertently learn and replicate in their summaries leading to skewed or unfair representations.
  3. Handling Specialized or Technical Language
    1. While LLMs are generally trained on a wide range of general texts, they may not accurately capture specialized or technical language in fields like law, medicine, or other highly technical fields. This can be alleviated by feeding it more specialized and technical text. Lack of training in specialized jargon can affect the quality of summaries when used in these fields.

Conclusion

It's clear that extractive summarization is more than just a handy tool; it's a growing necessity in our information-saturated age where we’re inundated with walls of text every day. By harnessing the power of technologies like BERT, we can see how complex texts can be distilled into digestible summaries, saving us time and helping us to further comprehend the texts being summarized.

Whether for academic research, business insights, or just staying informed in a technologically advanced world, extractive summarization is a practical way to navigate the sea of information we’re surrounded by. As natural language processing continues to evolve, tools like extractive summarization will become even more essential, helping us to quickly find and understand the information that matters most in a world where every minute counts.

Original. Reposted with permission.

Kevin Vu manages Exxact Corp blog and works with many of its talented authors who write about different aspects of Deep Learning.

More On This Topic

  • Building a Knowledge Graph for Job Search Using BERT
  • How to Train a Joint Entities and Relation Extraction Classifier…
  • Classifying Long Text Documents Using BERT
  • Web LLM: Bring LLM Chatbots to the Browser
  • Approaches to Text Summarization: An Overview
  • Getting Started with Automated Text Summarization

ChatGPT can now read its responses aloud — here’s how to listen to them

ChatGPT will read its responses aloud to you

Don't feel like just reading a response you receive from OpenAI's ChatGPT? Well, now you can sit back and listen to it. Already available in the mobile app and rolling out to the website, a feature called Read Aloud will do just that — read the response aloud to you.

To try this out on your mobile device, install the ChatGPT app for iOS/iPadOS or Android if you don't already have it. Sign in with your account. Ask the bot to generate some content — a story, a poem, a haiku, an essay, a joke — whatever might sound mellifluous to your ears.

Also: How to use ChatGPT

After the response appears, press down on any part of it. From the pop-up menu, select Read Aloud. ChatGPT then reads it aloud to you. From a floating toolbar at the top, you can pause and restart the reading, skip ahead or go back 15 seconds, and even stop it altogether.

You can also listen to any past responses. Just tap the three-lined hamburger icon at the upper left to access your past conversations. Tap the one you want to hear and again press down on it and select Read Aloud.

Also: The best AI chatbots

What if you don't like the voice reading the response? No problem. Tap the hamburger icon, select your name at the bottom, and then tap the settings for Voice. Choose among such voices as Breeze, Cover, Ember, Juniper, and Sky. Just tap the voice to hear how it sounds. Tap Confirm when you find the voice of your dreams.

Read Aloud works easily enough in the mobile app. I didn't yet see it on the ChatGPT website, so it's likely still in the process of rolling out online.

OpenAI has certainly been at the forefront of generative AI. But other rivals have been nipping at its heels looking for a piece of the action. Those include Microsoft with Copilot, Google with Gemini, and even Anthropic with Claude AI (which has been boasting about its new Claude 3 model). That means OpenAI needs to keep innovating with ChatGPT.

Also: I asked Gemini and GPT-4 to explain deep learning AI, and Gemini won hands down

Sure, a reading feature seems like a minor enhancement. Some of the folks who responded to OpenAI's X post about Read Aloud grumbled that the company didn't announce something more exciting, like GPT-5 or the text-to-video model Sora. But any feature that spices up the overall experience can help. And hopefully, we'll get updates on GPT-5, Sora, and other new items in the mix before too long.

Artificial Intelligence

YOLOv9: A Leap in Real-Time Object Detection

Object detection has seen rapid advancement in recent years thanks to deep learning algorithms like YOLO (You Only Look Once). The latest iteration, YOLOv9, brings major improvements in accuracy, efficiency and applicability over previous versions. In this post, we’ll dive into the innovations that make YOLOv9 a new state-of-the-art for real-time object detection.

A Quick Primer on Object Detection

Before getting into what's new with YOLOv9, let's briefly review how object detection works. The goal of object detection is to identify and locate objects within an image, like cars, people or animals. It's a key capability for applications like self-driving cars, surveillance systems, and image search.

The detector takes an image as input and outputs bounding boxes around detected objects, each with an associated class label. Popular datasets like MS COCO provide thousands of labeled images to train and evaluate these models.

There are two main approaches to object detection:

  • Two-stage detectors like Faster R-CNN first generate region proposals, then classify and refine the boundaries of each region. They tend to be more accurate but slower.
  • Single-stage detectors like YOLO apply a model directly over the image in a single pass. They trade off some accuracy for very fast inference times.

YOLO pioneered the single-stage approach. Let's look at how it has evolved over multiple versions to improve accuracy and efficiency.

Review of Previous YOLO Versions

The YOLO (You Only Look Once) family of models has been at the forefront of fast object detection since the original version was published in 2016. Here's a quick overview of how YOLO has progressed over multiple iterations:

  • YOLOv1 proposed a unified model to predict bounding boxes and class probabilities directly from full images in a single pass. This made it extremely fast compared to previous two-stage models.
  • YOLOv2 improved upon the original by using batch normalization for better stability, anchoring boxes at various scales and aspect ratios to detect multiple sizes, and a variety of other optimizations.
  • YOLOv3 added a new feature extractor called Darknet-53 with more layers and shortcuts between them, further improving accuracy.
  • YOLOv4 combined ideas from other object detectors and segmentation models to push accuracy even higher while still maintaining fast inference.
  • YOLOv5 fully rewrote YOLOv4 in PyTorch and added a new feature extraction backbone called CSPDarknet along with several other enhancements.
  • YOLOv6 continued to optimize the architecture and training process, with models pre-trained on large external datasets to boost performance further.

So in summary, previous YOLO versions achieved higher accuracy through improvements to model architecture, training techniques, and pre-training. But as models get bigger and more complex, speed and efficiency start to suffer.

The Need for Better Efficiency

Many applications require object detection to run in real-time on devices with limited compute resources. As models become larger and more computationally intensive, they become impractical to deploy.

For example, a self-driving car needs to detect objects at high frame rates using processors inside the vehicle. A security camera needs to run object detection on its video feed within its own embedded hardware. Phones and other consumer devices have very tight power and thermal constraints.

Recent YOLO versions obtain high accuracy with large numbers of parameters and multiply-add operations (FLOPs). But this comes at the cost of speed, size and power efficiency.

For example, YOLOv5-L requires over 100 billion FLOPs to process a single 1280×1280 image. This is too slow for many real-time use cases. The trend of ever-larger models also increases risk of overfitting and makes it harder to generalize.

So in order to expand the applicability of object detection, we need ways to improve efficiency – getting better accuracy with less parameters and computations. Let's look at the techniques used in YOLOv9 to tackle this challenge.

YOLOv9 – Better Accuracy with Less Resources

The researchers behind YOLOv9 focused on improving efficiency in order to achieve real-time performance across a wider range of devices. They introduced two key innovations:

  1. A new model architecture called General Efficient Layer Aggregation Network (GELAN) that maximizes accuracy while minimizing parameters and FLOPs.
  2. A training technique called Programmable Gradient Information (PGI) that provides more reliable learning gradients, especially for smaller models.

Let's look at how each of these advancements helps improve efficiency.

More Efficient Architecture with GELAN

The model architecture itself is critical for balancing accuracy against speed and resource usage during inference. The neural network needs enough depth and width to capture relevant features from the input images. But too many layers or filters lead to slow and bloated models.

The authors designed GELAN specifically to squeeze the maximum accuracy out of the smallest possible architecture.

GELAN uses two main building blocks stacked together:

  • Efficient Layer Aggregation Blocks – These aggregate transformations across multiple network branches to capture multi-scale features efficiently.
  • Computational Blocks – CSPNet blocks help propagate information across layers. Any block can be substituted based on compute constraints.

By carefully balancing and combining these blocks, GELAN hits a sweet spot between performance, parameters, and speed. The same modular architecture can scale up or down across different sizes of models and hardware.

Experiments showed GELAN fits more performance into smaller models compared to prior YOLO architectures. For example, GELAN-Small with 7M parameters outperformed the 11M parameter YOLOv7-Nano. And GELAN-Medium with 20M parameters performed on par with YOLOv7 medium models requiring 35-40M parameters.

So by designing a parameterized architecture specifically optimized for efficiency, GELAN allows models to run faster and on more resource constrained devices. Next we'll see how PGI helps them train better too.

Better Training with Programmable Gradient Information (PGI)

Model training is just as important to maximize accuracy with limited resources. The YOLOv9 authors identified issues training smaller models caused by unreliable gradient information.

Gradients determine how much a model's weights are updated during training. Noisy or misleading gradients lead to poor convergence. This issue becomes more pronounced for smaller networks.

The technique of deep supervision addresses this by introducing additional side branches with losses to propagate better gradient signal through the network. But it tends to break down and cause divergence for smaller lightweight models.

YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information

YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information https://arxiv.org/abs/2402.13616

To overcome this limitation, YOLOv9 introduces Programmable Gradient Information (PGI). PGI has two main components:

  • Auxiliary reversible branches – These provide cleaner gradients by maintaining reversible connections to the input using blocks like RevCols.
  • Multi-level gradient integration – This avoids divergence from different side branches interfering. It combines gradients from all branches before feeding back to the main model.

By generating more reliable gradients, PGI helps smaller models train just as effectively as bigger ones:

Experiments showed PGI improved accuracy across all model sizes, especially smaller configurations. For example, it boosted AP scores of YOLOv9-Small by 0.1-0.4% over baseline GELAN-Small. The gains were even more significant for deeper models like YOLOv9-E at 55.6% mAP.

So PGI enables smaller, efficient models to train to higher accuracy levels previously only achievable by over-parameterized models.

YOLOv9 Sets New State-of-the-Art for Efficiency

By combining the architectural advances of GELAN with the training improvements from PGI, YOLOv9 achieves unprecedented efficiency and performance:

  • Compared to prior YOLO versions, YOLOv9 obtains better accuracy with 10-15% fewer parameters and 25% fewer computations. This brings major improvements in speed and capability across model sizes.
  • YOLOv9 surpasses other real-time detectors like YOLO-MS and RT-DETR in terms of parameter efficiency and FLOPs. It requires far fewer resources to reach a given performance level.
  • Smaller YOLOv9 models even beat larger pre-trained models like RT-DETR-X. Despite using 36% fewer parameters, YOLOv9-E achieves better 55.6% AP through more efficient architectures.

So by addressing efficiency at the architecture and training levels, YOLOv9 sets a new state-of-the-art for maximizing performance within constrained resources.

GELAN – Optimized Architecture for Efficiency

YOLOv9 introduces a new architecture called General Efficient Layer Aggregation Network (GELAN) that maximizes accuracy within a minimum parameter budget. It builds on top of prior YOLO models but optimizes the various components specifically for efficiency.

https://arxiv.org/abs/2402.13616

YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information
https://arxiv.org/abs/2402.13616

Background on CSPNet and ELAN

Recent YOLO versions since v5 have utilized backbones based on Cross-Stage Partial Network (CSPNet) for improved efficiency. CSPNet allows feature maps to be aggregated across parallel network branches while adding minimal overhead:

This is more efficient than just stacking layers serially, which often leads to redundant computation and over-parameterization.

YOLOv7 upgraded CSPNet to Efficient Layer Aggregation Network (ELAN), which simplified the block structure:

ELAN removed shortcut connections between layers in favor of an aggregation node at the output. This further improved parameter and FLOPs efficiency.

Generalizing ELAN for Flexible Efficiency

The authors generalized ELAN even further to create GELAN, the backbone used in YOLOv9. GELAN made key modifications to improve flexibility and efficiency:

  • Interchangeable computational blocks – Previous ELAN had fixed convolutional layers. GELAN allows substituting any computational block like ResNets or CSPNet, providing more architectural options.
  • Depth-wise parametrization – Separate block depths for main branch vs aggregator branch simplifies fine-tuning resource usage.
  • Stable performance across configurations – GELAN maintains accuracy with different block types and depths, allowing flexible scaling.

These changes make GELAN a strong but configurable backbone for maximizing efficiency:

In experiments, GELAN models consistently outperformed prior YOLO architectures in accuracy per parameter:

  • GELAN-Small with 7M parameters beat YOLOv7-Nano's 11M parameters
  • GELAN-Medium matched heavier YOLOv7 medium models

So GELAN provides an optimized backbone to scale YOLO across different efficiency targets. Next we'll see how PGI helps them train better.

PGI – Improved Training for All Model Sizes

While architecture choices impact efficiency at inference time, training process also affects model resource usage. YOLOv9 uses a new technique called Programmable Gradient Information (PGI) to improve training across different model sizes and complexities.

The Problem of Unreliable Gradients

During training, a loss function compares model outputs to ground truth labels and computes an error gradient to update parameters. Noisy or misleading gradients lead to poor convergence and efficiency.

Very deep networks exacerbates this through the information bottleneck – gradients from deep layers are corrupted by lost or compressed signals.

Deep supervision helps by introducing auxiliary side branches with losses to provide cleaner gradients. But it often breaks down for smaller models, causing interference and divergence between different branches.

So we need a way to provide reliable gradients that works across all model sizes, especially smaller ones.

Introducing Programmable Gradient Information (PGI)

To address unreliable gradients, YOLOv9 proposes Programmable Gradient Information (PGI). PGI has two main components designed to improve gradient quality:

1. Auxiliary reversible branches

Additional branches provide reversible connections back to the input using blocks like RevCols. This maintains clean gradients avoiding the information bottleneck.

2. Multi-level gradient integration

A fusion block aggregates gradients from all branches before feeding back to the main model. This prevents divergence across branches.

By generating more reliable gradients, PGI improves training convergence and efficiency across all model sizes:

  • Lightweight models benefit from deep supervision they couldn't use before
  • Larger models get cleaner gradients enabling better generalization

Experiments showed PGI boosted accuracy for small and large YOLOv9 configurations over baseline GELAN:

  • +0.1-0.4% AP for YOLOv9-Small
  • +0.5-0.6% AP for larger YOLOv9 models

So PGI's programmable gradients enable models big and small to train more efficiently.

YOLOv9 Sets New State-of-the-Art Accuracy

By combining architectural improvements from GELAN and training enhancements from PGI, YOLOv9 achieves new state-of-the-art results for real-time object detection.

Experiments on the COCO dataset show YOLOv9 surpassing prior YOLO versions, as well as other real-time detectors like YOLO-MS, in accuracy and efficiency:

Some key highlights:

  • YOLOv9-Small exceeds YOLO-MS-Small with 10% fewer parameters and computations
  • YOLOv9-Medium matches heavier YOLOv7 models using less than half the resources
  • YOLOv9-Large outperforms YOLOv8-X with 15% fewer parameters and 25% fewer FLOPs

Remarkably, smaller YOLOv9 models even surpass heavier models from other detectors that use pre-training like RT-DETR-X. Despite 4x fewer parameters, YOLOv9-E outperforms RT-DETR-X in accuracy.

These results demonstrate YOLOv9's superior efficiency. The improvements enable high-accuracy object detection in more real-world use cases.

Key Takeaways on YOLOv9 Upgrades

Let's quickly recap some of the key upgrades and innovations that enable YOLOv9's new state-of-the-art performance:

  • GELAN optimized architecture – Improves parameter efficiency through flexible aggregation blocks. Allows scaling models for different targets.
  • Programmable gradient information – Provides reliable gradients through reversible connections and fusion. Improves training across model sizes.
  • Greater accuracy with fewer resources – Reduces parameters and computations by 10-15% over YOLOv8 with better accuracy. Enables more efficient inference.
  • Superior results across model sizes – Sets new state-of-the-art for lightweight, medium, and large model configurations. Outperforms heavily pre-trained models.
  • Expanded applicability – Higher efficiency broadens viable use cases, like real-time detection on edge devices.

By directly addressing accuracy, efficiency, and applicability, YOLOv9 moves object detection forward to meet diverse real-world needs. The upgrades provide a strong foundation for future innovation in this critical computer vision capability.

Open source LLMs – no more than a marketing term?

1709153626076

Image source: Dall-e

This week, the tech community has been abuzz with the announcement of the latest model from Mistral being closed source. This revelation confirms a suspicion held by many: the concept of open-source Large Language Models (LLMs) today is more a marketing term than a substantive promise.

Historically, open source has been championed as inherently positive. In doing so, it has increasingly been painting itself into a corner in its quest for seeing to be ‘good’.

At the heart of the open-source debate are the licensing frameworks: copyleft licenses, which mandate the public sharing of derivative works (ex the GPL licence). Over time, we had commercially friendly open source licenses where derivative works could be retained (ex the MIT licence).

The rise of cloud computing has further complicated the open-source landscape. As early as 2008, Richard Stallman warned of cloud computing’s potential to sideline open source by shifting the locus of control. This shift is now mirrored in the realm of LLMs, where the focus is not just on the models’ weights and architecture but also on the exclusive training data, which remains proprietary with all LLMs(open source or closed). This similarity between open and closed source models reveals that the challenges of transparency, bias, fairness, and privacy are universal, diluting the distinction between the two approaches.

Critics like Gary Marcus view open-source LLLMs with skepticism, fearing they pose unique threats.

The debate over open-source LLMs is reminiscent of broader discussions about the moral positioning of tech companies. Google’s recent controversies over Gemini images highlight the pitfalls of claiming a moral high ground in technology development. Such stances, while noble, can become untenable over time, especially when unforeseen challenges emerge.

In conclusion, the discourse surrounding open-source LLMs should be reframed to recognize them as one part of a broader ecosystem rather than a panacea. The development and deployment of LLMs are complex and resource-intensive processes that necessitate a pragmatic approach to innovation, collaboration, and regulation. As the landscape evolves, it will be interesting to observe how open-source initiatives adapt and how the community navigates the delicate balance between openness, innovation, and commercial viability.

The recent developments with Mistral and the ongoing dialogue around the AI act underscore the nuanced and multifaceted nature of these discussions, highlighting the need for a balanced and informed approach to the future of LLMs.

The views expressed here are my own and are not related to any organization I am associated with