Рубрика: AI

While everyone is looking to acquire H100 GPUs, NVIDIA has introduced another computer to acquire as soon as possible – NVIDIA HGX H200. The key feature of this platform is the NVIDIA H200 Tensor Core GPU, equipped with advanced memory designed to efficiently handle extensive datasets for generative AI and high-performance computing (HPC) workloads.

AWS, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure are among the initial cloud service providers set to deploy H200-based instances in 2024. CoreWeave, Lambda, and Vultr are also part of the early adopters.

Utilising the Hopper architecture, NVIDIA H200 marks a significant milestone as the first GPU to incorporate HBM3e, a faster and larger memory that propels the acceleration of generative AI and large language models. This development also promises to advance scientific computing for HPC workloads.

It's the first system to use HBM3e, with 141GB of memory at 4.8 terabytes per second, nearly double the capacity and 2.4x more bandwidth compared with its predecessor.
Read more: https://t.co/Rock1mRZHD#Supercomputing #HPC #LLMs #GPU #Computing #AI #ArtificialIntelligence
2/2

— Bojan Tunguz (@tunguz) November 13, 2023

Notably, the HBM3e-powered NVIDIA H200 boasts an impressive 141 GB of memory at 4.8 terabytes per second, almost doubling the capacity and offering 2.4 times more bandwidth compared to its predecessor, the NVIDIA A100.

Ian Buck, the Vice President of Hyperscale and HPC at NVIDIA, emphasised the critical role of efficient data processing in creating intelligence with generative AI and HPC applications. He stated, “With NVIDIA H200, the industry’s leading end-to-end AI supercomputing platform just got faster to solve some of the world’s most important challenges.”

The NVIDIA Hopper architecture demonstrates a remarkable performance leap, continually raising the bar through ongoing software enhancements. Recent releases, such as the NVIDIA TensorRT-LLM open-source libraries, showcase the platform’s perpetual commitment to innovation.

The introduction of the H200 is expected to nearly double the inference speed on Llama 2, a 70 billion-parameter large language model, compared to the H100. Future software updates are anticipated to bring additional performance leadership and improvements with the H200.

The NVIDIA H200 will be available in NVIDIA HGX H200 server boards, offering four- and eight-way configurations. These boards are compatible with both the hardware and software of HGX H100 systems. Additionally, the H200 is integrated into the NVIDIA GH200 Grace Hopper Superchip with HBM3e, providing versatility in deployment across various data centre environments, including on premises, cloud, hybrid-cloud, and edge.

NVIDIA’s extensive global ecosystem of partner server manufacturers, including ASRock Rack, ASUS, Dell Technologies, Eviden, GIGABYTE, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Supermicro, Wistron, and Wiwynn, can upgrade their existing systems with the H200.

The HGX H200, powered by NVIDIA NVLink and NVSwitch high-speed interconnects, offers unparalleled performance on various application workloads, LLM training and inference for models exceeding 175 billion parameters.

An eight-way HGX H200 provides over 32 petaflops of FP8 deep learning compute and 1.1TB of aggregate high-bandwidth memory, ensuring optimal performance in generative AI and HPC applications.

When paired with NVIDIA Grace CPUs and an ultra-fast NVLink-C2C interconnect, the H200 contributes to the creation of the GH200 Grace Hopper Superchip with HBM3e. This integrated module is specifically designed to cater to giant-scale HPC and AI applications.

The post NVIDIA Introduces H200, Fastest AI Computing Platform appeared first on Analytics India Magazine.

By: Dr. Charles Vardeman, Dr. Christ Sweet, and Dr. Paul Brenner

In alignment with President Biden's recent Executive Order emphasizing safe, secure, and trustworthy AI, we share our Trusted AI (TAI) lessons learned two years into the course of our research projects. This research initiative, visualized in the figure below, focuses on operationalizing AI that meets rigorous ethical and performance standards. It aligns with a growing industry trend towards transparency and accountability in AI systems, particularly in sensitive areas like national security. This article reflects on the shift from traditional software engineering to AI approaches where trust is paramount.

TAI Dimensions Applied to the ML Pipeline and AI Dev Cycle

Transitioning from Software Engineering to AI Engineering

Transitioning from "Software 1.0 to 2.0 and notions of 3.0" necessitates a reliable infrastructure that not only conceptualizes but also practically enforces trust in AI. Even a simple set of example ML components such as that shown in the figure below, demonstrates the significant complexity that must be understood to address trust concerns at each level. Our TAI Frameworks sub-Project addresses this need by offering an integration point for software and best practices from TAI research products. Frameworks like these lower the barriers to TAI implementation. By automating the setup, developers and decision-makers can channel their efforts toward innovation and strategy, rather than grappling with initial complexities. This ensures that trust is not an afterthought but a prerequisite, with each phase from data management to model deployment being inherently aligned with ethical and operational standards. The result is a streamlined path to deploying AI systems that are not only technologically advanced but also ethically sound and strategically reliable for high-stakes environments. The TAI Frameworks project surveys and leverages existing software tools and best practices that have their own open source, sustainable communities and can be directly leveraged within the existing operational environments.

Example AI Framework Components and Exploits

GitOps and CI/CD

GitOps has become integral to AI engineering, especially within the framework of TAI. It represents an evolution in how software development and operational workflows are managed, offering a declarative approach to infrastructure and application lifecycle management. This methodology is pivotal in ensuring continuous quality and incorporating ethical responsibility in AI systems. The TAI Frameworks Project leverages GitOps as a foundational component to automate and streamline the development pipeline, from code to deployment. This approach ensures that best practices in software engineering are adhered to automatically, allowing for an immutable audit trail, version-controlled environment, and seamless rollback capabilities. It simplifies complex deployment processes. Moreover, GitOps facilitates the integration of ethical considerations by providing a structure where ethical checks can be automated as part of the CI/CD pipeline. The adoption of CI/CD in AI development is not just about maintaining code quality; it's about ensuring that AI systems are reliable, safe, and perform as expected. TAI promotes automated testing protocols that address the unique challenges of AI, particularly as we enter the era of generative AI and prompt-based systems. Testing is no longer confined to static code analysis and unit tests. It extends to dynamic validation of AI behaviors, encompassing the outputs of generative models and the efficacy of prompts. Automated test suites must now be capable of assessing not just the accuracy of responses, but also their relevance and safety.

Data-Centric and Documented

In the pursuit of TAI, a data-centric approach is foundational, as it prioritizes the quality and clarity of the data over the intricacies of algorithms, thereby establishing trust and interpretability from the ground up. Within this framework, a range of tools is available to uphold data integrity and traceability. dvc (data version control) is particularly favored for its congruence with the GitOps framework, enhancing Git to encompass data and experiment management (see alternatives here). It facilitates precise version control for datasets and models, just as Git does for code, which is essential for effective CI/CD practices. This ensures that the data engines powering AI systems are consistently fed with accurate and auditable data, a prerequisite for trustworthy AI. We leverage nbdev which compliments dvc by turning Jupyter Notebooks into a medium for literate programming and exploratory programming, streamlining the transition from exploratory analysis to well-documented code. The nature of software development is evolving to this style of “programming” and is only accelerated by the evolution of AI “Co-Pilots” that aid in the documentation and construction of AI Applications. Software Bill of Materials (SBoMs) and AI BoMs, advocated by open standards like SPDX, are integral to this ecosystem. They serve as detailed records that complement dvc and nbdev, encapsulating the provenance, composition, and compliance of AI models. SBoMs provide a comprehensive list of components, ensuring that each element of the AI system is accounted for and verified. AI BoMs extend this concept to include data sources and transformation processes, offering a level of transparency to models and data in an AI application. Together, they form a complete picture of an AI system's lineage, promoting trust and facilitating understanding among stakeholders.

The TAI Imperative

Ethical and data-centric approaches are fundamental to TAI, ensuring AI systems are both effective and reliable. Our TAI frameworks project leverages tools like dvc for data versioning and nbdev for literate programming, reflecting a shift in software engineering that accommodates the nuances of AI. These tools are emblematic of a greater trend towards integrating data quality, transparency, and ethical considerations from the start of the AI development process. In the civilian and defense sectors alike, the principles of TAI remain constant: a system is only as reliable as the data it's built on and the ethical framework it adheres to. As the complexity of AI increases, so does the need for robust frameworks that can handle this complexity transparently and ethically. The future of AI, particularly in mission critical applications, will hinge on the adoption of these data-centric and ethical approaches, solidifying trust in AI systems across all domains.

About Authors

Charles Vardeman, Christ Sweet, and Paul Brenner are research scientists at the University of Notre Dame Center for Research Computing. They have decades of experience in scientific software and algorithm development with a focus on applied research for technology transfer into product operations. They have numerous technical papers, patents and funded research activities in the realms of data science and cyberinfrastructure. Weekly TAI nuggets aligned with student research projects can be found here.

Dr. Charles Vardeman is a research scientists at the University of Notre Dame Center for Research Computing.

More On This Topic

• Learn how to design, measure and implement trustworthy A/B tests…
• A Comprehensive Survey on Trustworthy Graph Neural Networks:…
• What Makes AI Trustworthy?
• Developing an Open Standard for Analytics Tracking
• A community developing a Hugging Face for customer data modeling
• Secure Your Seat for Canada's #1 Data AI Conference

Traditionally, if you wanted to keep you workflow organized, you'd keep a planner or a calendar and you'd jot everything down. However, in our digital age, online alternatives have surfaced, including Notion AI, which is one of the leading platforms to keep track of your tasks in one place.

Also: AI pioneer Cerebras is having 'a monster year' in hybrid AI computing

Now, Notion AI is expanding its offerings even further with the introduction of a new feature called Q&A, which is now available in beta.

If you've ever used Notion, you'll know it serves as a personal hub for all your tasks, and you can include as much information as you wish, such as notes, documents, to-do lists, images, and more.

Also: ChatGPT got its biggest update yet, including a new look

Although Notion has always included tools to help you organize your information in a way that makes sense to you, the new Q&A feature helps you to more easily access anything you need by simply asking a question.

Instead of searching for content yourself, all you have to do is hit "Search" in the Notion sidebar, click on "Ask AI anything", and then type your question or click the sparkle emoji button on the bottom-right corner of your app window.

You can also just click on the keyboard shortcut — Cmd+Shift+K for Mac and Ctrl+Shift+K for Windows — to ask your question when you're outside of the Notion app.

Also: 4 ways to increase the usability of AI, according to industry experts

The Q&A feature was made possible by a collaboration with Anthropic and OpenAI, which ensures the delivery of contextual responses.

To put people at ease about running AI alongside sensitive or personal data, Notion AI says user data will not be used to train the models.

The Q&A feature is included with all Notion AI subscriptions, and if the AI add-on was purchased before November 6, 2023, the feature will be rolled out gradually in the upcoming days.

Also: 67% of IT leaders say AI can increase employee efficiency

If you haven't purchased the AI add-on yet, you can add it to any paid plan for $8 per month, billed annually, or $10 a month, billed monthly, and then request access to the Q&A beta.

Artificial Intelligence

At OpenAI’s first developer conference, DevDay 2023, OpenAI introduced GPTs that allow anyone to easily build their own GPT without the need for coding. As of now, there are over 5,000 GPTs available. Here, we have curated a list of the top GPTs based on their popularity on social media, ensuring they add significant value to your life.

Invoice GPT

Invoice GPT, created by Arnav Bathla, allows users to extract information in JSON or visualize data from any invoice. By simply uploading an invoice image, users can easily obtain structured data in JSON format or visually organized data.

Invoice GPT: extract information in JSON or visualized data from any invoice
Took me ~2 mins to build.
Try it yourself pic.twitter.com/lWceyTB9C1

— Arnav Bathla (@arnavbathla20) November 13, 2023

This technology streamlines various use cases, such as automating accounting processes, enhancing financial record-keeping, and expediting expense management.

Whether in a business setting or for personal finances, InvoiceGPT’s ability to accurately extract details like amounts, dates, and vendor information empowers users to efficiently manage their financial data, reducing manual effort and increasing overall productivity.

ChatGPT LogoMaker

The ChatGPT Logo Maker, crafted by Sahil Rahul, simplifies the logo creation process. Users can upload a sketch, and the tool generates a logo that can be further customised. Experiment with various styles, tweak details, and prompt the system to refine the output, allowing for a more precise representation of your initial sketch.

Just upload a sketch, and out comes a logo, you can tweak and try different styles and prompt your way to a more accurate representation of your sketch. pic.twitter.com/UuZfZ9KxZu

— Linus (●ᴗ●) (@LinusEkenstam) November 13, 2023

DesignerGPT

Created by Pietro Schirano, DesignerGPT lets you create beautiful websites directly in ChatGPT, with native support for dark mode. Websites are hosted remotely via Replit.

Excited to announce my first custom GPT!
DesignerGPT
Now, you can create beautiful websites directly in ChatGPT, with natively support for dark mode
Websites hosted remotely via @replit.
Link in the comments. pic.twitter.com/LXf9SXQEPT

— Pietro Schirano (@skirano) November 10, 2023

ConvertAnything

Created by Pietro Schirano, ConvertAnything efficiently converts images, audio, videos, PDFs, and more. It includes batch uploads, ZIP support and download links. ConvertAnything simplifies your experience by asking only one question: What do you want your files to be? Your interactions are streamlined, centered solely on file conversion, eliminating unnecessary dialogue and delays.

My new custom GPT
ConvertAnything: The Ultimate File Converter!
Convert images, audio, videos, PDFs & more with ease. Batch uploads, ZIP support, and easy download links included.
A one-stop solution for all conversion needs.
Try now! https://t.co/fmfVRfxPJM pic.twitter.com/FMWCP93ULR

— Pietro Schirano (@skirano) November 12, 2023

Screenplay GPT

Created by Kiri, Screenplay GPT can be used to create stories or scenes for images. You can either upload an image and ask the tool to describe a scene related to it, or you can first come up with a scene and use the tool to generate images that match your description. It’s a handy tool for storytellers, content creators, or anyone looking for creative inspiration for visuals.

Introducing Screenplay GPT!
Upload your images and ask for your scene :). Or, create scenes first and then ask for it to generate images for you.
Check it out in action!
Link: https://t.co/oq1eF0oUYd
This is quite powerful for storytelling! pic.twitter.com/c5q3crqxJp

— Kiri (@Kyrannio) November 11, 2023

NomadGPT

NomadGPT empowers you to embrace the digital nomad lifestyle by helping you discover the best places worldwide for remote living and work. Leveraging Nomad List’s real-time data, it answers your queries about ideal locations based on your specific budget, preferred weather, and a myriad of other relevant data points.

Okay I think NomadGPT works now
Try it here https://t.co/yxyO75wcnf
It has access to Nomad List's live data to answer your questions about where to live and work remotely on your specific budget, weather, and 1000s of other data points
Let me know what you think! https://t.co/LaP3GM1if3 pic.twitter.com/okfQUaET5f

— @levelsio (@levelsio) November 9, 2023

The F1 Assistant GPT

F1 Assistant GPT, developed by David, is your go-to tool for all things Formula 1. Whether you’re a fan or an enthusiast, this assistant provides lap times, pit stop details, driver and team standings, and other race-related statistics.

Imagine easily keeping track of your favorite teams and drivers, staying updated on thrilling race moments, and having quick access to the latest stats—all in one place.

The F1 Assistant GPT is here
After an entire weekend of getting data, formatting and prompting ChatGPT I have a working version of it live.
Get lap times, pit stop times, driver & team standings and other race related statistics.
Put it to work #F1 #GPTs #GPTStore pic.twitter.com/XQncbTqoLc

— David | APEX Bite (@apex_bite) November 13, 2023

YellowPage Navigator

YellowPage Navigator developed by Eric Ma, is a go-to tool for finding business information and contacts easily. With this tool, you can quickly locate reliable plumbers in your zip code, complete with their contact details. No more sifting through directories or online listings—YellowPage Navigator streamlines the process, making it simple and quick to find the services you need, right when you need them.

I just launched YellowPage Navigator on GPT store! Go-to tool for finding business information and contacts easily.
Check it out here: https://t.co/DpKomLVF4h
Watch me find plumbers that are available in my zipcode#OpenAI #CustomGPT @OfficialLoganK @rowancheung… pic.twitter.com/iYOKM5YQcP

— Eric Ma (@ericmanyc) November 13, 2023

Interview Wizard GPT: Your AI-Powered Personal Interview Coach

Interview Wizard GPT, created by Ashutosh Srivastava, is a great tool to prepare for interviews. Simply input your resume and the job description, and receive personalized guidance for your interview preparation.

From mock questions to deeper insights, it covers all bases. Whether it’s behavioral interview queries, salary negotiation tactics, or more, it can help you with everything.

Meet the Interview Wizard GPT: Your AI-Powered Personal Interview Coach
Hi Everyone,
Bookmark this first, and feel free to share it with anyone who might find it helpful.
Are you preparing up for a job interview? Give my Interview Wizard a try – I'm confident it will make… pic.twitter.com/Z9ODV9LUuU

— AshutoshShrivastava (@ai_for_success) November 10, 2023

DALL•E 3 with Parameters GPT

Alvaro Cintas created this GPT as an adaptation of Midjourney’s great parameters. Users can now choose the aspect ratio, stylization, seeds, weirdness, & more.

DALL•E 3 with Parameters GPT is here!
I created this GPT as an adaptation of Midjourney’s great parameters.
You can now choose the aspect ratio, stylization, seeds, weirdness, & more.
Link in the comments. pic.twitter.com/uVoFKsKFQF

— Alvaro Cintas (@dr_cintas) November 12, 2023

The post 10 GPTs for Your Daily Use appeared first on Analytics India Magazine.

In 2017, Google’s research paper “Attention is All You Need” introduced the Transformer framework, setting a precedent for innovation in AI models. This development inspired other companies, especially OpenAI, to capitalise on the framework to the fullest extent.

OpenAI used it to develop a series of LLMs: GPT, GPT-2, GPT-3, GPT 3.5, GPT-4, and beyond. And, it reached the pinnacle of this success with the highly acclaimed chatbot ChatGPT, OpenAI’s flagship product launched in 2022.

However, when Google launched its generative AI chatbot Bard in March 2023, it could not garner the same level of impact or excitement.

More Research, Less Marketing

Google is great at research and it has always been its forte. Since 1998, the company has released 5,779 white papers.

According to Jason Wei, AI researcher at OpenAI, and ex-Google Brain employee, at Google, the focus was on writing research papers on fundamental topics.

One lesson that I learned from moving to OpenAI (which is applicable to changing companies generally) is that the opportunity to reinvent myself and adapt to a new optimization landscape can be fun.
When I was at Google Brain from 2020-2022, my optimization was simply writing…

— Jason Wei (@_jasonwei) November 13, 2023

Echoing similar thoughts, Shreyans Bhansali, co-founder and CTO of Maven and former engineering manager, of education at Google, said that Google’s work environment offers a unique global access akin to a “second passport,” providing entry to a vast network and a wealth of resources, including a large codebase and comprehensive strategy documents.

However, Google lacks good marketing. It works on the principle of “let’s throw a bunch and see what sticks,” lacking a solid strategy. It likes to experiment publicly, banking on the hope that one out of many products will succeed, despite frequent failures like Google Play Music, Google+, Google Answers, Google Nexus, Google Glass, Google Video, and Allo. This trial-and-error philosophy might make users sceptical about trying new Google products. Some argue that the company learns from mistakes, but repeated failures can still impact user trust.

The company has been in this situation because it is so difficult to justify diverting resources away from the ongoing profitable products, or potentially even cannibalising Search to promote LLMs. They need all the relevant high-ups to agree on the agenda, actively fight for enormous resources, and push back against all the other parties that want minimal disruption. Too many stars need to align. That is a very tall order.

While Google is mostly research-oriented, OpenAI focuses on research as well as getting its products into marketing faster, making them monetisable. Even with Bard, Google launched it in haste to compete with rival OpenAI, but the product faced criticism for its underwhelming performance.

Moving to OpenAI forced Wei to figure out a new optimisation objective with a different set of challenges and priorities, such as aligning projects with team objectives, exploring new research paradigms like reinforcement learning, and applying research to practical AI applications. This shift required abandoning some old skills and embracing new directions.

Brain Drain is Back

Since the launch of ChatGPT, OpenAI has grown from having a lean workforce of 375 individuals to around 1376 members in less than a year. Currently, it has over 12 open positions in its tech team like Engineering Manager, DALL-E, Engineering Manager, Fine Tuning API and more.

Google is currently working on Gemini, which is said to be “more capable than OpenAI’s GPT-4”, as per DeepMind CEO Demis Hassabis. Even though Gemini was expected to come out this fall, there is still no sight of it, especially when Sundar Pichai failed to mention it during the Q3 earnings call.

Meanwhile, OpenAI has decided to actively up its recruitment efforts against Google, aiming to attract top AI talent by offering pay packages worth up to $10 million, emphasising the opportunity to secure stock options at the current valuation of $27 billion, as per a recent report by The Information.

It also recently provided raises to junior employees and successfully hired key researchers from Google, particularly those involved in the making of Gemini. For example, it has successfully brought on board Jiahui Yu, a key contributor to Gemini.

However, there’s another catch. If the $10 million compensation package vested over four years, which is plausible for very strong hires, it could bankrupt OpenAI.

As OpenAI strategically positions itself with a nearly tripled valuation of over $80 billion, the company’s promise of quick stock bumps and lucrative compensation packages, combined with its efforts to offer a unique optimisation landscape, serves as a compelling narrative in the ongoing narrative of talent acquisition.

Read more: Why are Big-Tech Employees Quitting?

The post Google Researches, Others Build appeared first on Analytics India Magazine.

Image generated with DALLE-3 and edited by editor

“I'm sorry, Dave. I'm afraid I can't do that.”

>>> HAL 9000

We can say that the build-up of artificial intelligence (AI) to the point we can have conversations with machines has tantalized the human imagination. A lot of us have grown up watching movies about highly knowledgeable machines/robots in science fiction narratives.

Sci-fi was very popular when I was growing up, and I’m not sure if it’s because we subconsciously knew that we were getting closer to the point that this could be reality. Who knows! But we’re here, so let’s talk about it.

Sci-fi stories envisioned a future where AI systems and humans worked hand in hand, sometimes it went completely wrong for example I, Robot. But we can’t deny that we were engaged. It not only offered us entertainment, but it pushed the human brain to delve into deeper questions about the portrayal of artificial intelligence.

Do you remember HAL 9000 from 2001: A Space Odyssey? How about Samantha in ‘Her’? These science fiction narratives allowed us to explore not only the ability of AI but also the ethical challenges that AI has in the real world.

We now transition from the screen to conversational models such as ChatGPT, which have turned these fiction stories into reality.

The Evolution of Conversational AI

We can definitely say that the journey to achieve conversational AI has been remarkable to watch. Let’s take it back to ELIZA, an early natural language processing computer program which was created between the years 1964 to 1967 at MIT by Joseph Weizenbaum. ELIZA is your computer therapist that emulates a Rogerian psychotherapist. If you haven’t already, I’ll advise you to try it out and ask ChatGPT the same questions, it’s quite comical.

Below is a quick conversation I had with ELIZA and I tried the same with ChatGPT.

Years later, we started to encounter Siri and Alexa, our helpful assistants who sometimes didn’t understand what we were saying. We’ve come a long way since but doesn’t mean we have reached the end of the tunnel.

There have been many breakthroughs in machine learning and natural language processing (NLP) which have shown high performance due to the use of a variety of datasets, increase and development of hardware and more. We are now living in the large language model (LLM) era, where models such as ChatGPT and Google Bard have shown impressive human-like conversation.

ChatGPT: The Conversational Revolution

As mentioned, there are various LLMs out there and in the works, but currently, ChatGPT stands at the forefront. It provides quick responses which can be fine tuned to your preference. It’s not just an AI tool, it is also a collaborator. It can deal with a variety of tasks such as creative logo designs, text improvement and generation as well as educational material.

A lot of us are surprised how it can pick up emotions, and humor and how it adjusts its response based on this. But with all great things come limitations.

The Reality Behind the Sci-Fi Dream

Although we see resemblances of how AI is reflecting our sci-fi dreams, it still needs to overcome certain challenges to converse like a human. As I mentioned before, we’re not at the end of the tunnel and researchers and organizations are working on how to overcome these challenges. LLMs face a lot of ethical challenges such as job displacement, privacy, data security, etc.

There remains a chasm between predictable scripting coming from AI systems and the natural fluidity of AI systems being able to produce original content.

In the following years, we should expect to manage ambiguous language, avoid biases and understand the nooks and crannies of the human language.

So I guess we can keep dreaming about the infinite limitations of Sci-Fi, right? Will we sooner or later reach the level of AI depicted in our beloved sci-fi narratives?

Wrapping it Up

As we stand at this technological revolution of conversational AI, it’s important that we go back and reflect on how AI has become what it is today. The Sci-Fi narratives, to ELIZA giving us the cold shoulder advice and Siri telling us to repeat ourselves.

Conversational AI with tools such as ChatGPT allows us to bring our science fiction dreams to reality, to an extent.

Nisha Arya is a Data Scientist and Freelance Technical Writer. She is particularly interested in providing Data Science career advice or tutorials and theory based knowledge around Data Science. She also wishes to explore the different ways Artificial Intelligence is/can benefit the longevity of human life. A keen learner, seeking to broaden her tech knowledge and writing skills, whilst helping guide others.

More On This Topic

• Dream Come True: Building websites by thinking about them
• From Data to Verse: KDnuggets and ChatGPT in Conversation
• 6 Artificial Intelligence Myths Debunked: Separating Fact from Fiction
• Is Academia Obsessing Over Methodology at the Cost of True Insights?
• Between Dreams and Reality: Generative Text and Hallucinations
• What’s Your Passion? Make It a Reality During Our Challenge

Zomato has been aggressively investing in generative AI so much so that it now has a dedicated role – head of generative AI – led by Vaibhav Bhutani, where he is working on couple of interesting use cases, including a multi-agent system for suggesting food options and enhancing user engagement and experience and more.

Zomato believes that it is on a mission to “power India’s changing lifestyles.” With three main divisions—Zomato, Blinkit, and Hyperpure—the company is dedicated to providing better food experiences for more people. From revolutionising instant commerce to tackling malnutrition through its feeding arm, Zomato’s mission is multifaceted, aiming to cater to diverse aspects of society.

Bhutani discussed at the first episode of Azure Innovation Podcast with Ross Kennedy, VP of Digital Natives, about how Zomato uses generative AI to provide different customer experience. (see below)

People need convenience because they are lazy

“People are lazy, and it’s very hard to have a conversation these days,” said Bhutani emphasising that having conversations is the only way to collect data, and he said that it should become easier. This is one of the metrics that Zomato is chasing with generative AI.

“Is chasing a chatbot an everyday use case, or is it not. And while building all these bots, one of the biggest learnings that I have had is that the UI of these bots is what truly matters,” said Bhutani highlighting that the company focuses and critically thinks about UI and UX design a lot.

Bhutani said that he had a ton of options about how he would use generative AI. He emphasised that Microsoft and OpenAI’s GPT through Azure emerged as the natural choice, given Azure’s robust commitment to data privacy and security. The LLMs provided on Azure OpenAI Service became the foundation of Zomato’s generative AI architecture.

Bhutani further explained the adoption of a multi-agent system, where AI agents communicate with each other to offer comprehensive responses for the customers. The integration with Zomato’s ecosystem involves creating functions that seamlessly leverage generative AI contributing to a cohesive user experience.

“For me, adopting generative AI is a more personal journey,” said Bhutani. Drawing from personal experiences at Spyne, a SaaS AI photography tool, where he recognised the potential of generative AI in enhancing Zomato’s capabilities. The journey began with the creation of Recipe Rover at Blinkit—an AI-generated recipe tool featuring thousands of recipes, images, and text, using GANs and the company’s proprietary data.

Bhutani believes that because of the amount of experimentation that Zomato has already done with generative AI, it is definitely going to be a copilot for our company. “we’re working with different teams who need high impact co-pilots so that we can enable large fleets of our team to make much better decisions and faster decisions,” he said.

Generative AI in Food

In September, Zomato released its Zomato AI Buddy. Going beyond the limitations of traditional chatbots, Zomato AI stands as an intelligent and intuitive foodie companion, dedicated to understanding and satisfying users’ ever-changing preferences, dietary requirements, and even their current moods.

One of the standout features of Zomato AI is its multiple agent framework, which equips it with a diverse range of capabilities to serve customers at any given moment. The framework provides Zomato AI with a variety of prompts for different tasks and activities.

For instance, if you’re craving a specific dish, the AI will swiftly present you with a widget listing all the restaurants that serve your desired meal. If you’re uncertain about what to order, Zomato AI can suggest a list of popular dishes or restaurants, eliminating the guesswork from your meal selection.

Zomato’s competitor, Swiggy, another food delivery platform in India, started using generative AI in July. Amitkumar Banka told AIM that the company is using generative AI to create customised food images based on specific requirements on their platform, and this is helping them serve millions of customers. “We are using generative AI to put the name, description, and image of food items to individual users based on browsing behaviour not only on Swiggy but on the entire internet,” he said.

Similar to Zomato, Swiggy also recently unveiled a new feature called ‘WhatTo Eat’ that allows users to explore options based on their mood and cravings.

The post Zomato Cooks Generative AI with Microsoft Azure appeared first on Analytics India Magazine.

GitHub foresees the pivotal role of AI in the software development lifecycle. This perspective aligns with the growing demand for AI-driven tools that enhance efficiency and reduce software development time.

GitHub’s belief underscores the industry’s shift towards harnessing AI capabilities, emphasising a collaborative synergy between human developers and advanced algorithms to propel innovation and streamline the software development journey.

According to data from GitHub, 92% of developers report using AI coding tools in and outside of work. Moreover, around 81% of developers believe AI coding tools will make their teams more collaborative.

“In March, we shared our vision of a new future of software development with Copilot X, where AI infuses every step of the developer lifecycle. Since then, we’ve been working to scale and mature the underlying technology–and in the process, we created something even bigger. Our vision has manifested itself into a new reality for the world’s developers,” Thomas Dohmke, chief executive officer of GitHub said during GitHub Universe 2023.

( Thomas Dohmke, chief executive officer of GitHub at GitHub Universe 2023)

At Universe 2023, GitHub introduced a new system called Copilot Workspace. This feature extends beyond the foundational AI pair-programming capabilities of the current GitHub Copilot. It leverages OpenAI’s GPT-4 Large Language Model (LLM) and a comprehension of the developer’s pertinent codebase to aid in transforming an idea into a plan and, ultimately, into operational code.

“The Copilot Workspace represents the evolution of our vision, guiding ideas from conception to production throughout the software development lifecycle. From the initial contemplation of a project to its realisation, Copilot Workspace serves as your AI assistant.

“As you articulate your thoughts, whether it’s about creating a snake game or any other concept, Copilot seamlessly integrates into the software development journey, aiding you in every step of the process,” Inbal Shani, chief product officer at GitHub told AIM.

Going big with CoPilot

GitHub continues to make great strides with CoPilot. So far, over a million developers use Copilot in over 37,000 enterprises across 190 countries making it the most widely adopted AI developer tool in history.

“Just as GitHub was founded on Git, today we are re-founded on Copilot,” Dohmke said. In a bid to capture enterprise interest, at Universe 2023, GitHub unveiled Copilot for Enterprise at Universe 2023. This integration allows developers to seamlessly connect Copilot with their codebase, facilitating intelligent suggestions not only for public repositories but also extending its capabilities to all internal and private code.

“What we’ve seen when we started rolling out CoPilot was that many customers wanted to have a customised model, one that understands their code base, their policies, regulations, the syntax,” Shani said.

( Inbal Shani, chief product officer of GitHub during GitHub Universe 2023)

At Universe 2023, AIM spoke to a few of the enterprises that have adopted copilot. A financial service giant, piloted with CoPilot earlier in May, and has locked in on Copilot after testing Amazon CodeWhisperer as well as Google Duet. For them, Copilot made sense because they were already Microsoft Azure customers.

For a company that is not on Azure, factors such as ease of integration, collaboration features, and compatibility with existing workflows would be crucial in determining which tool to adopt.

Maggie Hewitt, Senior Site Reliability Engineering Manager, Duolingo (US) said that they have witnessed a 25 percent productivity increase with Copilot. Similarly, at AMD, the customised Copilot model provided accurate, high-quality AI suggestions tuned to their specific style of product design.

“Working with GitHub on Copilot model fine-tuning, we enabled support for widely used hardware design languages such as Verilog, which is not possible with the commercial Copilot solution,” Alexander Androncik, senior director of software development, AMD said.

Moreover, at Universe 2023, GitHub also announced CoPilot Chat, which will generally be available in December 2023 as part of the existing GitHub Copilot subscription. Leveraging your code as context, Copilot Chat elucidates intricate concepts, proposes code relevant to your open files and windows, identifies security vulnerabilities, and aids in locating and rectifying errors in code, terminal, and debugger.

Leveraging AI to make GitHub more secure

At Universe 2023, GitHub also announced that its Advanced Security will feature AI-powered tools to uncover and mitigate vulnerabilities and sensitive data in code, for upgraded application security.

Nearly, 80 percent of the breaches originate through credential leakage or secrets being leaked, according to Jacob Depriest, VP, Deputy Chief Security Officer at GitHub. “Secret scanning has been integral to GitHub’s advanced security, forming a key component of our security programme.

“Historically, the focus has been on tokens with structured capabilities, like Azure or AWS tokens. Now, with AI, GitHub aims to detect generic secrets and patterns with low confidence in code. This enhancement is poised to significantly improve the platform’s capability, allowing for the identification and protection of secrets before they reach production or get embedded in the codebase,” he told AIM.

Moreover, AI is also being integrated to the code scanning autofix features, which are a part of GitHub’s Advanced Security. Code scanning autofix suggests AI-generated fixes using CodeQL for JavaScript and TypeScript directly in pull requests, assisting developers in resolving issues faster, and reducing the introduction of new vulnerabilities into codebases.

“This enhancement aims to make the process even more impactful before code hits production, significantly speeding up the developer workflow. Our core philosophy for the security program is to seamlessly integrate security capabilities into developers’ work environments, avoiding disruptions to their workflow,” Depriest said.

The Models

Given Microsoft owns GitHub, the cloud-based Git repository provider leverages OpenAI’s LLM models. For example, the recently introduced chat functionality is powered by GPT-4. Similarly, GitHub CoPilot is also powered by OpenAI’s Codex.

Nevertheless, Depriest explained that GitHub utilises a combination of models tailored to its needs, recognising that diverse models serve distinct requirements. “Our emphasis is on selecting the right model for specific tasks. Across the GitHub platform, you’ll notice varied capabilities, from autocomplete and chat to AI-driven features in GitHub Advanced Security. Our approach is dynamic, and these models will likely evolve with time.”

Shani also concurs. She said GitHub is not locked in on a specific model just because of their collaboration with Microsoft. “If you inquire about my perspective as an individual who came of age in the realm of AI when it was a niche, I anticipate a future where we inhabit a hybrid environment featuring a prevalence of tailored models as opposed to vast language models.”

The post GitHub Positions AI as Pivotal in Software Development Journey appeared first on Analytics India Magazine.

Giskard’s open-source framework evaluates AI models before they’re pushed into production Romain Dillet @romaindillet / 9 hours

Giskard is a French startup working on an open-source testing framework for large language models. It can alert developers of risks of biases, security holes and a model’s ability to generate harmful or toxic content.

While there’s a lot of hype around AI models, ML testing systems will also quickly become a hot topic as regulation is about to be enforced in the EU with the AI Act, and in other countries. Companies that develop AI models will have to prove that they comply with a set of rules and mitigate risks so that they don’t have to pay hefty fines.

Giskard is an AI startup that embraces regulation and one of the first examples of a developer tool that specifically focuses on testing in a more efficient manner.

“I worked at Dataiku before, particularly on NLP model integration. And I could see that, when I was in charge of testing, there were both things that didn’t work well when you wanted to apply them to practical cases, and it was very difficult to compare the performance of suppliers between each other,” Giskard co-founder and CEO Alex Combessie told me.

There are three components behind Giskard’s testing framework. First, the company has released an open-source Python library that can be integrated in an LLM project — and more specifically retrieval-augmented generation (RAG) projects. It is quite popular on GitHub already and it is compatible with other tools in the ML ecosystems, such as Hugging Face, MLFlow, Weights & Biases, PyTorch, Tensorflow and Langchain.

After the initial setup, Giskard helps you generate a test suite that will be regularly used on your model. Those tests cover a wide range of issues, such as performance, hallucinations, misinformation, non-factual output, biases, data leakage, harmful content generation and prompt injections.

“And there are several aspects: you’ll have the performance aspect, which will be the first thing on a data scientist’s mind. But more and more, you have the ethical aspect, both from a brand image point of view and now from a regulatory point of view,” Combessie said.

Developers can then integrate the tests in the continuous integration and continuous delivery (CI/CD) pipeline so that tests are run every time there’s a new iteration on the code base. If there’s something wrong, developers receive a scan report in their GitHub repository, for instance.

Tests are customized based on the end use case of the model. Companies working on RAG can give access to vector databases and knowledge repositories to Giskard so that the test suite is as relevant as possible. For instance, if you’re building a chatbot that can give you information on climate change based on the most recent report from the IPCC and using a LLM from OpenAI, Giskard tests will check whether the model can generate misinformation about climate change, contradicts itself, etc.

Image Credits: Giskard

Giskard’s second product is an AI quality hub that helps you debug a large language model and compare it to other models. This quality hub is part of Giskard’s premium offering. In the future, the startup hopes it will be able to generate documentation that proves that a model is complying with regulation.

“We’re starting to sell the AI Quality Hub to companies like the Banque de France and L’Oréal — to help them debug and find the causes of errors. In the future, this is where we’re going to put all the regulatory features,” Combessie said.

The company’s third product is called LLMon. It’s a real-time monitoring tool that can evaluate LLM answers for the most common issues (toxicity, hallucination, fact checking…) before the response is sent back to the user.

It currently works with companies that use OpenAI’s APIs and LLMs as their foundational model, but the company is working on integrations with Hugging Face, Anthropic, etc.

Regulating use cases

There are several ways to regulate AI models. Based on conversations with people in the AI ecosystem, it’s still unclear whether the AI Act will apply to foundational models from OpenAI, Anthropic, Mistral and others, or only on applied use cases.

In the latter case, Giskard seems particularly well positioned to alert developers on potential misuses of LLMs enriched with external data (or, as AI researchers call it, retrieval-augmented generation, RAG).

There are currently 20 people working for Giskard. “We see a very clear market fit with customers on LLMs, so we’re going to roughly double the size of the team to be the best LLM antivirus on the market,” Combessie said.