What runs ChatGPT? Inside Microsoft’s AI supercomputer

What runs ChatGPT? Inside Microsoft’s AI supercomputer Sponsored Content by Microsoft/NVIDIA June 15, 2023 by Jeremy Chapman, Director Microsoft Mechanics

Wonder what it takes to run the infrastructure that hosts ChatGPT with over 100 million users today? Get an inside look at the AI supercomputer infrastructure built to run ChatGPT and other large language models. Go behind the scenes with Mark Russinovich, Microsoft Azure CTO, in this Microsoft Mechanics show and learn:

  • How Microsoft collaborated with NVIDIA to deliver purpose-built AI infrastructure with NVIDIA GPUs
  • How Project Forge checkpointing works to restore job states if a long training job fails or needs to be migrated
  • How we used Low Ranking Adaptive (LoRA) fine-tuning to update a fraction of the base model for more training throughput and smaller checkpoints
  • How UK-based company, Wayve, is using Azure’s AI supercomputer infrastructure for self-driving cars
  • And how Confidential Computing works with Azure AI to combine datasets without sharing personally identifiable information for secure multiparty collaborations.

Why purpose-built Infrastructure for AI

AI is one of the most inspiring areas of technology evolution and is emerging as a key differentiator for businesses worldwide. Performance requirements for AI, however, are significantly different from other enterprise applications. Unlike conventional workloads, increasingly sophisticated AI models with billions of parameters require massive amounts of processing power plus lightning-fast networking and storage.

Advancements in AI infrastructure are at the foundation of that revolution. AI infrastructure is what powers your AI innovations and AI workloads require infrastructure built specifically for compute-intensive, large-scale AI workloads. An “AI-first” approach to infrastructure can accelerate model training and inference, increase performance and accuracy, and accelerate AI innovation.

Microsoft provides full stack cloud infrastructure purpose-built for AI. Engineered at every layer for AI, Azure AI infrastructure provides performance, scalability, and built-in security needed to build, train, and deploy the most demanding AI workloads with confidence, at any scale.

Learn more about Azure AI capabilities:

  • Microsoft/NVIDIA Solution Channel — HPCwire
  • Azure AI Infrastructure

Learn more about Microsoft Mechanics:

As Microsoft’s official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.

  • Subscribe to Microsoft Mechanics YouTube
  • Follow Microsoft Mechanics Blog
  • Subscribe to Microsoft Mechanics Podcast
  • Follow us on Twitter, LinkedIn, Instagram

#MakeAIyourReality

Related

10 ChatGPT Plugins for Data Science Cheat Sheet

Plugins for Data Science

ChatGPT is helping all of us, even data scientists, be more productive in a shorter period of time. The initial use cases and capabilities of ChatGPT were mind-blowing, and the situation has only improved with time. With plugins now being widely available to ChatGPT users, we thought it would be a good time to point out which plugins would be good candidates for data folks to be checking out and using to their advantage.

10 ChatGPT Plugins for Data Science Cheat Sheet

ChatGPT plugins are valuable third-party applications that can be easily integrated into the ChatGPT platform to bring automation and enhance its functionality. You can streamline workflows, automate tasks, and add new features to the platform.

For an overview of what we believe to be the 10 of the best ChatGPT plugins for data science, check out our latest cheat sheet, conveniently named 10 ChatGPT Plugins for Data Science Cheat Sheet.

You'll find plugins for coding, analysis, web searching, document interrogation, and more.

Check it out now, and check back soon for more.

More On This Topic

  • ChatGPT for Data Science Cheat Sheet
  • ChatGPT for Data Science Interview Cheat Sheet
  • The ChatGPT Cheat Sheet
  • Machine Learning with ChatGPT Cheat Sheet
  • GitHub CLI for Data Science Cheat Sheet
  • Docker for Data Science Cheat Sheet

New EU Regulations Will Be A Thorn In Google Bard’s Side

Earlier this week, the European Union passed a new set of rules focused on safe and transparent AI. Coincidentally, Google’s rollout of its Bard chatbot was met with a roadblock in Ireland as the Irish Data Protection Commission has stopped the launch of Bard in the country.

Amidst the new EU regulation, it seems that Bard will not only face issues from the DPC, but also from other data privacy regulators across Europe. In the bill signed yesterday, companies are now required to institute a new set of safeguards, as well as provide better transparency on what is done with user data.

Coming back to the Irish DPC’s block of Google Bard, the regulator stated that the company had not put in a detailed briefing prior to the chatbots launch. While Google has requested the required paperwork from the regulator, this is sure to push back the launch of Google Bard, which was slated for its EU release this week.

To make things worse for the ChatGPT challenger, the chatbot has received a score of 75 out of 100 for its lackluster privacy policies. In a report by the Common Sense Privacy Program, Bard received a warning note from the watchdog due to the various privacy issues with Bard’s privacy policy.

Set against the stringent regulatory environment in Europe, Bard’s flimsy user protection may not stand a chance. Under the new regulation, chatbots, like Bard and ChatGPT, are required to disclose where content is AI-generated, as well as have safeguards against generating illegal content. However, it seems that this isn’t going to stop Google. A spokesperson from the company stated, “We said in May that we wanted to make Bard more widely available, including in the European Union, and that we would do so responsibly, after engagement with experts, regulators and policymakers.”

Even if it comes to changing something fundamental in Bard, it seems that Google is not going to be stopped from rolling out the chatbot in the EU. This is no surprise, considering how easily the tech giant flaunts EU rules, as seen by its recent €2.42 billion fine due to antitrust violations.

The post New EU Regulations Will Be A Thorn In Google Bard’s Side appeared first on Analytics India Magazine.

Your Ultimate Guide to Chat GPT and Other Abbreviations

Your Ultimate Guide to Chat GPT and Other Abbreviations

What do all these abbreviations — ML, AI, AGI — mean?

ML (machine learning) is an approach to solving difficult computational problems – instead of coding using a programming language you build an algorithm that “learns” the solution from data samples.

AI (artificial intelligence) is a field of computer science dealing with problems (e.g., image classification, working with human language) that are difficult to solve using traditional programming. ML and AI go hand in hand, with ML being a tool to solve problems formulated in AI.

AGI (artificial general intelligence) — is the correct term for what popular culture usually implies by AI – the ability of computers to achieve human-like intellectual capabilities and broad reasoning. It is still the holy grail for researchers working in the AI field.

What is a Neural Network?

An artificial neural network (ANN) is a class of ML algorithms and data structures (or models for short) so called because it was inspired by the structure of biological neural tissue. But this doesn’t completely mimic all the biological mechanisms behind it. Rather, ANNs are complicated mathematical functions that are based on ideas from living species biology.

When I read “the model has 2 billion parameters” what does this mean?

Neural networks are layered structures consisting of uniform units interconnected with each other in a network. The way these units are interconnected is called architecture. Each connection has an associated number called weight and the weights store information the model learns from data. So, when you read “the model has 2 billion parameters,” it means that there are 2 billion connections (and weights) in the model, and it roughly designates the information capacity of the neural network.

What does Deep Learning mean?

Neural networks have been studied since the 1980s but made a real impact when the computer games industry introduced cheap personal supercomputers known as graphical processing units (GPUs). Researchers adapted this hardware for the neural network training process and achieved impressive results. One of the first deep learning architectures, the convolutional neural network (CNN), was able to carry out sophisticated image recognition that was difficult with classical computer vision algorithms. Since then, ML with neural networks has been rebranded as deep learning, with “deep” referring to the complicated NN architectures the networks are able to explore.

Where can I get some more details on how this tech works?

I’d recommend videos by Grant Sanderson available on his animated math channel.

What does the Large Language Model mean?

To work with human language using computers, language must be defined mathematically. This approach should be sufficiently generic to include the distinctive features of every language. In 2003 researchers discovered how to represent language with neural networks and called it the neural probabilistic language model or LM for short. This works like predictive text in a mobile phone – given some initial sequence of words (or tokens), the model can predict the next possible words with their respective probabilities. Continuing this process using previously generated words as input (this is autoregression) – the model can generate text in the language for which it was trained.

When I read about language models, I often encounter the term “transformer”. What is this?

Representing sequences of items was a challenging problem for neural networks. There were several attempts to solve the problem (mostly around variations of recurrent neural networks), which yielded some important ideas (e.g., word embedding, encoder-decoder architecture, and attention mechanism). In 2017 a group of Google researchers proposed a new NN architecture that they called a transformer. It combined all these ideas with effective practical implementation. It was designed to solve the language translation problem (hence the name) but proved to be efficient for capturing the statistical properties of any sequence data.

Why everyone talks about OpenAI?

OpenAI experimented with transformers to build a neural probabilistic language model. The results of their experiments are called GPT (generative pre-trained transformer) models. Pre-trained means they were training the transformer NN on a large body of texts mined on the Internet and then taking its decoder part for language representation and text generation. There were several generations of GPTs:

  • GPT-1: an initial experimental model to validate the approach
  • GPT-2: demonstrated ability to generate coherent human language texts and zero-shot learning – the ability to generalize to domains for which it was never specifically trained (e.g., language translation and text summarization, to name a few)
  • GPT-3 was a scale-up of the architecture (1.5 billion parameters of the GPT-2 vs 175 billion of the largest GPT-3) and was trained on a larger and more variate body of text. Its most important feature is the ability to produce texts in a wide range of domains by just seeing only a few examples in the prompt (hence the term few short learning) without any special fine-tuning or pre-training.
  • GPT-4: an even larger model (the exact characteristics are not disclosed), larger training datasets, and multimodality (text is augmented with image data).

Given the enormous number of parameters GPT models have (in fact, you need a huge computational cluster with hundreds to thousands of GPUs to train and serve these models), they were called Large Language Models (LLMs).

What’s the difference between GPT-3 and ChatGPT

The original GPT-3 is still a word prediction engine and thus is mostly of interest to AI researchers and computational linguists. Given some initial seed or prompt, it can generate text infinitely, which makes little practical sense. The OpenAI team continued to experiment with the model, trying to fine-tune it to treat prompts as instructions to execute. They fed in a large dataset of human-curated dialogues and invented a new approach (RLHF – reinforcement learning from human feedback) to significantly speed up this process with another neural network as a validator agent (typical in AI research). They released a model called InstructGPT as an MVP based on a smaller GPT-3 version and in November 2022 released a full-featured version called ChatGPT. With its simple chatbot and web UI, it changed the IT world.

What is the language model alignment problem?

Given that LLMs are just sophisticated statistical machines, the generation process could go in an unexpected and unpleasant direction. This type of result is sometimes called an AI hallucination, but from the algorithmic perspective, it is still valid, though unexpected, by human users.

Raw LLMs require treatment and additional fine-tuning with human validators and RLHF, as previously mentioned. This is to align LLMs with human expectations, and not surprisingly the process itself is called alignment. This is a long and tedious procedure with considerable human work involved; this could be considered LLM quality assurance. The alignment of the models is what distinguishes OpenAI/Microsoft ChatGPT and GPT-4 from their open-source counterparts.

Why there is a movement to stop the further development of language models?

Neural networks are black boxes (a huge array of numbers with some structure on top). There are some methods to explore and debug their internals but the exceptional generalization qualities of GPTs remain unexplained. This is the main reason behind the ban movement – some researchers think we are playing with fire (science fiction gives us fascinating scenarios of AGI birth and technological singularity) before we get a better understanding of the processes underlying LLMs.

What are the practical use cases of LLMs?

The most popular include:

  • Large text summarization
  • Vice versa — generating text from summary
  • Text styling (mimicking an author or character)
  • Using it as a personal tutor
  • Solving math/science exercises
  • Answering questions on the text
  • Generating programming code from short descriptions

Are the GPTs the only LLMs available now?

GPTs are the most mature models with API access provided by OpenAI and Microsoft Azure OpenAI services (if you need a private subscription). But this is the frontier of AI and many interesting things have happened since the release of ChatGPT. Google has built its PaLM-2 model; Meta open-sourced their LLaMA models for researchers, which spurred lots of tweaks and enhancements (e.g., Alpaca from Stanford) and optimization (now you can run LLMs on your laptop and even smartphone).

Huggingface provides BLOOM and StarCoder and HuggingChat – which are completely open source, without the LLaMA research-only limitation. Databricks trained their own completely open-source Dolly model. Lmsys.org is offering its own Vicuna LLM. Nvidia’s deep learning research team is developing its Megatron-LM model. The GPT4All initiative is also worth mentioning.

However, all these open-source alternatives are still behind OpenAI’s major tech (especially in the alignment perspective) but the gap is rapidly closing.

How can I use this technology?

The easiest way is to use OpenAI public service or their platform API playground, which offers lower-level access to the models and more control over network inner workings (specify system context, tune generation parameters, etc). But you should carefully review their service agreements since they use user interactions for additional model improvements and training. Alternatively, you can choose Microsoft Azure OpenAI services, which provide the same API and tools but with private model instances.

If you are more adventurous, you can try LLM models hosted by HuggingFace, but you’ll need to be more skilled with Python and data science tooling.
Denis Shipilov is experienced Solutions Architect with wide range of expertise from distributed systems design to the BigData and Data Science related projects.

More On This Topic

  • The Ultimate Guide To Different Word Embedding Techniques In NLP
  • The Ultimate Guide to Data Engineer Interviews
  • The Ultimate Guide to Acing Coding Interviews for Data Scientists
  • Baize: An Open-Source Chat Model (But Different?)
  • Meet Gorilla: UC Berkeley and Microsoft’s API-Augmented LLM Outperforms…
  • Bark: The Ultimate Audio Generation Model

Serving Generative AI just got a lot easier with OctoML’s OctoAI

octoai-template-quickstart

The OctoAI compute service offers templates to use familiar generative AI programs such as Stable Diffusion 2.1, but customers can also upload their own completely custom models.

The fervent interest in running artificial intelligence programs, especially of the generative variety, such as OpenAI's ChatGPT, is producing a cottage industry of services that can streamline the work of putting AI models into production.

Also: 92% of programmers are using AI tools, says GitHub developer survey

On Wednesday, startup OctoML, which cut its teeth improving AI performance across diverse computer chips and systems, unveiled a service to smooth the work of serving up predictions, the "inference" portion of AI, called OctoAI Compute Service. Developers who want to run large language models and the like can upload their model to the service, and OctoAI takes care of the rest.

That could, says the company, bring a lot more parties into AI serving, not just traditional AI engineers.

"The audience for our product is general application developers that want to add AI functionality to their applications, and add large language models for question answering and such, without having to actually put those models in production, which is still very, very hard," said OctoML co-founder and CEO Luis Ceze in an interview with ZDNET.

Generative AI, said Ceze, is even harder to serve up than some other forms of AI.

"These models are large, they use a lot of compute, it requires a lot of infrastructure work that we don't think developers should have to worry about," he said.

Also: OctoML CEO: MLOps needs to step aside for DevOps

OctoAI, said Ceze, is "basically a platform that typically would have been built by someone using a proprietary model" such as OpenAI for ChatGPT. "We have built a platform that is equivalent, but that lets you run open-source models.

"We want people to focus on building their apps, not building their infrastructure, to let people be more and more creative."

The service, which is now available for general access, has been in limited access with some early customers four about a month, said Ceze, "and the reception has exceeded everyone's expectations by a very wide margin."

Although the focus of OctoAI is inference, the program can perform some limited amount of what's known as "training," the initial part of AI where a neural net is developed by being given parameters for a goal to optimize.

Also: How ChatGPT can rewrite and improve your existing code

"We can do fine-tuning, but not training from scratch," said Ceze. Fine-tuning refers to follow-on work that happens after initial training but before inference, to adjust a neural network program to the particularities of a domain, for example.

"We want people to focus on building their apps not building their infrastructure, to let people be more and more creative," says OctoML co-founder and CEO Luis Ceze.

That emphasis has been a "deliberate choice" of OctoML's, said Ceze, given inference is the more frequent task in AI.

"Our work has been focused on inferencing because I've always had a strong conviction that for any successful model, you're going to do a lot more inferencing than training, and this platform is really about putting models into production, and in production, and for the lifetime of models, the majority of the cycles go to inference."

OctoML's team is well-versed in the intricacies of putting neural networks into production given their central role in developing the open-source project Apache TVM. Apache TVM is a software compiler that operates differently from other compilers. Instead of turning a program into typical chip instructions for a CPU or GPU, it studies the "graph" of compute operations in a neural net and figures out how best to map those operations to hardware based on dependencies between the operations.

Also: How to use ChatGPT to create an app

The OctoML technology that came out of Apache TVM, called the Octomizer, is "like a component of an MLOps, or DevOps flow," said Ceze.

The company "took everything we've learned" from Apache TVM and from building the Octomizer, said Ceze, "and produced a whole new product, OctoAI, that basically wraps all of these optimization services, and then full serving, and fine-tuning, a platform that enables users to take AI into production and even run it for them."

Also: AI and DevOps, combined, may help unclog developer creativity

The service "automatically accelerates the model," chooses the hardware, and "continually moves to what's the best way to run your model, in a very, very turnkey way."

Using CURL at the commend line is a simple way to get started with templated models.

The result for customers, said Ceze, is that the customer no longer needs to be explicitly an ML engineer; they can be a generalist software developer.

The service lets customers either bring their own model, or draw upon a "a set of curated, high-value models," including Stable Diffusion 2.1, Dolly 2, LLaMa, and Whisper for audio transcription.

Also: Meet the post-AI developer: More creative, more business-focused

"As soon as you log in, as a new user, you can play with the models right away, you can clone them and get started very easily." The company can help fine-tune such open-source programs to a customer's needs, "or you can come in with your fully-custom model."

So far, the company sees "a very even mix" between open-source models and their own custom models.

The initial launch partner for infrastructure for OctoAI is Amazon's AWS, which OctoML is reselling, but the company is also working with some customers who want to use capacity they are already procuring from public cloud.

Uploading a fully-custom model is another option.

For the sensitive matter of how to upload customer data to the cloud for, say, fine-tuning, "We started from the get-go with SOC2 compliance, so we have very strict guarantees of mixing user data with other users' data," said Ceze, referring to the SOC2 voluntary compliance standard developed by the American Institute of CPAs. "We have all that, but we have customers asking for more now for private deployments, where we actually carve out instances of the system and run it in their own VPC [virtual private cloud.]"

"That is not being launched right away, but it's in the very, very short-term roadmap because customers are asking for it."

Also: Low-code platforms mean anyone can be a developer — and maybe a data scientist, too

OctoML goes up against parties large and small who also sense the tremendous opportunity in the surge in interest to serve up predictions with AI programs since ChatGPT.

MosaicML, another young startup with deep technical expertise, in May introduced MosaicML Inference, deploying large models. And Google Cloud last month unveiled G2, which it says is "purpose-built for large inference AI workloads like generative AI."

Ceze said there are several things to differentiate OctoAI from competitors. "One of the things that we offer that is unique is that we are focused on fine-tuning and inference," rather than "a product that has inference merely as a component." OctoAI, he said, also distinguishes itself by being "purpose-built for developers to very quickly build applications" rather than having to be AI engineers.

Also: AI design changes on the horizon from open-source Apache TVM and OctoML

"Building an inference focus is really hard," observed Ceze, "you have very strict requirements" including "very strict uptime requirements." When running a training session, "if you have faults you can recover, but in inference, the one request you miss could make a user upset."

"I can imagine people training the models with MosaicML and then bringing them to OctoML to serve up," added Ceze.

OctoML is not yet publishing pricing, he said. The service initially will allow prospective customers to use it for free, from the sign-up page, with an initial allotment of free compute time, and "then pricing is going to be very transparent in terms of here is what it costs for you to use compute, and here's how much performance we're giving you, so that it becomes very, very clear, not just the compute time, and you're getting more work out of the compute that you're paying for." That will be priced, ultimately, on a consumption basis, he said, where customers pay only for what they use.

Also: The 5 biggest risks of generative AI, according to an expert

There's a distinct possibility that a lot of AI model serving could move from being an exotic, premium service currently to being a commodity.

"I actually think that AI functionality is likely to be commoditized," said Ceze. "It's going to be exciting for users, just like having spell checking or grammar checking, just imagine every text interface having generative AI."

At the same time, "the larger models will still be able to do some things the commodity models can't," he said.

Scale means that complexity of inference will be a challenging problem to solve for some time, he said. Large language models of 70 billion or more parameters generate a hundred "giga-ops," one hundred billion operations per second, for every word that they generate, noted Ceze.

Also: This new technology could blow away GPT-4 and everything like it

"This is many orders of magnitude beyond any workload that's ever run," he added, "And it requires a lot of work to make a platform scale to a global level.

"And so I think it's going to bring business to a lot of folks, even if it's commoditized, because we are going to have to squeeze performance and compute capabilities from everything we have in the cloud and GPUs and CPUs and stuff on the edge."

Artificial Intelligence

How to write better ChatGPT prompts (and this applies to most other text-based AIs, too)

sample-image-16-9-red.jpg

GIGO. It's an acronym dating back to 1957 when the US Army's William Mellin explained to a newspaper reporter that computers were limited by their input. It means "Garbage In, Garbage Out" and it applies as much to today's incredible generative AI tools as it did back in 1957 to those clunky, old tube-based computing devices.

In the case of an AI like ChatGPT, there's a lot of opportunity for both garbage in and garbage out. Keep in mind that we really don't have any idea what's been fed into it as part of its training. We know that its perception of facts is so fluid, it's practically qualified to be an American politician. In fact, OpenAI (the makers of ChatGPT) co-founder John Schulman says, "Our biggest concern was around factuality because the model likes to fabricate things."

Also: GPT-3.5 vs GPT-4: Is ChatGPT Plus worth its subscription fee?

Keep that in mind, because no matter how good your prompts are, there's always the possibility that the AI will simply make stuff up. That said, there's a lot you can do when crafting prompts to ensure the best possible outcome. That's what we'll be exploring in this how-to.

How to create effective ChatGPT prompts

To avoid the GIGO factor, your best bet is to write prompts that encourage the LLM (large language model) within ChatGPT to provide the best possible answers.

In fact, writing effective prompts has become its own highly-paid discipline, "prompt engineering." ZDNET's Sabrina Ortiz wrote an article entitled, "Do you like asking ChatGPT questions? You could get paid (a lot) for it," that explored this new employment category.

Also: Need an AI essay writer? Here's how ChatGPT (and other chatbots) can help

So, pay attention to the rest of this article. Who knows? Reading it could help you build the skills to become one of those highly-paid prompt engineers. Apparently, these gigs can pay from $175,000 to $335,000 per year. Hmm… See ya!

I am planning to travel to Spain in a few months and would like to learn some basic Spanish to help me communicate with local residents. I am looking for online resources that are suitable for beginners and provide a structured and comprehensive approach to learning the language. Can you recommend some online resources for learning Spanish as a beginner?

In this case, rather than just asking about learning resources, the context helps focus the AI on learning how to communicate on the ground with local residents. Here's another example:

I am a business owner interested in exploring how blockchain technology can be used to improve supply chain efficiency and transparency. I am looking for a clear and concise explanation of the technology and examples of how it has been used in the context of supply chain management. Can you explain the concept of blockchain technology and its potential applications in supply chain management?

In this example, rather than just asking for information on blockchain and how it works, the focus is specifically on blockchain for supply chain efficiency and how it might be used in a real-world scenario. Go ahead and feed that prompt into ChatGPT. Its answer is very interesting.

Also: Leadership alert: The dust will never settle and generative AI can help

I'll give you one more example that's far more in-depth. It shows how to construct a detailed prompt. One note: I limit the answer to 500 words because ChatGPT seems to break somewhere between 500 and 700 words, leaving the stories in mid-sentence and not resuming properly when asked to continue. I'm hoping future versions can provide more extensive answers, because premises like the one I'm about to share seem to generate some really fun story beginnings.

Write a short story for me, no more than 500 words.

The story takes place in 2339, in Boston. The entire story takes place inside a Victorian-style bookstore that wouldn't be out of place in Diagon Alley. Inside the store are the following characters, all human:

The proprietor: make this person interesting and a bit unusual, give them a name and at least one skill or characteristic that influences their backstory and possibly influences the entire short story.

The helper: this is a clerk in the store. His name is Todd.

The customer and his friend: Two customers came into the store together, Jackson and Ophelia. Jackson is dressed as if he's going to a Steampunk convention, while Ophelia is clearing coming home from her day working in a professional office.

Another customer is Evangeline, a regular customer in the store, in her mid-40s. Yet another customer is Archibald, a man who could be anywhere for 40 to 70 years old. He as a mysterious air about himself and seems both somewhat grandiose and secretive. There is something about Archibald that makes the others uncomfortable.

A typical concept in retail sales is that there's always more inventory "in the back," where there's a storeroom for additional goods that might not be shown on the shelves where customers browse. The premise of this story is that there is something very unusual about this store's "in the back."

Put it all together and tell something compelling and fun.

You can see how more detail provides more the AI can work with. First, feed "Write me a story about a bookstore" into ChatGPT and see what it gives you. Then feed in the above prompt and you'll see the difference.

The AI's response using the product manager identity included this sentence: I can confidently say that this is one of the most innovative and revolutionary products in the smart home industry.

The AI's response using the caregiver identity included this sentence: The device's ability to set reminders and alarms can be particularly helpful for elderly individuals who may have trouble remembering to take their medication or attend appointments.

Also: 5 ways to explore the use of generative AI at work

And the AI's response using the journalist identity included this sentence: From a journalistic perspective, the Echo has made headlines due to privacy concerns surrounding the collection and storage of user data.

You can see how different identities allow the AI to provide different perspectives as part of its response. I'll expand that idea a bit by showing you how to let the AI do a thought experiment. Let's look at some of the issues that went into the creation of something like an Alexa:

The year is 2012. Siri has been out for the iPhone for about a year, but nothing like an Alexa smart home device has been released. The scene is an Amazon board meeting where the Echo smart assistant based on Alexa has just been proposed.

Provide the arguments, pro and con, that board members at that meeting would have been likely to discuss as part of their process of deciding whether or not to approve spending to invest in developing the device.

Feel free to also include participation by engineering design experts and product champions, if that provides more comprehensive perspective.

Here's a quick tip. Making minor changes to your prompts can significantly change ChatGPT's response. For example, when I changed the phrase, "Provide the arguments, pro and con, that…" to "Provide the pro and con arguments as dialog, that…," ChatGPT rewrote its answer, switching from a list of enumerated pros and cons to an actual dialog between participants.

One of my favorite things to do is ask ChatGPT to justify its responses. I'll use phrases like "Why do you think that?" or "What evidence supports your answer?" Often, the AI will simply apologize for making stuff up and come back with a new answer. Other times, it might give you some useful information about its reasoning path. In any case, don't forget to apply the tips I provide for having ChatGPT cite sources.

If you have a fairly long conversation with ChatGPT, you'll start to notice that the AI loses the thread. This is clearly not unique just to AIs. If you have a fairly long conversation with most friends, family, co-workers, someone is bound to lose the thread. That said, when you're in a conversation with ChatGPT, you can use the same techniques you use with friends. Gently guide the AI back on track, and remind it what the topic is as well as what you're trying to explore.

More prompt-writing tips

  • Feel free to re-ask the question. ChatGPT will often change its answer with each ask.
  • Make small changes to your prompts to guide it into giving you a better answer.
  • ChatGPT will retain its awareness of previous conversations as long as the current page is open. If you leave that page, it will lose awareness. To be clear, ChatGPT will also sometimes lose the thread of the conversation "just because," so be aware you may need to start over from time to time.
  • Similarly, opening a new page will start the discussion with fresh responses.
  • Answers over about 500 words sometimes break down. Be sure to specify the length of the response you want.
  • You can correct and clarify prompts based on how the AI answered previously. If it's misinterpreting you, you may be able to just tell it what it missed and continue on.
  • Rephrase questions if it doesn't want to answer what you're asking. Use personas to elicit answers that it might not otherwise want to give.
  • If you want sources cited, tell it to support or justify its answers.
  • Keep experimenting.
  • Sometimes ChatGPT just fails. Keep trying, but also be willing to give up and move on to other tools. It's not perfect…yet.

What type of prompts work best with ChatGPT?

Part of what makes ChatGPT so compelling is you can ask it almost anything. That said, keep in mind that it's designed to provide written answers. If you want a list of websites, you're better off talking to Google.

Also: 92% of programmers are using AI tools, says GitHub developer survey

If you want some form of computation, talk to Wolfram Alpha. Give ChatGPT open-ended prompts, encourage creativity (yeah, and we thought creativity would be strictly the purview of humans), don't be afraid to share personal experiences or emotions, and remember that the AI's knowledge ends in 2021.

What do I do if ChatGPT refuses to answer or I don't like its answer?

There are some guardrails built into ChatGPT. It tends to close down if you ask it political questions, for example. That's what's built into the system. While you might be able to tease out an answer, it's probably not going to provide great value. That said, feel free to keep asking it questions with different phrasing or perspectives.

Also: How does ChatGPT actually work?

You can ask as many questions as you want (although it did once tell me I asked it too many questions and suggested I come back later). Yes, in fact, it basically said, "Go away kid, you're bothering me." To be fair, after hours of questions, I probably was. This thing can be as, or more, addictive than watching puppy videos on YouTube.

You can follow my day-to-day project updates on social media. Be sure to follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

More on AI tools

MeitY and Google to Collaborate on Cyber Safety 

The Ministry of Electronics and Information Technology (Meity) has announced that it will collaborate with Google to create awareness about cyber safety among Indian citizens.

We’re joining hands 🤝 with @GoI_MeitY on their @SSOIndia campaign – 📣 to raise awareness about online safety & security.
Stay tuned for #HitPause, a @YouTubeIndia series ⏸ that will help people evaluate and share online information responsibly.@mygovindia

— Google India (@GoogleIndia) June 15, 2023

Google will work with MeitY to enhance its “Stay Safe Online” campaign through various initiatives. These initiatives encompass literacy programmes focused on cyber safety, the creation of multilingual digital content related to online safety, and the development of educational material on navigating information online.

Google will introduce a series of videos featuring popular YouTube creators with the aim of educating users on responsible information sharing online. This series will delve into specific skills that can help individuals better assess the quality and reliability of information found on the internet.

“India is taking big strides towards its goal of becoming a trillion dollar digital economy. With over 700 million Indians using the internet, and the country driving transformative innovations with the power of digital technologies, it is important to ensure that the internet is inclusive and safe for all,” Sanjay Gupta, Country Head and Vice President, Google India, said.

The post MeitY and Google to Collaborate on Cyber Safety appeared first on Analytics India Magazine.

How do different personas in an organization see data quality?

Document control and management concept, Businessman review proc

Data Engineers:

We look into Data Engineering, which combines three core practices around Data Management, Software Engineering, and I&O. This focuses primarily on reconstituting data into usable consumer forms by building and operationalizing data pipelines across various data and analytics platforms.

Data Engineers, aka producers of the data, must be robust across data modeling, pipeline development, and software engineering and ensure that data always is reliable and made available per business service level agreement. This commitment requires executing a variety of checks across a large amount of data at speed and preventing any insufficient data from reaching downstream.

Due to the scale of data, this engineer-focused validation is wide and shallow and includes validation at the surface or metadata levels of data. These verification methods spread from API checks preventing issues before data lands (Shift Left) to detecting problems at the warehouse or lake house using anomaly monitoring (Shift Right). Both of these approaches revolve around metadata or custom-level SQL checks using observability to prevent or detect unknown issues before consumers use the data or meet agreed-upon Data Contracts/ SLAs with consumers/internal/external stakeholders.

Data Scientists or Data Analysts:

While in the world of Data Scientists or Data Analysts aka consumers of the data, the most common statement one would hear is “Garbage in, garbage out”. In other words, the model or report developed is only as good as the data that is being fed in. Here the data quality assessments are defined primarily by the use case and vary for each model and report generated.

The data quality checks performed are more business-centric and have to be measured continuously to manage, and resolve any known issues using the data context. Primarily, the consumers of the data or downstream applications want to ensure that the data they use for building reports and analytical models are business fit.

This requires performing narrow and deep business checks with aggregate analysis, deterministic rules, statistical measures, data accuracy, integrity, custom checks, and other dimensions of data quality that are necessarily not the focus of upstream or producers of data.

E.g., a data scientist building the next big model for optimizing route planning for the distribution of materials produced may be interested in certain characteristics of the data that may not be critical for a marketing function that is focused on targeting the younger generation demographic for new revenue.

This evaluation and need for data quality change from time to time and also dictates the need for decentralized data ownership or evaluation of data quality in their context.

Data Stewards:

Now, let’s not forget about the Data Stewards. This role is very confusing in the world of the modern data landscape and is even turned more confusing with decades of broken promises from Data Governance platforms. The word “Stewardship” means “supervising or taking care of something” and in this case, yes even in decentralized data ownership, there are a set of business terms or critical data elements (business terms) that needs to be maintained consistently to maintain business integrity e.g., Bank Routing Number, SSN, Customer ID.

This requires consistent data quality checks applied across the organization’s varying data assets based on business context (semantics) irrespective of where and how it is defined in the technical metadata. These checks require the automated discovery of semantics and a consistent measure of quality checks is a must and a need for “Agile” Data Stewardship or Federated teams.

Data Leaders:

Last but not least, Data Leaders aka Business Stakeholders are focused on what could make their next big win and want to understand how they could use the data on hand to enable new or improvised strategies toward positive business outcomes. A Gartner study shows that by 2025, 80% of data and analytics governance initiatives shall focus on business, and through 2024, 50% of organizations will adopt modern DQ solutions to better support their digital business initiatives.

This clearly shows how business leaders are focused on identifying key business processes, their KPIs and KRIs, and underlying data or metric assets that have a direct linkage to organizational mission-critical priorities. Having baselined values of business KPIs/KRIs before starting DQ initiatives and measuring continuously shall help to have the leaders a top-to-bottom-down view and more importantly which domains, and/or applications need to improve data quality. This view of “Top-down” business quality checks is essential for assessing key performance indicators, and benchmark measures for data quality against any initiative’s outcomes. Also, Privacy, Security, and Compliance Leaders need at a minimum continuous validation and assurance depending on the classification or type of data to ensure 100% regulatory compliance.

Get Started today with DQLabs and explore our platform!

The ABCs of data management

Compliance rules Businesswoman working digital tablet Document M

Introduction

Have you ever had to rummage through your garage for that one tool, only to unearth it, oddly, in the kitchen drawer months later? Data is just like that tool—utterly fruitless if you can’t locate it when you need it most. That’s where the spotlight shines on our superstar—Data Management. So, fasten your seatbelts as we journey through the ABCs of data management.

What is data management?

Ladies and gentlemen, welcome to the world of Data Management – the belle of the digital ball, the driving force behind the business machine. In simple terms, Data Management is the practice of collecting, safeguarding, and leveraging data securely, efficiently, and cost-effectively. Imagine having a digital librarian, but instead of a stern “shush,” this one makes your data effortlessly accessible. Regardless of your organization’s size, this magic works wonders.

Components of data management

types-data-management
Source: https://blog.hubspot.com/website/data-management

Data Management isn’t a solitary figure; it’s an ensemble of components playing a beautiful symphony. First, we have Data Governance—the conductor—setting the pace, defining rules, and ensuring compliance. Next, there’s Data Quality, the refined soprano ensuring data is accurate, relevant, and above all, usable.

Moving on to Data Integration, our melodious tenor—uniting varied data sources, providing a holistic view of business operations. Lastly, we shouldn’t overlook Data Security, the steady bass line that safeguards your precious data from breaches and leaks. When all these components harmonize, the performance is simply captivating.

Why is data management crucial?

In the digital era we live in, data management is as essential as a well-brewed morning coffee. It brings order to chaos, transforming your business into a beacon in the vast data darkness. With robust data management, decision-making becomes more potent, efficiency surges, and compliance with governance and regulatory standards becomes a breeze. On the flip side, poor data management is like driving blind—no one can anticipate the dangers lurking ahead.

Understanding data management software

Enter Data Management Software—the unsung hero, your tireless digital butler. This incredible tool collects, validates, stores, protects, and processes data, ensuring the uninterrupted operation of your business. From SQL databases to Hadoop-based systems, the array of data management software ready to serve you is astonishingly diverse.

Final word

Data Management isn’t merely a trend; it’s the beacon of the future. And remember, data is as good as the ability to retrieve it. Embrace data management today, navigate your business with the precision of a master mariner, and set sail for uncharted success.

And that, dear reader, was the ABCs of Data Management—interesting, enlightening, and not as complex as you initially thought. Now, you’re ready to explore this data-driven universe with renewed vigor and a refined understanding. Steer your ship wisely.

Vanishing Gradient Problem: Causes, Consequences, and Solutions

Vanishing Gradient Problem: Causes, Consequences, and Solutions
Image by Editor

The sigmoid function is one of the most popular activations functions used for developing deep neural networks. The use of sigmoid function restricted the training of deep neural networks because it caused the vanishing gradient problem. This caused the neural network to learn at a slower pace or in some cases no learning at all. This blog post aims to describe the vanishing gradient problem and explain how use of the sigmoid function resulted in it.

Sigmoid function

Sigmoid functions are used frequently in neural networks to activate neurons. It is a logarithmic function with a characteristic S shape. The output value of the function is between 0 and 1. The sigmoid function is used for activating the output layers in binary classification problems. It is calculated as follows:

Vanishing Gradient Problem, Explained

On the graph below you can see a comparison between the sigmoid function itself and its derivative. First derivatives of sigmoid functions are bell curves with values ranging from 0 to 0.25.

Vanishing Gradient Problem, Explained

Our knowledge of how neural networks perform forward and backpropagation is essential to understanding the vanishing gradient problem.

Vanishing Gradient Problem, Explained

Forward Propagation

The basic structure of a neural network is an input layer, one or more hidden layers, and a single output layer. The weights of the network are randomly initialized during forward propagation. The input features are multiplied by the corresponding weights at each node of the hidden layer, and a bias is added to the net sum at each node. This value is then transformed into the output of the node using an activation function. To generate the output of the neural network, the hidden layer output is multiplied by the weights plus bias values, and the total is transformed using another activation function. This will be the predicted value of the neural network for a given input value.

Vanishing Gradient Problem, Explained

Back Propagation

As the network generates an output, the loss function(C) indicates how well it predicted the output. The network performs back propagation to minimize the loss. A back propagation method minimizes the loss function by adjusting the weights and biases of the neural network. In this method, the gradient of the loss function is calculated with respect to each weight in the network.

In back propagation, the new weight(wnew) of a node is calculated using the old weight(wold) and product of the learning rate(ƞ) and gradient of the loss function Vanishing Gradient Problem, Explained.

Vanishing Gradient Problem, Explained

With the chain rule of partial derivatives, we can represent gradient of the loss function as a product of gradients of all the activation functions of the nodes with respect to their weights.Therefore, the updated weights of nodes in the network depend on the gradients of the activation functions of each node.

For the nodes with sigmoid activation functions, we know that the partial derivative of the sigmoid function reaches a maximum value of 0.25. When there are more layers in the network, the value of the product of derivative decreases until at some point the partial derivative of the loss function approaches a value close to zero, and the partial derivative vanishes. We call this the vanishing gradient problem.

With shallow networks, sigmoid function can be used as the small value of gradient does not become an issue. When it comes to deep networks, the vanishing gradient could have a significant impact on performance. The weights of the network remain unchanged as the derivative vanishes. During back propagation, a neural network learns by updating its weights and biases to reduce the loss function. In a network with vanishing gradient, the weights cannot be updated, so the network cannot learn. The performance of the network will decrease as a result.

Method to overcome the problem

The vanishing gradient problem is caused by the derivative of the activation function used to create the neural network. The simplest solution to the problem is to replace the activation function of the network. Instead of sigmoid, use an activation function such as ReLU.

Rectified Linear Units (ReLU) are activation functions that generate a positive linear output when they are applied to positive input values. If the input is negative, the function will return zero.

Vanishing Gradient Problem, Explained

The derivative of a ReLU function is defined as 1 for inputs that are greater than zero and 0 for inputs that are negative. The graph shared below indicates the derivative of a ReLU function

Vanishing Gradient Problem, Explained

If the ReLU function is used for activation in a neural network in place of a sigmoid function, the value of the partial derivative of the loss function will be having values of 0 or 1 which prevents the gradient from vanishing. The use of ReLU function thus prevents the gradient from vanishing. The problem with the use of ReLU is when the gradient has a value of 0. In such cases, the node is considered as a dead node since the old and new values of the weights remain the same. This situation can be avoided by the use of a leaky ReLU function which prevents the gradient from falling to the zero value.

Another technique to avoid the vanishing gradient problem is weight initialization. This is the process of assigning initial values to the weights in the neural network so that during back propagation, the weights never vanish.

In conclusion, the vanishing gradient problem arises from the nature of the partial derivative of the activation function used to create the neural network. The problem can bevworse in deep neural networks using Sigmoid activation function. It can be significantly reduced by using activation functions like ReLU and leaky ReLU.

Tina Jacob is passionate about data science and believes life is about learning and growing no matter what it brings.

More On This Topic

  • Mastering Clustering with a Segmentation Problem
  • Genetic Programming in Python: The Knapsack Problem
  • DeepMind’s AlphaFold & the Protein Folding Problem
  • How to Select an Initial Model for your Data Science Problem
  • The Range of NLP Applications in the Real World: A Different Solution To…
  • 90% of Today's Code is Written to Prevent Failure, and That's a Problem