Nvidia Announces ‘1 Exaflops’ AI Supercomputer with 256 Grace-Hopper Superchips

Nvidia Announces ‘1 Exaflops’ AI Supercomputer with 256 Grace-Hopper Superchips May 29, 2023 by Tiffany Trader

A new system from Nvidia might live up to the moniker "AI supercomputer": the DGX GH200. Announced on Monday at Computex in Taipei, the newest addition to Nvidia’s burgeoning portfolio ties together 256 Grace-Hopper Superchips, connected by 36 NVLink Switches, to provide over 1 exaflops of FP8 AI performance (or nearly 9 petaflops of FP64 performance). The system further touts 144TB of unified memory, 900 GB/s of GPU-to-GPU bandwidth and 128 TB/s bisection bandwidth. Nvidia is readying the product for end-of-year availability.

“We’re building it now,” said Nvidia CEO Jensen Huang during the livestreamed Computex keynote. “Every component is in production.”

“We [designed] DGX GH200 as a new tool for the development of the next generation generative AI models and capabilities,” said Ian Buck, VP and general manager of hyperscale and HPC at Nvidia, in a press briefing held earlier.

Nvidia CEO Jensen Huang shows off the Grace Hopper Superchip live at Computex (May 29, Taipei)

Hyperscalers Google, Meta and Microsoft – not surprisingly – have already queued up to kick the proverbial tires. AWS is not on that early access list.

“[These hyperscalers] will be the first to get access to the DGX GH200 to understand the new capabilities of Grace Hopper, and the multi-node NVLink that allows all those GPUs to work together as one,” said Buck.

One of the system’s top attributes is its 144TB of addressable memory, a combination of the HBM memory of the Hopper GPUs and the LPDDR5X memory of all the Grace CPUs in the network.

Figure shows DGX memory over generations. The DGX GH200 includes the memory of the Arm “Neoverse V2” CPUs, each with 480GB of LPDDR5X memory, and the H100 GPUs, each with 96GB HBM3 memory for this variant. Source: Nvidia.

DGX President Charlie Boyle highlighted the significance of this leap in memory (see chart at right). “For some applications, one of our [customers’] historic issues has been ‘my GPU acceleration is fantastic, but my application working space has to fit into GPU memory.’ We have customers with massive applications that they need to accelerate that are much bigger than the 640 gigabytes of memory provided by the Nvidia DGX H100 [equipped with 2 Intel CPUs and 8 H100 GPUs].

“The foundation of the DGX GH200 is being able to connect both via hardware in our NVLink Switch system, and software with all of the CUDA primitives, all of our communication libraries, so that customers can operate that system, because there’s 256 discrete computers, there’s 256 operating systems running in there, but our software works together with all of that so that you can launch a single job on the entire memory space using all the GPU capacity and either get a job you couldn’t do previously done or highly accelerate [one you could],” said Boyle.

To illustrate the potential speedups, Nvidia shared the following internal benchmarking projections, showing improvements from 2.2x (for the 1T GPT3) all the way to 6.3x (for the 40TB Distributed Join).

Performance comparisons going from a DGX H100 cluster with InfiniBand to a DGX GH200 fully NVLinked system. For each workload, the number of GPUs is the same for grey and green columns. Source: Nvidia.

Boyle compared the introduction of the fully NVLink-connected DGX GH200 to the arrival of NVLink in 2016. “For NVLink, which debuted with Pascal, we said every system beyond this is going to have this technology in it, and future Nvidia’s systems, future DGX systems, will also have this external NVLink capability.”

Nvidia is presenting the DGX GH200 as a 256-GPU system, which is the fully-configured version. But customers can buy in at 32 nodes, 64 nodes or 128 nodes, and can upgrade at any point along the way. “If somebody started with a 32-, they can buy another 32-; all the switching is already there,” said Boyle. “You plug in a few cables, you’ve got 64- and up and up.”

The fully configured (256-GPU) Nvidia DGX GH200 has 96 L1 NVLink Switches and 36 L2 NVLink Switches. Source: Nvidia.

This DGX GH200 is Nvidia’s first multi-rack DGX system. Each rack houses 16 Grace Hopper GH200 nodes and 256 nodes fill up 16 racks, as shown in the full article-header image, here. Bigger systems are also possible using InfiniBand to connect multiple DGX GH200s.

In fact, Nvidia is building one such mega-system – its own DGX GH200-based AI supercomputer, named Helios (the name was predicted by HPCwire) – to advance research and development and power the training of very large AI models. The system connects four DGX GH200 systems – for a total of 1,024 Grace Hopper Superchips – using Nvidia’s Quantum-2 InfiniBand networking. Nvidia is planning to bring the system online by the end of the year. Helios will provide about 4 exaflops of AI performance (FP8), and while it’s not the intended use case, would deliver ~34.8 theoretical peak petaflops of traditional FP64 performance. That would put it in the running for a spot in the top decile of the Top500 list if Nvidia opts to submit to the bi-annual list.

No HGX version of the new DGX has been announced yet, but it sounds like one is in the works. Similar to the HGX H100 design that is the foundation of Nvidia’s DGX H100 and which Nvidia makes available to hyperscalers and other system partners to customize to their specifications, Nvidia indicated that the DGX GH200 would be likewise available in an “HGX” form but the company is not making a specific announcement at this time.

On that point, Buck said, “all of these hyperscalers have their own system designs and datacenter designs and further optimize and take what we build in DGX as a blueprint in general, to build their own servers and optimize further for their datacenter infrastructure,” adding, “the components and building blocks and pieces inside of DGX will be made available to these hyperscalers so that they can take it and expand on the design for their custom datacenters and server designs. That product is known as HGX.”

That said, Nvidia is announcing the modular-focused MGX server specification. MGX is “an open, flexible and forward-compatible system reference architecture for accelerated computing,” said Buck.

(Informally, the ‘D’ in DGX stands for datacenter, the ‘H’ for hyperscale, ‘C’ is for cloud, there’s an Omniverse-targeted OVX, and now ‘M’ for modular, but this usage is not officially sanctioned by Nvidia’s PR department).

The modular architecture design was created to standardize server designs – pertaining to the mechanical, thermal and power aspects of servers – which may be outfitted with GPUs, CPUs and DPUs from Nvidia and others (both x86 and Arm). “By standardizing, the different components become replaceable, interchangeable and future-proofed to forward Nvidia products and others. With the new MGX reference architecture, we see the possibility to create a new design in as little as two months at a fraction of the cost,” said Buck, who had cited that the design process as it exists currently can take as long as 18 months.

MGX will support the following form factors:

  • Chassis: 1U, 2U, 4U (air or liquid cooled)
  • GPUs: Full Nvidia GPU portfolio including the latest H100, L40, L4
  • CPUs: Nvidia Grace CPU Superchip, GH200 Grace Hopper Superchip, x86 CPUs
  • Networking: Nvidia BlueField-3 DPU, ConnectX-7 network adapters

Three MGX server designs. Source: Nvidia.

If you’re wondering how MGX is different from HGX, Nvidia provided this explanation: “MGX differs from Nvidia HGX in that it offers flexible, multi-generational compatibility with Nvidia products to ensure that system builders can reuse existing designs and easily adopt next-generation products without expensive redesigns. In contrast, HGX is based on an NVLink-connected multi-GPU baseboard tailored to scale to create the ultimate in AI and HPC systems.”

In other words, HGX is “just a baseboard,” while MGX is a full reference architecture.

MGX is compatible with the Open Compute Project and Electronic Industries Alliance server racks and is supported by Nvidia’s full software stack, including Nvidia AI Enterprise.

ASRock Rack, ASUS, GIGABYTE, Pegatron, QCT and Supermicro have begun to implement MGX into their product design process. Two such products were announced today with planned availability in August: QCT’s S74G-2U system is based on the Nvidia GH200 Grace Hopper Superchip, and Supermicro’s ARS-221GL-NR system employs the Nvidia Grace CPU Superchip.

Another launch partner, Softbank, is relying on MGX to craft custom servers that will be deployed in its hyperscale datacenters across Japan. The design that Softbank created using the blueprints will help them dynamically allocate GPU resources in multi-use settings, for example to support both generative AI and 5G workloads.

Nvidia also declared that the GH200 Grace Hopper Superchip is in full production. In total, Nvidia says it now has more than 400 system configurations based on its latest CPU and GPU architectures – including Nvidia Grace, Nvidia Hopper and Nvidia Ada Lovelace – targeting the demand for generative AI. The systems all align with Nvidia’s software stack, including Nvidia AI Enterprise, Omnivere and the RTX platform.

No external system wins have yet been announced for the DGX GH200, but several Grace Hopper Superchip systems have been announced previously, using the same GH200 Superchips. Alps, the new Swiss supercomputing infrastructure at CSCS, is slated to debut the hybrid Arm-GPU architecture, while the U.S. prepares for the arrival of its first Grace Hopper system, “Venado,” at Los Alamos National Laboratory. Grace Hopper Superchips will also power the new Shaheen III supercomputer at KAUST. All three supercomputers are being built by HPE and are exected to be fully operational and available to researchers next year.

Nvidia declined to answer a question about the power draw of the DGX GH200 system, and likewise didn’t provide pricing info, but noted that all DGX products are sold through partners who help set the final customer prices. Regarding TDP, if you take the DGX H100 as a proxy, and multiple the max power consumption of 10.2kW by 32 (to get to 256 GPUs), that comes out to 326.4kW. We’ll update when the actual power spec becomes available.

Related

About the author: Tiffany Trader

With over a decade’s experience covering the HPC space, Tiffany Trader is one of the preeminent voices reporting on advanced scale computing today.

How Media Leverage Innumeracy to Create Click-bait

How Media Leverage Innumeracy to Create Click-bait
A typical heat dome

I focus here on a typical example. The kind of topics you regularly find in news outlets. At the face of it, the title is not a lie. It is a true fact. But the result of cherry-picking or the choice of words. Many people with a statistical background would consider it as misleading at best. There may or may not be political motivation behind the article. But definitely, the main goal is click bait to generate impressions and advertising revenue.

On May 14, 2023, I read an article entitled “Northwest Heat Wave Could Smash Records In Washington, Oregon”. I explain why this is misleading: how it exploits the innumeracy of most readers. There were indeed pretty high temperatures, even government warnings about the risk of heat stroke and so on. Somewhat exaggerated, but the main issue is the use of words such “all time record”, even if indeed it was true.

Obviously temperatures are getting hotter in summer in Seattle. This is a reality. But it is much less dramatic than what the authors make it to be in that article. In short, the article is not a good example to illustrate the problem. The same applies to many similar articles popping up regularly, regardless of the location.

What the Article Says

According to the author, a​ few record highs were set on Friday, including in Portland, Oregon (90 degrees), and Olympia, Washington (85 degrees). This may very well be true. It does not sound that high, but for May and in Seattle, it could be a 30-year record. The author pointed that out. Since I don’t deny this statement, why do I claim it is misleading? I now explain what is problematic in the statement in question.

What is the Probability of an All-time Record?

To answer the question in relation to the article, you must re-phrase it. Indeed, the question is about the probability for a 30-year record occurring in a given year, if you look at 30 years worth of historical data. It does not matter which day, and this is the critical part missing in the article. The author picked up the day when it actually took place, but it could have been three weeks later or sooner. Thus turning an otherwise relatively rare event, into a world record. The goal is to get people to click to read the article. If telling the reader upfront about the actual probability for this to happen, no one would read it. The probability is a lot higher than most people think, and thus not as exceptional as the author makes it to be.

Mathematical Computation of the Odds

The probability that the record day of a specific year is also a 30-year record, over a 30-year time period, is 1/30. Because each year always has a day that is the record of the year, and there are 30 years. By chance, the probability that the yearly record is also a 30-year record, is thus 1/30. What about the probability for a specific day to be a 30-year record? Obviously, p = 1 / (30 x 365). Very small, and that is the author’s argument.

Put it differently, the probability of zero day that is a 30-year record, in a given year is p365. Thus, the probability of at least one such record in a given year is 1 – p365. That number turns out to be 3.28%. Again, about 1/30. I do not consider 3% to be such a small chance to qualify as “exceptional event” or “30-year record”.

Conclusions

By playing with words and cherry-picking, news outlets can regularly find events that they can market as incredibly rare. They do so by misrepresenting the laws of probability. And the public, due to innumeracy, actually believes in the exceptional rarity of the events discussed. Most likely, this is not done on purpose. The authors are just as ignorant as their public. Unfortunately, it can lead to some dramatization and eventually government policies that do not address a reality, but an artificial (exaggerated) reality: a perception. This is true with public health and in many other domains. Regardless of political orientation, the publishers make the same mistakes.

They present the information in a way that is not technically a lie, indeed an actual fact. But the words surrounding it is what I called distortion. The drive to generate clicks further fuel this trend. As well as competition: a newspaper failing to mention the “exceptional heat wave” in question, or mentioning the 3% chance instead a “once in 30 years”, would be erroneously perceived as ignorant.

On the other hand, posting by end of December that 2 days this year, and 3 days last year, in 5 separate events, where the top 5 records over a 30-year time period, would have a lot more weight. Saying that 3 days in a row are top records given that one of them is already a top record, also has little weight, due to very strong autocorrelation in daily temperatures. Yet the public tends to be impressed by statistics like these. In the end, widespread innumeracy is also what allows lotteries to be successful. Incidentally, I don’t play the lottery, I create ones: see here.

About the Author

vgr2-1

Vincent Granville is a pioneering data scientist and machine learning expert, founder of MLTechniques.com and co-founder of Data Science Central (acquired by TechTarget in 2020), former VC-funded executive, author and patent owner. Vincent’s past corporate experience includes Visa, Wells Fargo, eBay, NBC, Microsoft, CNET, InfoSpace. Vincent is also a former post-doc at Cambridge University, and the National Institute of Statistical Sciences (NISS).

Vincent published in Journal of Number Theory, Journal of the Royal Statistical Society (Series B), and IEEE Transactions on Pattern Analysis and Machine Intelligence. He is also the author of “Intuitive Machine Learning and Explainable AI”, available here. He lives in Washington state, and enjoys doing research on stochastic processes, dynamical systems, experimental math and probabilistic number theory.

Roll wants to recreate dolly shots and more using generative AI

Roll wants to recreate dolly shots and more using generative AI Kyle Wiggers 18 hours

Those familiar with Faizan Buzdar, who was until recently the VP of product management at Box, likely associate the entrepreneur with Convo, the digital workspace platform popular among newsrooms (including this one). But Buzdar, whose background is in electronics engineering, has long held a fascination with video and visual effects.

“A lifelong video and photography enthusiast, I’d been making videos on my own for years but noticed video production had largely remained manual with little innovation in recent decades, especially for time-consuming tasks like video editing,” Buzdar told me via email. “Meanwhile, I noticed that iPhone camera and sensor technology had step-function improvements over the past few years, becoming almost equivalent in image quality to DSLRs.”

So while at Box, Buzdar says that he decided to try combining video — an increasingly popular medium — with innovations in AI and machine learning to attempt to improve the video capturing and editing experience. Buzdar tapped Adeel Abbas, a video engineer who while at Twitter contributed to the infrastructure powering the site’s livestreaming features, alongside Saj Khan, Fahad Yaqub and fellow Box exec Michelle Oh to explore the frontiers of tech-accelerated video production.

Roll is the result. A new app for iOS, it delivers bokeh, multicam shots, motion graphics and — perhaps most intriguing to me — “AI-simulated” sliders, dollies and jibs.

Roll

Image Credits: Roll

“Our mission is to disrupt the world of high-quality video production, and become the new standard for video content creation,” Buzdar continued. “Creating great video is a massive upfront investment in gear, equipment, learning how to use that gear, software for editing — we’re getting rid of all of that.”

Roll, which is aimed at the “prosumer” market (think influencers and podcasters, but also businesses creating their own marketing material), consists of two products: the Roll iPhone app and web app. The iPhone app captures and records video and then automatically uploads it to Roll’s cloud for storage and processing. The web app, meanwhile, is where footage can be previewed, accessed, shared, downloaded and edited by one or a team of content creators.

Of course, video apps are a dime a dozen. So what makes Roll different? For one, the app’s aimed at use cases that most camera apps aren’t, Buzdar says — like remote video interviews, video podcasts and customer testimonials. While Zoom, Microsoft Teams and Google Meet fill the need to some degree, Buzdar argues that they’re not designed for “high-quality” video production.

Roll also employs a number of real-time effects to (ostensibly) offer a greater range of post-production choices than most video-capturing apps. For example, Roll records in the HEVC standard, delivering roughly twice the bitrate and higher image quality for the same file size. And Roll can record and process up to two camera shots — a wide-angle shot and close-up shot — at once, allowing users to create videos with effectively “multi-camera” perspectives.

Roll

The Roll editing interface. Image Credits: Roll

Granted, multicam isn’t particularly unique — roll’s far from the first app to offer it. But Buzdar says that where the magic lies is in the post-processing. Roll leverages generative AI to recreate rooms in 3D space so that content creators can move a video-game-like virtual camera around, simulating movements like panning from side to side with a dolly or crane.

“Today, generative AI is too often associated with creating fake content out of thin air,” Buzdar said. “That’s not our philosophy. We don’t generate fake pixels, people or scenes. We’re using generative AI purely as a tool for productivity — we want to democratize access to higher quality video production.”

Buzdar explained that Roll’s AI was trained to understand the 3D depth in a scene, using data to measure depth and shapes independent of the person sitting in the room. Roll started training its algorithms with open source datasets commonly used for benchmarking in academia, but then internally recorded over 22,000 video calls, creating its own rich database.

The results aren’t half bad — at least in the demo footage that Buzdar showed me. Some of Roll’s AI-generated pans broach the uncanny valley, the result of unnatural warping on objects in the background as the virtual camera swivels by. But in short scenes, the AI effects are convincing enough — and an eye-catching addition to what’d otherwise be a dull remote interview.

“We’ve researched this quite a bit, and we’ve not seen anyone use AI in the same way we are — pairing iPhone sensor data with large AI models in the cloud,” Buzdar said. “Our technology provides foundational capabilities to simulate any visual effects a user would want.”

Roll

Call recording with Roll. Image Credits: Roll

Any visual effects sounds like a bit of a stretch. But Roll has other, more realistic algorithmic tricks up its sleeve. As Roll records video, it gathers metadata for use later in the video production process, including recording and lighting conditions, the distance from the camera to the subject and the position of the subject’s face and body. The metadata’s used to automatically adjust the cameras and sensors on the phone as well as provide feedback and instruction for composition and lighting.

Similar to a few other “AI-enabled” mobile video editors on the market, Roll also taps the metadata to create a fully realized, multicamera reel in its editing cloud — no manual editing required. (Users can still change and adjust the camera angles or add camera movements and visual effects if they choose.) In the near future, Roll will be able to publish directly to social media, including TikTok, YouTube and Instagram — in both the appropriate resolution and aspect ratio.

“Today, video production requires many pieces of hardware and software to fully complete,” Buzdar said. “With every single step, when the video and audio file hops from one software to another, it loses context and just becomes a ‘dumb’ file that is passed around. We have fundamentally rebuilt the entire video production ‘stack’ from scratch. Cutting across traditional software boundaries, we’ve applied AI to deliver a transformative capture-to-publish experience that vertically integrates and automates the entire remote video production workflow.”

So how does Roll plan to make money? The company’s so far raised cash from traditional VC sources — Buzdar wouldn’t say where, exactly. But in terms of revenue generation, Buzdar hopes Roll will eventually grow to serve the needs of corporate organizations — specifically their in-house corporate marketing and video teams, who will pay some sort of fee for Roll’s services.

“Video production is ripe for disruption from the cloud,” Buzdar said. “Attributes like large file sizes, complex processing and the need for multiperson edits and review cycles make it the perfect candidate to have exponential benefits from cloud computing like scalable storage, AI, compute, and real-time sharing and collaboration.”

There’s surely truth to that. As for whether Roll will be the disruptor, time will tell.

Council Post: AI’s Evolutionary Journey – Model-Centric to Data-Centric to Decision-Centric

Model-centric and data-centric are two popular approaches in the field of artificial intelligence (AI) that emphasize the importance of either the model or the data in the development and performance of AI systems. Some examples in the model centric approach:

Convolutional neural networks (CNNs) for image recognition and recurrent neural networks (RNNs) for natural language processing are examples of deep learning models. These models are built with intricate architectures and several layers to learn and extract characteristics directly from the input.

GANs, or generative adversarial networks: A generator network and a discriminator network are the two main components of GANs. While the discriminator aims to discriminate between authentic and fraudulent samples, the generator makes artificial data samples. GANs concentrate on producing accurate data by harnessing the power of the model.

All the above training techniques and model hyper parameter selection for the above deep learning model essentially try to tweak the model to best-solve the task using a given dataset. Similarly, some examples for the data centric approach can be:

  • Transfer Learning: Pre-trained models that have learnt generic characteristics via training on big datasets are used in transfer learning. The requirement for substantial training on task-specific data is subsequently reduced since these models are then fine-tuned on specific tasks with less data.
  • Data Augmentation: To provide more training examples, data augmentation approaches entail adding transformations or alterations to existing data sets. For instance, flipping, rotating, or adding noise to photos might improve the variety of the training data for image classification.

Both approaches have their merits and trade-offs. In one hand, Data-centric AI can be advantageous when dealing with limited prior knowledge or rapidly changing domains. By focusing on data collection and curation, it aims to capture the diversity and complexity of real-world scenarios, allowing models to learn from the data directly. This approach can be more flexible and adaptive to changing environments. However, data-centric AI heavily relies on the availability of high-quality and representative datasets, which can be challenging to obtain.

On the other hand, Model-centric AI can be effective in situations where domain expertise and prior knowledge are essential, such as in complex problem domains like natural language processing or computer vision. It allows researchers to design specific architectures and algorithms tailored to the problem at hand, resulting in high accuracy and performance. However, model-centric AI often requires significant computational resources and extensive training time. A famous example of this model-centric AI approach, exemplified by large-scale language models like GPT-3, require significant computational resources and extensive training time due to their complexity and the vast amount of data they process. The training process involves pre-training on unlabeled text data and fine-tuning on specific tasks. The computational demands stem from the models’ large number of parameters, necessitating high-performance infrastructure.

The move towards decision centric AI

Model centric works when the data is clean, well curated etc. Before trying to improve any model the data quality has to be top notch. However, in the real world, data is not ideal. Hence, a decision centric approach is more practical.

In this sense, a decision-centric approach is crucial because it fosters creative problem-solving at those “moments of truth” while simultaneously enhancing operational effectiveness. To enable users to take the appropriate action at the appropriate moment, people, rules, data, and processes must all work together. Companies that want to react quickly and effectively to dynamic market developments must adopt a decision-centric strategy. It makes it possible for the organisation to swiftly adjust to changes in operational procedures and company strategy in addition to learning new things. As a result, it is an essential component of any toolkit for business agility. It promotes uniformity in decision-making across the board and offers crucial feedback loops. The framework offers a context for placing judgements into “business moments,” regardless of whether these decisions are backed by AI, machine learning, business rules, or any other technology technique.

Decision-centric AI represents the next stage in the evolution of AI, where the focus shifts from solely making predictions to actively making informed decisions. This approach involves integrating data analysis, machine learning, and optimization techniques to generate optimal decisions based on desired objectives and constraints. Decision-centric AI will enable organizations to make smarter, data-driven decisions that align with their goals and requirements.

By incorporating contextual information, business constraints, and user preferences, decision-centric AI systems will optimize decision-making processes across diverse domains such as healthcare, finance, supply chain management, and customer service. It will enhance transparency and explainability, allowing users to understand the reasoning behind the decisions made by AI systems.

This will build trust and facilitate the acceptance and adoption of AI technologies. Ultimately, decision-centric AI has the potential to revolutionize how organizations operate, enabling them to make more accurate, efficient, and impactful decisions, thereby driving innovation, productivity, and competitiveness. It is also important to remember that decision centric AI is about augmenting human decision-making rather than replacing it.

The evolution in AI system development and application from model-centric to data-centric to decision-centric methods symbolises this development. At first, the emphasis was on creating intricate models that could learn directly from data. However, the shortcomings of this strategy prompted a move towards a focus on data gathering and curation. The value of high-quality datasets was acknowledged by the data-centric approach. The decision-centric approach was developed with the aim of generating favourable results based on facts in real time and being able to justify those judgements. Unknown circumstances are a daily problem for businesses, but by adding quantified pattern data and adaptable scenarios to the mix, choices may be made more quickly, consistently, and confidently.

The shift from model-centric AI to data-centric AI has allowed organizations to leverage vast amounts of data for improved performance and adaptability. However, the evolution does not stop there. The emerging trend of decision-centric AI holds great promise in transforming the way we make decisions by integrating data analysis, machine learning, and optimization techniques. Ultimately, decision-centric AI has the potential to revolutionize organizations; operations, driving innovation, productivity, and competitiveness through more accurate and impactful decision-making processes. The future of AI lies in the integration of data, models, and decisions, paving the way for a more intelligent and informed world.

This article is written by a member of the AIM Leaders Council. AIM Leaders Council is an invitation-only forum of senior executives in the Data Science and Analytics industry. To check if you are eligible for a membership, please fill out the form here.

The post Council Post: AI’s Evolutionary Journey – Model-Centric to Data-Centric to Decision-Centric appeared first on Analytics India Magazine.

NVIDIA announces new class of supercomputer and other AI-focused data center services

August 9, 2019 Santa Clara / CA / USA - The NVIDIA logo and symbol displayed on the facade of one of their office buildings located in the Company's campus in Silicon Valley
Image: Sundry Photography/Adobe Stock

On May 28 at the COMPUTEX conference in Taipei, NVIDIA announced a host of new hardware and networking tools, many focused around enabling artificial intelligence. The new lineup includes the 1-exaflop supercomputer, the DGX GH200 class; over 100 system configuration options designed to help companies host AI and high-performance computing needs; a modular reference architecture for accelerated servers; and a cloud networking platform built around Ethernet-based AI clouds.

The announcements — and the first public talk co-founder and CEO Jensen Huang has given since the start of the COVID-19 pandemic — helped propel NVIDIA in sight of the coveted $1 trillion market capitalization.

Jump to:

  • What makes the DGX GH200 for AI supercomputers different?
  • NVIDIA’s new AI enterprise tools are powered by supercomputing
  • Faster networking for AI in the cloud
  • MGX Server Specification coming soon
  • Other news from NVIDIA at COMPUTEX
  • Alternatives to NVIDIA’s supercomputing chips

What makes the DGX GH200 for AI supercomputers different?

NVIDIA’s new class of AI supercomputers take advantage of the GH200 Grace Hopper Superchips, and the NVIDIA NVLink Switch System interconnect to run generative AI language applications, recommender systems and data analytics workloads (Figure A). It’s the first product to use both the high-performance chips and the novel interconnect.

Figure A

A closeup of the Grace Hopper chip from NVIDIA
The Grace Hopper chip is the backbone of many of NVIDIA’s supercomputing and artificial intelligence products and services. Image: NVIDIA

NVIDIA will offer the DGX GH200 to Google Cloud, Meta and Microsoft first. Next, it plans to offer the DGX GH200 design as a blueprint to cloud service providers and other hyperscalers. It is expected to be available by the end of 2023.

The DGX GH200 is intended to let organizations run AI from their own data centers. 256 GH200 superchips in each unit provide 1 exaflop of performance and 144 terabytes of shared memory.

Specifically, NVIDIA explained the NVLink Switch System enables the GH200 chips to bypass a conventional CPU-to-GPU PCIe connection, increasing the bandwidth while reducing power consumption.

Mark Lohmeyer, vice president of compute at Google Cloud, pointed out in an NVIDIA press release that the new Hopper chips and NVLink Switch System can “address key bottlenecks in large-scale AI.”

“Training large AI models is traditionally a resource- and time-intensive task,” said Girish Bablani, corporate vice president of Azure infrastructure at Microsoft, in the NVIDIA press release. “The potential for DGX GH200 to work with terabyte-sized datasets would allow developers to conduct advanced research at a larger scale and accelerated speeds.”

NVIDIA will also keep some supercomputing capability for itself; the company plans to work on its own supercomputer called Helios, powered by four DGX GH200 systems.

NVIDIA’s new AI enterprise tools are powered by supercomputing

Another new service, the NVIDIA AI Enterprise library, is designed to help organizations access the software layer of the new AI offerings. It includes more than 100 frameworks, pretrained models and development tools. They are appropriate for the development and deployment of production AI including generative AI, computer vision, speech AI and others.

On-demand support from NVIDIA AI experts will be available to help with deploying and scaling AI projects. It can help deploy AI on data center platforms from VMware and Red Hat or on NVIDIA-Certified Systems.

SEE: These are the top-performing supercomputers in the world.

Faster networking for AI in the cloud

NVIDIA wants to help speed up Ethernet-based AI clouds with the accelerated networking platform Spectrum-X (Figure B).

Figure B

Components of the Spectrum-X accelerated networking platform.
Components of the Spectrum-X accelerated networking platform. Image: NVIDIA

“NVIDIA Spectrum-X is a new class of Ethernet networking that removes barriers for next-generation AI workloads that have the potential to transform entire industries,” said Gilad Shainer, senior vice president of networking at NVIDIA, in a press release.

Spectrum-X can support AI clouds with 256 200Gbps ports connected by a single switch or 16,000 ports in a two-tier spine-leaf topology.

Spectrum-X does so by utilizing Spectrum-4, a 51Tbps Ethernet switch built specifically for AI networks. Advanced RoCE extensions bringing together the Spectrum-4 switches, BlueField-3 DPUs and NVIDIA LinkX optics create an end-to-end 400GbE network optimized for AI clouds, NVIDIA said.

Spectrum-X and its related products (Spectrum-4 switches, BlueField-3 DPUs and 400G LinkX optics) are available now, including ecosystem integration with Dell Technologies, Lenovo and Supermicro.

MGX Server Specification coming soon

In more news regarding accelerated performance in data centers, NVIDIA has released the MGX server specification. It is a modular reference architecture for system manufacturers working on AI and high-performance computing.

“We created MGX to help organizations bootstrap enterprise AI,” said Kaustubh Sanghani, vice president of GPU products at NVIDIA.

Manufacturers will be able to specify their GPU, DPU and CPU preferences within the initial, basic system architecture. MGX is compatible with current and future NVIDIA server form factors, including 1U, 2U, and 4U (air or liquid cooled).

SoftBank is now working on building a network of data centers in Japan which will use the GH200 Superchips and MGX systems for5G services and generative AI applications.

QCT and Supermicro have adopted MGX and will have it on the market in August.

Other news from NVIDIA at COMPUTEX

NVIDIA announced a variety of other new products and services based around running and using artificial intelligence:

  • WPP and NVIDIA Omniverse came together to announce a new engine for marketing. The content engine will be able to generate video and images for advertising.
  • A smart manufacturing platform, Metropolis for Factories, can create and manage custom quality-control systems.
  • The Avatar Cloud Engine (ACE) for Games is a foundry service for video game developers. It enables animated characters to call on AI for speech generation and animation.

Alternatives to NVIDIA’s supercomputing chips

There aren’t many companies or customers aiming for the AI and supercomputing speeds NVIDIA’s Grace Hopper chips enable. NVIDIA’s major rival is AMD, which produces the Instinct MI300. This chip includes both CPU and GPU cores, and is expected to run the 2 exaflop El Capitan supercomputer.

Intel offered the Falcon Shores chip, but it recently announced that this would not be coming out with both a CPU and GPU. Instead, it has changed the roadmap to focus on AI and high-powered computing, but not include CPU cores.

Person using a laptop computer.

Subscribe to the Cloud Insider Newsletter

This is your go-to resource for the latest news and tips on the following topics and more, XaaS, AWS, Microsoft Azure, DevOps, virtualization, the hybrid cloud, and cloud security.

Delivered Mondays and Wednesdays Sign up today

Using ChatGPT for job applications could backfire on you, according to this study

ChatGPT on mobile

Generative AI tools have made many tedious tasks easier, including applying for jobs. AI chatbots like ChatGPT can generate resumes, cover letters, essays, and more within seconds.

However, this study shows that just because you can may not mean you should.

Also: These are my 5 favorite AI tools for work

The Talent Cloud company, iCIMS, released its "Class of 2023" report, which analyzes college graduates' career expectations and compares them to those of HR professionals and employers.

The rise of ChatGPT's popularity occurred as these grads were in their senior year. As a result, the way they approached the job search was affected by it.

Also: ChatGPT vs Bing Chat vs Google Bard: Which is the best AI chatbot?

Out of the 1,000 U.S. college seniors surveyed, nearly half (47%) said they were interested in using AI chatbots to write their resumes or cover letters, and 25% of Gen Z said they had already used AI for those purposes.

"Generative AI bots (like ChatGPT) have the potential to help both candidates and employers be more productive and communicate better," says Al Smith, chief technology officer, iCIMS in the report.

However, nearly 40% of the 500 HR professionals surveyed said that the use of AI on job applications would be a definite dealbreaker.

Also: How does ChatGPT actually work?

It is important to note that there are currently no reliable AI detectors. As long as you make sure to edit the AI-generated text so that it is accurate and reflects who you are, an HR professional shouldn't be able to tell the difference between an AI-assisted application and a human one.

More on AI tools

You can now send Snapchat’s AI chatbot photos and get a response (warning, it’s creepy)

Snapchat My AI Snaps announcement

As if chatting with Snapchat's My AI chatbot wasn't creepy enough, now you can send it photos through "My AI Snaps."

On Wednesday, Snapchat announced that Snapchat+ users are able to Snap the chatbot photos and receive unique, generative AI messages from the chatbot in return.

Also: Snapchat's AI-powered chatbot has arrived. Here are my thoughts

For example, Snapchat says users can send it a photo of their grocery haul and possibly get a recipe for the ingredients in the picture.

The company also discloses that, like all messages with My AI, the Snaps will be stored and could be used to improve the model.

While it is common for most generative AI models to store data to improve their performance, this particular use case raises concerns due to the storage of photos, which are more personal in nature than text conversations.

Also: LinkedIn is using AI to make it easier for recruiters to reach you

Although the feature says it is limited to Snapchat+ users, I wanted to see if I could test the feature without a Plus subscription. To my surprise, I was able to both send "My AI" a photo and get a response.

I sent My AI and Snap of a glass of water on my work desk. The chatbot immediately opened the Snap and started a conversation with me about drinking water.

The initial response was, "Looks like you are staying hydrated!" and the conversation eventually ended with it saying that it was my "virtual friend".

Also: LinkedIn is using AI to make it easier for recruiters to reach you

Snapchat's initial My AI release was received poorly by the public, and people even took to other social media platforms like TikTok and Twitter to express their frustrations with the app and the chatbot.

The implementation of this feature is likely Snapchat's attempt at redemption from that. With a daily active user base of 383 million, surely some of them will find this less creepy than I do, right?

Artificial Intelligence

AI has created areas so grey, you could write a song about it

sound waves abstract

These days, it feels like I needs to put a disclaimer at the top of my stories so no one assumes any part of the content is generated by artificial intelligence (AI). However, only human brains were involved in the making of this piece — one writer's brain and two editors' brains, to be precise, and none have been implanted with a chip.

Unfortunately, we're now at a stage where we can no longer easily distinguish between humans and robots. We've not reached the all-controlling level of Skynet yet, but the power of AI today marks a pivotal point for a technology that had been chugging along — mostly in the backend — for years and is now finally more accessible, and understood, by the general public.

Also: AI could automate 25% of all jobs. Here's which are most (and least) at risk

This emergence is all thanks to generative AI platform, ChatGPT, which has fascinated many of us with its ability to mimic humans and assist with various tasks, including software coding, creating travel itineraries, and composing email messages and essays. Venture beyond ChatGPT and you'll find other AI-powered applications that can produce images and songs "inspired" by popular artists and writers.

And herein lies the crux of a debate over where the lines should be drawn on how AI is used in some industries.

For me, in my work as a journalist, the lines are crystal clear. Factual inaccuracy and plagiarism are big red flags. It is for these reasons that tools such as ChatGPT have absolutely no role to play in my craft, not even as a research assistant.

Also: How to use ChatGPT: Everything you need to know

I'm guessing lawyers probably share my concerns, particularly after one of their peers in New York was called out for citing judiciary decisions of past cases that never existed. Yep, he let ChatGPT do the research and it generated content based on false sources.

The lines, however, may not always be as clear.

AI increasingly is used to create music based on the styles of popular artists, but also produce songs that are "sung" by a voice that sounds very much like a specific pop star. Singapore-based singer Stefanie Sun, for instance, apparently recorded a cover of Avril Lavigne's Complicated — except she actually didn't.

To an untrained ear, the AI-generated voice sounds just like Sun, who has sold more than 30 million records since her debut in 2000. Her fans, though, say her AI counterpart is easily distinguishable because it lacks the emotive nuances of the singer.

That perception, however, could change, Sun herself has acknowledged. In a blog post last week, she joked that her AI persona is enjoying more fame now that her own heyday is over, and that it's impossible to compete with someone capable of releasing new albums in mere minutes.

Also: How I tricked ChatGPT into telling me lies

Sun says AI has got better at processing and piecing together information to form opinions and thoughts — something humans were once convinced could not be replicated. The singer adds that it may only be a matter of time before AI makes further advancements and is able to mimic human emotions.

"This new technology will be able to churn out exactly everything, everyone," Sun writes. "You are not special. You are already predictable and also, unfortunately, malleable."

The singer's label reportedly isn't considering legal action because there's currently a lack of regulation around generative AI.

While Sun sees her AI counterpart as a potential competitor, Canadian singer-songwriter Grimes is more open to the idea of music created using an AI version of her voice. That is, if anyone who does so shares the royalties 50/50. Grimes has invited immitators to register their music via her website, where she plans to make her vocal samples available to aid the AI process. "I think it's cool to be fused with a machine and I like the idea of open sourcing all art and killing copyright," Grimes tweets.

Others in her industry are less generous with the new revenue model.

Also: ChatGPT is more like an 'alien intelligence' than a human brain, says futurist

US rapper Ice Cube said in a podcast interview he would sue anyone who makes a song with his AI-generated voice as well as the platform that plays it.

His comments come on the heels of a song called Heart On My Sleeve, which was presumably created by AI and made to sound like rapper-singer-songwriter Drake and singer-songwriter The Weeknd. The two Canadians are known collaborators.

Heart On My Sleeve went viral on various platforms including TikTok and Spotify, before it was removed at the request of the singers' record label. Copies of it still are available on YouTube.

The source behind the song reportedly created it using AI models trained on the artists' works, styles, and voices.

There are lawyers who are better equipped — and articles featuring interviews with lawyers — that have already debated the potential legal issues around AI-generated songs like Heart On My Sleeve, so I'm not going to do that here. Suffice to say the song throws up a bunch of questions around fair use and misrepresentation, and some correlation to professional impressionists or impersonators and tribute acts.

What truly matters as AI becomes everything, everywhere

I am, however, keen to draw some parallels with how human artists and musicians find inspiration for their work. We often hear how great songwriters are influenced by those who came before them. Bruno Mars cites Elvis Presley and the Beach Boys among his music influences, while Billie Eilish points to the Beatles and Green Day.

These artists grew up listening to and learning from musicians, applying what they feel are most in sync with their own styles and creating their own art.

Also: AI can write your emails, reports, and essays. But can it express your emotions?

In some ways, that's exactly what large language models and generative AI tools such as ChatGPT do. They produce new works based on what they learn from past works. The only significant difference is that human minds are shaped and influenced by works we admire as we grow and learn, while AI models are not inherently partial and have the compute capacity to not be discriminatory over what they choose to learn as they grow.

So, assuming no copyright was breached and there is no misrepresentation, why should AI-generated content that draws inspiration from famous works be any different from human-generated content that also draws inspiration from famous works? And aren't most products based on basic foundational structures and best practices anyway?

That's roughly the argument that British singer-songwriter Ed Sheeran used in the lawsuit he won against Marvin Gaye's estate, in which he was found not liable for copyright infringement. Sheeran's lawyer Ilene Farkas told jurors that similarities in the chord progressions and rhythms used in Gaye's and Sheeran's songs in question were "the letters of the alphabet of music". "These are basic musical building blocks that songwriters now and forever must be free to use, or all of us who love music will be poorer for it," Farkas said.

Also: Just how big is this generative AI? Think internet-level disruption

Musician and YouTuber Rick Beato says it plainly: "You cannot copyright a chord progression."

So, where does that leave humans, as AI continues its growth trajectory and its use becomes pervasive? How can we differentiate ourselves when we have to compete against an entity with a greater capacity to process and learn?

I think we have to continue to innovate and be creative in how we apply our knowledge. We must put our own unique spin on top of these basic foundations and incorporate elements not commonly used by others.

Just like when the internet emerged and then became popular, we can't let access to new technologies like AI make us lazy. "Don't keep repeating things," Beato says.

I recently moderated a roundtable discussion when I cheekily said my questions for the participants were generated by my human brain, without the assistance of AI. "But why not?" a couple of the attendees asked.

My response to that question was a no-brainer (pun intended). A generative AI tool like ChatGPT could very well have come up with a list of brilliant questions based on the roundtable dialog, which ironically enough was on AI. However, it would unlikely be able to adapt and modify the questions in real time, as the conversation moved along.

Also: AI bots have been acing medical school exams

I always have a list of questions ready at the start of every discussion I moderate, but I'm constantly following up with new ones based on insights participants share as the roundtable conversation progresses. I tweak my questions along the way to adapt to the evolving discourse, which is often filled with references to local industry developments and personal anecdotes previously unshared.

All of these insights, including my cheesy sense of humor, can't be easily reproduced by an AI model — for now at least. And that's how I hope my knowledge and skills can retain some relevance in the AI era.

After all, there is tremendous potential for what AI could bring to healthcare, and there's even more urgency to address issues around AI ethics and data security — before it's too late.

Artificial Intelligence

Character.AI, the a16z-backed chatbot startup, tops 1.7M installs in first week

Character.AI, the a16z-backed chatbot startup, tops 1.7M installs in first week Sarah Perez @sarahintampa / 10 hours

Demand for AI via consumer mobile apps has been climbing, with market leader OpenAI’s ChatGPT mobile app topping half a million downloads in its first six days. Now, another AI app is touting its own launch success, as the a16z-backed Character.AI app is claiming to have pulled in over 1.7 million new installs in less than a week on the market. The AI app maker, which announced a whopping $150 million in Series A funding earlier this year, valuing its business at $1 billion, offers customizable AI companions with distinct personalities, as well as the ability for users to create their own characters.

While there are a number of these AI character generators available on today’s app stores, interest in Character.AI has a lot to do with its founders. The Palo Alto-based startup was created by Noam Shazeer and Daniel De Freitas, AI experts who previously led a team of researchers at Google that built LaMDA (Language Model for Dialogue Applications), a language model that helps power conversational AI experiences.

At Google, the founders had become frustrated with the company’s hesitancy to roll out AI chatbots to other researchers and the general public, including through integrations with other Google products, like Assistant, The Wall Street Journal reported. Believing AI would revolutionize search and other areas, the duo ultimately decided to leave Google in late 2021, despite pleas from CEO Sundar Pichai to stay and continue their work on LaMDA. That same November, Shazeer and De Freitas founded Character Technologies, now home to Character.AI.

The mobile version of the AI chatbot platform launched globally to iOS and Android users on May 23, where it had a strong showing on Google Play in particular. Within the first 48 hours, the app saw more than 700,000 Android installs, putting it ahead of top entertainment apps like Netflix, Disney+, and Prime Video, for example. That trend is continuing in the days following the launch, the company confirmed to TechCrunch, thanks to installs in large Android markets like Indonesia, the Philippines, and Brazil. The U.S. is also a leading market for Android downloads, the company said.

Image Credits: Character.AI

Interest in the mobile app was also heightened by the popularity of the web experience that preceded it. Ahead of the app’s launch, the Character.AI web app was topping 200 million visits per month, Character.AI claims, and users were spending on average 29 minutes per visit — a figure the company says eclipses ChatGPT by 300%.

In addition, Character.AI reports that users quickly become engaged after first use. For instance, the company noted this month that once users send their first message to a character, their engagement rates jump to more than 2 hours average time on platform. To date, users have created over 10 million custom AI characters, the company added.

The team, still just 30 people, has been particularly busy this month, having launched its premium service c.ai+, which offers a similar set of perks as ChatGPT Plus, like faster response times, access during peak periods of usage, and early access to new features. It also announced a strategic partnership with Google Cloud for building and training its AI models. The deal will see the startup using Google Cloud’s Tensor Processor Units to train and infer LLMs (large language models) faster and more efficiently, it said.

While some startups massage their reported download figures to make them sound better than they are, it seems Character.AI’s numbers are likely accurate. We asked third-party app intelligence provider data.ai to compare Character.AI’s numbers with their own estimates and data.ai came back with figures that actually showed the apps had a higher number of downloads. (The apps are so new, third-party estimates may not be accurate at this time — but, if anything, these third-party metrics do confirm there is initial consumer demand.)

However, while ChatGPT has demonstrated staying power following its launch — it’s still No. 3 today on the top free apps chart in the U.S. — Character.AI demand seems to have declined slightly post-launch. As of the time of writing, the iOS app is charting at No. 13 in the U.S. App Store’s Entertainment category. On U.S. iOS, the app was No. 4 overall for its first two days, but declined since to a rank of No. 89 overall. On U.S. Android, it reached No. 5 on 5/27 but dropped to No. 27 as of today, data.ai told us.

Asked if Character.AI was blowing some of its huge chunk of change on marketing spend to generate its initial installs, the company claims that was not the case. The app launched without a dedicated marketing budget and 99% of its downloads were organic, the startup said.

NVIDIA: From Chipmaker to Trillion-Dollar AI Powerhouse

From being a renowned chipmaker for computer graphics to an industry titan momentarily reaching a trillion-dollar valuation, Nvidia's journey is one for the business books. This rapid ascent is a testament to the growing significance of artificial intelligence (AI) in shaping the future of technology. Notably, Nvidia's stock soared over 5% in a single day, briefly nudging the company into the exclusive trillion-dollar club. This surge was primarily due to the anticipated “surging demand” driven by AI advancements, which have been the backbone of Nvidia's market strategy.

Evolving with the AI Revolution

Nvidia, founded in 1993, started its journey by crafting computer chips dedicated to processing graphics, specifically for gaming. This niche focus gradually shifted when Jensen Huang, Nvidia's co-founder, decided to invest in enhancing their chips' functionality. Huang's gamble hinged on the potential of AI, well before it was the buzzword of the technology sector. This foresight is a significant part of Nvidia's success story.

Today, Nvidia's hardware forms the bedrock of many AI applications. It dominates the AI landscape, holding an estimated 95% of the market for machine learning, which was an unthinkable achievement just a few years ago. For instance, the notable AI chatbot ChatGPT, which stirred the AI industry upon its launch, was trained using a cluster of Nvidia's graphics processing units (GPUs).

Challenges on the Horizon

Despite the triumphant headlines, the journey towards consistently maintaining a trillion-dollar valuation is fraught with challenges. While the pandemic gave Nvidia a boost, it's worth noting that the firm's overall revenue growth plateaued last year, and profits took a significant hit. Furthermore, maintaining the supply to meet the ever-growing demand poses another hurdle, particularly with competitors like AMD and Intel striving to create their own AI-centric offerings.

In addition to these market dynamics, Nvidia also grapples with ethical considerations. As a leading producer of chips for AI products, the company faces scrutiny over its responsibility in ensuring the ethical use of its technology amidst growing apprehensions about AI's societal impact.

Justifying the Valuation: A Balancing Act

Currently, Nvidia's market value surpasses Intel by more than eight times. Interestingly, this impressive valuation comes despite Intel's higher reported revenue in the previous year. This juxtaposition demonstrates the market's high expectations of Nvidia and the potential of AI, but it also raises questions about the justification of such high valuations.

While it's clear that AI is a critical growth area, some believe that Nvidia's shares may be “priced ahead of the curve,” with the expectation that the company will singlehandedly lead the AI revolution. This perception may not take into account the increasing competition and emerging alternatives in the AI space. It's reminiscent of past market dynamics, where once-favored tech giants saw their trillion-dollar valuations crumble as quickly as they rose.

Nvidia's temporary entry into the trillion-dollar club marks a significant milestone in the company's journey and the broader AI industry. It underscores the potential of AI and the high stakes involved, but it also highlights the volatility of the market and the challenges that lie ahead in maintaining such a lofty valuation. As the AI landscape continues to evolve, it remains to be seen how Nvidia navigates these challenges to stay ahead of the curve.