Рубрика: AI

The open source community has a love affair with Llama, Meta’s open source large language model. So much so that, many of the models that are coming up in the generative AI ecosystem are based on top of Llama 2, and developers are using the model for all purposes. Why should AMD stay behind?

While demonstrating the kernel performance of the newly released AMD Instinct MI300X Accelerator at the Advancing AI event, Lisa Su, CEO of AMD, said that MI300X performs 1.2 times better than NVIDIA H100 on a single kernel when running Meta’s Llama 2 70B. “We are going to talk about this a lot,” Su said, speaking about Llama 2.

Su further goes on and demonstrates that when it comes to inferencing Llama 2, one single server of AMD which consists of eight MI300X, performs 1.4 times faster than the server of an H100. Clearly, the performance of the MI300X is demonstrated the best with Llama 2.

Moreover, Kevin Scott, CTO of Microsoft discussed with Su on stage how AMD is contributing in Microsoft’s AI journey. While talking about the announcement at Ignite about using MI300X on Azure, Scott said that he is eager to Bring up GPT-4 and Llama 2 on MI300X and seeing the performance, and rolling it into production is something Scott said that he has been waiting for eagerly.

The mutual love for open source

“As important as the hardware is, software is what really drives innovation,” Su said, talking about the ROCm 6, the latest version of AMD’s open source parallel computing offering, which is an alternative to NVIDIA’s CUDA. ROCm 6 is releasing in the coming week.

Victor Peng, President of AMD, showed how building a strong ecosystem has enabled the company to create a successful open source framework in ROCm. For demonstrating this,

Peng showcased how MI300X with ROCm 6 is eight times faster than MI250X with ROCm 5, when inference Llama 2 70B.

On smaller models such as Llama 2 13B, ROCm with MI300X showcased 1.2 times better performance than NVIDIA coupled with CUDA on a single GPU.

The most groundbreaking announcement is that Meta is partnering with AMD and the company would be using MI300X to build its data centres. To demonstrate this partnership, Ajit Matthews, Meta AI senior director engineering with Su said that MI300X is trained to be the fastest design-to-deployment solution in Meta’s history.

Meta has been using 150k H100 GPUs for research and training. It would be interesting to see how the shift to AMD MI300X takes place.

“We are investing for the future by building new experiences for people through our services, and advancing open technologies and research for the industry,” said Matthews. In July, Meta opened up Llama 2 family of models and, “we were blown away by the reception from the community,” he added.

“We believe that an open approach feeds to better and safer technology in the long run, as we have seen with the PyTorch and Open Compute project, and dozens of other AI models,” said Matthews, speaking about how AMD was also the co-founder of PyTorch with Meta.

“We completely agree with the vision and with the open ecosystem, and that being the path towards innovation with all the smart folks in the industry,” said Su also highlighting AMD’s focus on open software such as ROCm, along with the race towards open hardware with MI300X.

AMD’s long love for Meta

Meta has been working with AMD for EPYC CPUs since 2019. Meta also recently deployed AMD’s Genoa and Bergamo based servers at scale across its infrastructure. “We have been working with the Instinct GPUs, starting from the MI100, since 2020,” said Matthews. “We have also been benchmarking ROCm and working together for its support on PyTorch across each generation of AMD Instinct GPU.”

“As the Llama family of models continue to grow in size and power, which they will, the MI300X with its 192GB of memory and higher memory bandwidth meets the requirement for large language model inference,” added Matthews. He has also said he is pleased with the fact that AMD was specifically optimised ROCm for Llama models and Meta is seeing great performance numbers.

Not just bigger models, AMD is also focusing on edge computing with the introduction of Ryzen AI PCs, along with the first version of Ryzen AI software for building generative AI applications on your PCs by using pretrained models available on Hugging Face, such as Llama 2.

To demonstrate the performance of AMD Ryzen 8040, the newest version of its on-device neural processing unit (NPU), Su highlighted that Llama 2 7B performs 1.4 times faster than the previous versions. This marks the beginning of using small Llama 2 models on hardware powered by AMD.

The long running partnership with Meta, and the mutual love for open source is the reason why AMD loves Llama so much.

The post AMD Loves Llama So Much appeared first on Analytics India Magazine.

The potential of artificial intelligence (AI) systems to take over people's jobs is one of the most common fears surrounding the technology. It's partly why many people hesitate to try tools like text and image generators, and why there's such a strong demand for stronger AI regulation.

These concerns aren't entirely off-base. The UK's Department of Education recently published a study that found that 10-30% of occupations can be automated by AI, with most of these being white-collar jobs.

Also: Will AI hurt or help workers? It's complicated

Take the example of AI chatbots, which have the potential to take over jobs for telephone salespersons, solicitors, psychologists, teachers, and market and street traders and assistants, according to the study. These chatbots, such as ChatGPT and Google Bard, can handle routine and repetitive tasks easily and consistently, operate around the clock, and interact with many customers simultaneously.

The study also looked at the jobs most likely to be exposed to AI in general, including tools beyond AI chatbots, like AI-assisted diagnostic tools in healthcare and algorithmic trading in finance, for example. The top five jobs most exposed to AI in general include management consultants and business analysts, financial managers, accountants, and psychologists.

Also: Two-thirds of professionals think AI will increase their skills' value

"The occupations most exposed to AI include more professional occupations, particularly those associated with more clerical work and across finance, law, and business management roles," according to the research study.

The relationship between how likely jobs are to be replaced by AI and how likely their workers are to benefit from the use of AI was found to be very close. AI and automation can handle routine tasks, such as data entry and processing simple transactions. AI could also be used as a tool by human workers to focus on more challenging tasks that require deeper insight, empathy, or decision making.

Healthcare is a perfect example of how AI can enhance the work of professionals rather than replace them. AI can help diagnose tumors with complex imaging in less time than a human, but a human doctor must make final decisions and differential diagnoses, and they'll need to process all the patient's information after considering medical observations.

Also: CIOs assess generative AI's risk and reward for software engineers

The study stopped short of saying AI would replace these jobs with certainty, using the "most exposed to AI" qualifier instead. This qualifier means these jobs are likely to be aided or replaced altogether by AI tools or AI chatbots. The research used an approach that considered the abilities to perform different job roles and the extent to which these can be augmented or replaced by a selection of 10 common AI applications.

The occupations that are least exposed to AI tools include, unsurprisingly, those requiring manual work and skilled labor. Sports players top the list of occupations least exposed to AI, with roofers, construction workers, plasterers, and steel erectors.

How can workers avoid being replaced by AI?

AI has the power to replace some workers. However, AI can also enhance workers' roles to help them save time and dedicate more energy to personalized tasks, make inferences with empathy or personal knowledge, and provide in-depth service to customers who need more attention.

Tools like AI chatbots can make creating, summarizing, and translating text easier, allowing human workers to revise, edit, and make necessary corrections and customizations. AI tools, such as ChatGPT, Copilot, and Bard, can create article outlines and summarize large documents in seconds.

Also: With AI, organizations are now seeing software developers as great collaborators

These tools can also answer questions quickly, without needing to do an internet search or search databases. For example, an AI chatbot can be trained on extensive company material and data to answer queries that are specific to that enterprise.

Artificial Intelligence

AI on-edge is rapidly gaining traction. Manufacturers of phones and computers are gearing up to integrate AI capabilities into their devices. Yet, the influence of AI is not confined to just phones and laptops; soon, automobiles could host AI models running on the edge.

Sima.ai, a US-based chip-making company focusing on the edge, is gearing up to bring AI capabilities to automakers. In a recent conversation with AIM, Krishna Rangasayee, founder and CEO at Sima.ai, said that a significant transformation is underway in the automotive industry, transitioning from traditional computing to an AI/ML-based computing architecture.

Missing piece of the puzzle

“This shift is set to redefine the industry’s technological landscape because there’s no way that classic computing can keep up with the demands of the future.”

He believes Advanced Driver Assistance Systems (ADAS) will be all AI/ML-based. In recent times, ADAS has emerged as a central focus in the automotive industry, bringing about a transformative impact on safety, convenience, and the overall driving experience.

“While every car manufacturer is taking a varied approach, what’s really clear as a macro trend is that AI is going to be everywhere,” he said.

Moreover, Rangasayee also believes how we interact with a car will change from an infotainment perspective. There’s a trend towards multimodal interfaces, allowing interactions through touch or voice, introducing diverse user experiences.

“While everyone today is talking about Large Language Models (LLMs), the next transition in the industry is going to be Large Multimodal Models (LMMs) and that I think is going to be the pervasive architecture that’s going to touch everything because it is the closest thing that we have come to in mimicking human capacity.

He added that he is moving forward and his company will extend support to various elements, encompassing textual, audio, image, and video information.

We have already seen automobile companies like Mercedez testing ChatGPT in cars. However, to make this happen, you need specialised chips and this is the gap Sima wants to fill in the market.

“The problem is, if you run that on computer chips or chips designed for something completely else, like mobile phones, it’s very inefficient,” Harald Kroeger, Head of Automotive and Sales at SiMa.ai, said.

As car manufacturers transition to AI, Sima.ai wants to be the company that provides the hardware and software to do so efficiently- the missing piece of the puzzle- as Kroeger puts it.

Indian automakers at the forefront of AI

Sima poached Kroeger earlier this year due to his extensive industry experience spanning over three decades. He has worked with some of the most influential automotive companies globally, including Bosch, Rivian, Daimler, and Tesla.

Recently, Rangasayee and Kroeger were in India talking to automobile companies here along with Kroeger.

“I have been coming to India for the last 20 years and I can’t stop admiring the speed at which the factories here develop. I don’t see any companies in Europe embracing and utilising new technology with the speed and efficiency that we see in India.

“This is why we are bullish not only due to the outstanding engineering quality in the country but also because of the ways we can assist Indian companies in accessing the world’s best AI hardware, propelling them to the next level and surpassing other nations,” Kroeger said.

While it was too early to reveal the names of the automobile companies Sima is in talking too, according to Kroeger, he does mention that Sima is in talks with the top 10 Tier 1 car manufacturers in the world to help them transition to AI with Sima’s edge solutions.

Do autonomous vehicles have a future in India?

Kroeger, who served on a board of directors at Tesla for over a year, believes it is impossible to see autonomous vehicles running in the busy streets of Bangalore or New Delhi. Today, companies like Waymo run self-driving taxis in cities like San Francisco.

However, driving on Indian streets compared to the West is a world apart. “Even a blind man can drive in the US,” Rangasayee jokes.

Kroeger also contends that the prospect of autonomous vehicles navigating Indian cities seems implausible in the current decade or the next. Nevertheless, this does not rule out the possibility of identifying specific use cases for autonomous vehicles in India.

When conversing with Indian customers, according to Kroeger, the consensus is that autonomous cars may not navigate the challenging traffic conditions prevalent here. However, there is recognition of the excellent highways for long-distance routes. The idea of having a car that can handle the journey between cities, for example, Bangalore to Mysore or Chennai, and provide comfort during those extended drives holds significant value.

Moreover, Kroeger believes Indian customers are tech-savvy and eager for new technology. “They are forward-thinking, and the current discussions revolve around planning for the future. It’s anticipated that certain features, if not fully autonomous, will become standard in every car.”

The post Is India Sitting on the Cusp of an Autonomous Vehicle Revolution? appeared first on Analytics India Magazine.

Wipro is certified as the Best Firm For Data Scientists to work for by Analytics India Magazine (AIM) through its workplace recognition programme.

The Best Firm For Data Scientists certification surveys a company’s data scientists and analytics employees to identify and recognise organisations with great company cultures. AIM analyses the survey data to gauge the employees’ approval ratings and uncover actionable insights.

“For the past two decades, Wipro has been a pioneer in delivering AI projects, establishing market-leading expertise in data science, cognitive process automation, artificial intelligence, and machine learning. We’ve successfully applied these capabilities to solve intricate business challenges for numerous Fortune 500 organizations across diverse industries and geographies. Recognized repeatedly by partners, customers, and market analysts, we consistently rank among the leaders in AI services. Wipro is not only a preferred workplace for data scientists and ML engineers but also offers ample opportunities for learning and career growth. This outstanding recognition from AIM, reinforces our confidence in the right path and inspires us to push further” said Anindito De, Practice Head, AI Technology Services at Wipro.

Wipro is fueling the next wave of AI-driven innovation by infusing AI across their ecosystem—into every tool, platform, and solution, across every business function, process, and practice. Bringing together all AI capabilities, talent, and technology because combining human ingenuity and AI-powered technology is the key to unlocking the true value of AI.

Focused on building digital-era, AI-first intelligent enterprises, Wipro is building on their 620+ patents, propelled by 55,000 AI ecosystem practitioners, who are working in 20 innovation centers and digital pods across the globe, delivering over 2,000 AI client engagements. Wipro is designing pathways to the future and helping its clients realize their ambitions.

The analytics industry currently faces a talent crunch, and attracting good employees is one of the most pressing challenges that enterprises are facing.

The certification by Analytics India Magazine is considered a gold standard in identifying the best data science workplaces and companies participate in the programme to increase brand awareness and attract talent.

Best Firms for Data Scientists is the biggest data science workplace recognition programme in India. To nominate your organisation for the certification, please fill out the form here.

The post Wipro is Certified as a Best Firm for Data Scientists appeared first on Analytics India Magazine.

With the number of large language models (LLMs) in the market expected to grow and branch out, businesses will need a governance framework to manage their generative artificial intelligence (AI) applications.

Organizations will require layers of intelligence that pull together internal and external capabilities, said Frederic Giron, Forrester's vice president and senior research director.

Also: 3 ways to secure the best AI partner for your business

This approach will encompass the use of paid and open-source LLMs from third parties, such as OpenAI's ChatGPT, Anthropic's Claude, and Meta's Llama, and embedded AI tools, such as Salefsforce Einstein GPT. Organizations will also have their own AI models, including using generative AI, tapping general-purpose and specialized LLMs, and running various AI applications alongside key processes, policies, and business rules.

The approach will be underpinned by structured and unstructured data, with the latter expected to double amid the adoption of generative AI as companies deploy more conversational experiences for customers and employees, said Giron, who was speaking at the research firm's 2024 predictions briefing this week.

Also: Two divergent skills that matter in an AI world: Math and business development

User response and behavior should also be fed into a feedback loop and used to fine-tune the system.

These requirements underscore the need for businesses to have a generative AI application architecture to govern and ensure the use of these tools is safe and efficient, he said.

This framework should connect the application pipes, orchestrate requests into outputs, and pave the input and output gateways, so the organization can control what data goes into the AI models and ensure the responses comply with the rules the business has set.

The complexities around AI governance mean it might take a while before businesses will see real results from their adoption of a framework.

Forrester predicts the transformative impact of generative AI will benefit just 30% of Asia-Pacific firms over the next year. Giron pointed to key challenges related to data governance, quality, and infrastructure.

Also: 4 ways generative AI can stimulate the creator economy

To help businesses plug the gaps, he noted that service providers are investing in transforming how they operate and deliver their service models, including expanding their industry partnerships and releasing new platforms, such as AI studios and model comparisons.

This investment will drive better pricing models and, over a longer term, impact commercial models. The results will be more outcome-based and solution-based pricing structures, among others.

Also: Businesses need a new operating model to compete in an AI-powered economy

The analyst added that 56% of organizations expect employee productivity to be the leading use case for generative AI, followed by 48% that point to software development and testing. Another 48% see generative AI as an enabler of self-service data and analytics.

Unsurprisingly, generative AI is the biggest tech thunderstorm to hit in 40 years, according to Dane Anderson, Forrester's senior vice president of international research and product.

It follows previous "inflection point" technologies that saw cloud computing emerge in 2009 and the mobile internet and smartphone wave rise in 2007, Anderson said. The World Wide Web inflection took hold of the 1990s, while personal computers made their mark in 1981.

Some of these innovations brought about greater changes than others, creating both opportunities and challenges, he added. With the emergence of generative AI, the analyst predicted that "static" websites will gradually be abandoned over the next 20 years.

Also: Generative AI will far surpass what ChatGPT can do. Here's everything on how the tech advances

Users will instead evolve to prompt or ask a query, to which they will get a response that is continually updated in the backend — powered by generative AI — and customized for an improved interactive experience.

These changes to websites will further impact search, which will no longer be as central or critical as it is now, Anderson said.

Such significant transformations will play out over several years. In the shorter term, the anticipated emergence of more LLMs in the market means organizations will need to carefully assess their options and determine which models are best suited for the outcomes they want.

Anderson also noted the potential for more market players to start embedding generative AI capabilities for free into their existing customer enterprise applications.

Ultimately, the value for businesses is not in the layer where LLMs operate, said Leslie Joseph, Forrester's principal analyst, as this market segment will be commoditized as more LLMs pop up, he added.

Joseph urged software vendors to start integrating generative AI features into their products, rather than offering these tools primarily as their version of a ChatGPT equivalent. This refined approach will help drive a workplace environment where generative AI capabilities are more ingrained into how employees work and make the technology more affordable for businesses, he said.

Artificial Intelligence

Earlier this week, San Francisco-based Together AI, an organisation dedicated to democratising AI through an open-source cloud platform, secured a $102.5 million Series A investment from NVIDIA, Kleiner Perkins, and Emergence Capital.

“Open source is the future of AI,” said founder and CEO Vipul Ved Prakash, adding that “it will be a major thrust of how most organisations implement generative AI.”

The latest funding has come after the company raised a $32.5 million seed round from Lux Capital and other investors, propelling its valuation to $200 million since January. The company aims to empower developers globally, offering them the tools to build and integrate AI models seamlessly into their applications.

“Together AI is well positioned to be the platform of choice as enterprises look to control their proprietary IP while pushing their generative AI investments from prototype into production,” said Joseph Floyd, GP of Emergence Capital.

What Works for Them

The market’s confidence in Together AI’s trajectory, anticipated to create approximately $10 million in revenue, is derived from a customer base of around 20, highlighting the substantial market traction it has gained.

Its flagship product, Forge, is a significant driving force behind Together AI’s revenue surge. Contributing 90% of the company’s total revenue, Forge offers startups a comprehensive solution encompassing computing resources and model training software within a single package. Launched in June, Forge promises access to servers housing Nvidia’s A100 and H100 chips at a remarkable 20% of the cost compared to renting them from major cloud service providers like Amazon Web Services. The platform’s cost-effectiveness positions it as a formidable competitor, potentially surpassing Nvidia’s own GPU cloud service.

The robust computing infrastructure is central to their prowess, scaling up to a staggering 20 exaflops across multiple data centres in the US and EU. Their cloud infrastructure, boasting NVIDIA GPUs and networking, in partnership with AI cloud leaders like Crusoe Cloud and Vultr, is meticulously tailored for high-performance AI applications. This bespoke infrastructure grants the company a competitive edge, offering superior economics for pre-training and inference workloads.

Additionally, as startups and enterprises increasingly seek a generative AI strategy that avoids reliance on a single vendor. A key pillar of Together AI’s success lies in its diverse repository of open-source models, hosting an array of models such as Llama 2, Stable Diffusion, and RedPajama—its own curated set of open-source models and datasets. RedPajama, in particular, has already garnered substantial traction on model hub Hugging Face, with its 2.8 billion parameter version downloaded nearly 20,000 times last month.

By providing customers with software and computing resources to run models, Together AI stands shoulder to shoulder with high-profile startups such as CoreWeave, Lambda Labs, and Foundry Technologies. This strategic positioning has garnered substantial investor attention, especially amid the ongoing GPU shortage, where startups collectively raise billions to address the demand.

Fueling Advancements Through Research.

Crucially, TogetherAI’s industry-leading performance foundation and reliability rests on its commitment to core research. Their groundbreaking endeavours, including the release of the RedPajama-V2 dataset, the largest open dataset comprising 30 trillion tokens for training Language Models (LLMs), highlight their dedication to fostering advancements within the AI ecosystem.

Tri Dao, Chief Scientist at Together AI, and collaborators unveiled FlashAttention v2, a pivotal innovation utilised by leading entities such as OpenAI, Anthropic, Meta, and Mistral in developing top-tier LLMs. Moreover, the company’s strides in inference techniques, embodied in Medusa—a framework for accelerating LLM generation and Flash-Decoding—for long-context inference have culminated in the fastest inference stack for transformer models available through the Together Inference API.

Another allure is the distinguished lineup of founders at Together AI—which further fortifies investor confidence. Comprising Percy Liang, a Stanford University professor heading its Center for Research on Foundation Models, Chris Ré, a co-founder of SambaNova Systems and Snorkel AI, and former Apple executive Vipul Ved Prakash serving as the CEO, the ensemble embodies expertise and vision in the AI realm.

The company’s mission to champion open and decentralised systems echoes the path of MosaicML, acquired for $1.3 billion by Databricks, signalling the potential valuation trajectory for Together AI, which, if mirrored, could reach a soaring $650 million valuation.

The post Why Everyone’s Going Gaga over TogetherAI appeared first on Analytics India Magazine.

After the successful year with everyone wanting their hands on an H100, NVIDIA is ready with its next-gen GPUs. The company is rolling out GH200 GPU to selected companies that would be testing and deploying models on it.

Most recently, Bindu Reddy, the CEO of Abacus AI, the company that is focused on building generative AI applications and agents at scale, announced on X that it is going to receive GH200 supercomputers today. She said that she would like to focus on open source projects using the AI supercomputers starting January.

She also said that Abacus AI is probably the first of the few companies to receive it.

NVIDIA GH200, the successor of H100, are expected to be available by the end of this year, and systems running on them are expected to start by the second quarter of 2024.

In November, AWS had announced that it would be the first customer to use the NVIDIA GH200 on its cloud. This was after Google Cloud, Meta, and Microsoft were anticipated to be among the early adopters granted permission to utilise the DGX GH200 for investigating its potential in handling generative AI workloads.

Oracle Cloud Infrastructure (OCI) has also announced plans to use the GH200s for its cloud offerings.

NVIDIA also plans to share the DGX GH200 design as a model with cloud service providers and other hyperscalers, allowing them to tailor it to better suit their infrastructure.

On the other hand, Microsoft and Meta have also announced plans to integrate AMD’s recently launched Instinct MI300X accelerators for AI workloads. Meta is going to build its new data centre using MI300X, which AMD directly compares with H100.

Moreover, Intel is also about to announce its Gaudi3 AI accelerator on December 14th at its Intel ‘AI Everywhere’ conference. Intel showcased that Gaudi2, the previous version of its accelerator, is very close to NVIDIA’s H100, and was a cheaper alternative as well.

The post NVIDIA is Rolling Out GH200 for its Customers appeared first on Analytics India Magazine.