AI — Страница 929

1 out of 3 marketing teams have implemented AI in their workflows

Advancements in data and artificial intelligence (AI) technologies are prompting marketers to rethink, reconfigure, and revolutionize how companies connect with customers, according to Salesforce's 2024 State of Marketing Report. For the report's ninth edition, Salesforce surveyed nearly 5,000 marketers, generating 4,850 responses from marketing decision-makers across North America, Latin America, Asia-Pacific, and Europe.

Also: 6 ways AI can help launch your next business venture

Marketers are evolving their practices in a highly competitive landscape. They're looking to AI — both generative and predictive — to help personalize at scale and boost efficiency. Marketers rank AI adoption as both their number one priority and challenge. Here is the executive summary of the report captured in five key findings:

Marketers shore up their data foundations — Businesses have long struggled to connect disparate data points to create consistent, personalized experiences across customer journeys. Yet as third-party cookies are depreciated and AI proliferates, that quest is only becoming more critical — and challenging. Only 31% of marketers are satisfied with their ability to unify customer data sources.
Marketers embrace AI with an eye on trust — Marketers are intent on successfully applying AI in their operations with the right data, but are concerned about security and customer trust as adoption ramps up. Thirty-two percent of marketing organizations have implemented AI in their workflows, and an additional 43% are experimenting with it.
Full personalization remains a work in progress — What constitutes a "personalized experience" continues to mature, and there's a stark difference between how the highest- and lowest-performing marketing teams adapt. On average, high performers fully personalize across six channels, compared with underperformers who fully personalize across three.
Marketers seek unified analytics — There is no shortage of data sources, but putting that data to work is a challenge. Only 48% of marketers track customer lifetime value (CLV).
Deeper relationships emerge with ABM and loyalty programs — Companies are increasingly turning to strategies like account-based marketing (ABM) and loyalty programs for better customer acquisition and retention. Yet many of these programs' information sources remain disjointed, and so is the resulting customer experience. Thirty-nine percent of marketers say loyalty program functionalities are accessible across all touchpoints.

This article will focus on the report's findings related to AI and data-related research.

AI tops marketing agenda

Taking advantage of AI is the marketer's biggest priority — and biggest challenge. Maintaining trust, another key area of focus, is a core part of successful AI deployment. In fact, 68% of customers say advances in AI make it more important for companies to be trustworthy.

Marketer's top priorities:

Implementing or leveraging AI
Improving use of tools and technologies
Improving marketing ROI/attribution
Engaging with customers in real-time
Building/retaining trust with customers

Marketer's top challenges:

Difficulties implementing or leveraging AI
Engaging with customers in real time
Building/retaining trust with customers
Measuring marketing ROI
Creating a cohesive customer journey

Also: Ready to upskill? Look to the edge (where it's not all about AI)

The report found that marketers use an average of eight different marketing tools and technologies. All AI projects start as data projects. Capturing customer data from a large set of marketing tools that are not integrated creates a challenge for marketers to unify, harmonize, and create a 360-degree view of their customer engagement touchpoints. Only 32% of marketers are satisfied with how they use customer data to create relevant experiences.

Marketers shore up their data foundation

Marketers use an average of 9 different tactics across the entire customer journey. The most common data sources for marketers are:

Customer service data
Transactional data
Mobile apps
Web registrations
Loyalty programs
Subscriptions
Online learning platforms
Access to discounts
Interactive tools
Cause-based marketing

The report found that 38% of marketers don't use third-party data.

The modern marketer's challenge isn't a lack of first-party data — it's fully integrating this data across departments to glean insights, plan campaigns, and suppress messages from reaching the wrong audiences, to name a few examples.

Also: 5 reasons why I prefer Perplexity over every other AI chatbot

Only 31% of marketers are satisfied with their ability to unify customer data sources. Data integration across the enterprise app landscape is key to high-performance marketing. About two in five marketers still don't have real-time data at their disposal for crucial tasks, relying instead on potentially outdated insights — or even intuition. Even teams with live data are slowed down by their ability to activate it. While over half of marketers say data is available in real time to execute a campaign, 59% need the IT department's help to do so. Most customer data is trapped in business, with limited or no access to marketing.

Marketers embrace AI with an eye on trust

In 2022, 68% of marketers had a defined AI strategy. Today, 75% of marketers are already rolling up their sleeves and experimenting with or implementing AI. Yet a closer look reveals an uneven landscape. High performers are 2.5 times more likely than underperformers to have fully implemented AI within their operations. Already, generative AI use cases rank among marketers' favorites alongside more established predictive AI applications.

Top marketing AI use cases:

Automating customer interactions
Generating content
Analyzing performance
Automating data integration
Driving best offers in real time

Also: How to use ChatGPT to make charts and tables with Advanced Data Analysis

Marketers do have concerns with AI. Eighty-eight percent of marketers worry about missing out on generative AI's benefits, compared to 78% of sales and 73% of service colleagues. And 41% of CMOs cite data exposure as a top concern compared to 29% of VPs and 32% of team leads. Ninety-eight percent of marketing leaders believe trustworthy data is essential. But, just as the data must be trustworthy, so should its integration with AI.

Ranking of marketer's generative AI concerns:

Data exposure or leakage
Lack of necessary data
Lack of strategy or use cases
Inaccurate outputs
Copyright or intellectual property concerns
Distrust in generative AI
Biased outputs
Fear that AI will replace jobs
Adherence to brand guidelines
Difficulty learning how to use AI

Also: ChatGPT vs. Microsoft Copilot vs. Gemini: Which is the best AI chatbot?

The State of Marketing report reveals the importance of data and AI in terms of measuring success and meaningful outcomes. Performance metrics continue to focus on revenue. Marketers closely monitor their marketing/sales pipeline (64%) and funnel (63%). And 48% of marketers track customer lifetime value. The most important marketing metrics include the following: customer retention rates, customer acquisition costs, customer satisfaction, customer referral rates and volumes, and customer lifetime value.

To learn more about the 2024 State of Marketing report, you can visit here.

Artificial Intelligence

Gartner Predicts Worldwide Chip Revenue Will Gain 33% in 2024

It’s no secret the AI accelerator business is hot today, with semiconductor manufacturers spinning up neural processing units, and the AI PC initiative driving more powerful processors into laptops, desktops and workstations.

Gartner studied the AI chip industry and found that, in 2024, worldwide AI chip revenue is predicted to grow by 33%. Specifically, the Gartner report “Forecast Analysis: AI Semiconductors, Worldwide” detailed competition between hyperscalers (some of whom are developing their own chips and calling on semiconductor vendors), the use cases for AI chips, and the demand for on-chip AI accelerators.

“Longer term, AI-based applications will move out of data centers into PCs, smartphones, edge and endpoint devices,” wrote Gartner analyst Alan Priestley in the report.

Where are all these AI chips going?

Gartner predicted total AI chips revenue in 2024 to be $71.3 billion (up from $53.7 billion in 2023) and increasing to $92 billion in 2025. Of total AI chips revenue, computer electronics will likely account for $33.4 billion in 2024, or 47% of all AI chips revenue. Other sources for AI chips revenue will be automotive electronics ($7.1 billion) and consumer electronics ($1.8 billion).

Of the $71.3 billion in AI semiconductor revenue in 2024, most will come from discrete and integrated application processes, discrete GPUs and microprocessors for compute, as opposed to embedded microprocessors.

Discrete and integrated application processors saw the most growth in AI semiconductor revenue from devices in 2024. Image: Gartner

In terms of AI semiconductor revenue from applications in 2024, most will come from compute electronics devices, wired communications electronics and automotive electronics.

Gartner noticed a shift in compute needs from initial AI model training to inference, which is the process of refining everything the AI model has learned in training. Gartner predicted more than 80% of workload accelerators deployed in data centers will be used to execute AI inference workloads by 2028, an increase of 40% from 2023.

SEE: Microsoft’s new category of PCs, Copilot+, will use Qualcomm processors to run AI on-device.

AI and workload accelerators walk hand-in-hand

AI accelerators in servers will be a $21 billion industry in 2024, Gartner predicted.

“Today, generative AI (GenAI) is fueling demand for high-performance AI chips in data centers. In 2024, the value of AI accelerators used in servers, which offload data processing from microprocessors, will total $21 billion, and increase to $33 billion by 2028,” said Priestley in a press release.

AI workloads will require beefing up standard microprocessing units, too, Gartner predicted.

“Many of these AI-enabled applications can be executed on standard microprocessing units (MPUs), and MPU vendors are extending their processor architectures with dedicated on-chip AI accelerators to better handle these processing tasks,” wrote Priestley in a May 4 forecast analysis of AI semiconductors worldwide.

In addition, the rise of AI techniques in data center applications will drive demand for workload accelerators, with 25% of new servers predicted to have workload accelerators in 2028, compared to 10% in 2023.

The dawn of the AI PC?

Gartner is bullish about AI PCs, the push to run large language models locally in the background on laptops, workstations and desktops. Gartner defines AI PCs as having a neural processing unit that lets people use AI for “everyday activities.”

The analyst firm predicted that, by 2026, every enterprise PC purchase will be an AI PC. Whether this turns out to be true is as yet unknown, but hyperscalers are certainly building AI into their next-generation devices.

AI among hyperscalers encourages both competition and collaboration

AWS, Google, Meta and Microsoft are pursuing in-house AI chips today, while also seeking hardware from NVIDIA, AMD, Qualcomm, IBM, Intel and more. For example, Dell announced a selection of new laptops that use Qualcomm’s Snapdragon X Series processor to run AI, while both Microsoft and Apple pursue adding OpenAI products to their hardware. Gartner expects the trend of developing custom-designed AI chips to continue.

Hyperscalers are designing their own chips in order to have a better control of their product roadmaps, control cost, reduce their reliance on off-the-shelf chips, leverage IP synergies and optimize performance for their specific workloads, said Gartner analyst Gaurav Gupta.

“Semiconductor chip foundries, such as TSMC and Samsung, have given tech companies access to cutting-edge manufacturing processes,” Gupta said.

At the same time, “Arm and other firms, like Synopsys have provided access to advanced intellectual property that makes custom chip design relatively easy,” he said. Easy access to the cloud and a changing culture of semiconductor assembly and test service (SATS) providers have also made it easier for hyperscalers to get into designing chips.

“While chip development is expensive, using custom designed chips can improve operational efficiencies, reduce the costs of delivering AI-based services to users, and lower costs for users to access new AI-based applications,” Gartner wrote in a press release.

Google plans $2B investment for datacenter and cloud buildout in Malaysia

Google has unveiled plans to invest $2 billion in Malaysia, where it is looking to build its first data center and drive local capabilities in artificial intelligence (AI).

The new site will serve as a cloud region and support Google's digital services, including Maps, Workspace, and Search, the company said in a statement on Thursday. It also will power AI services that the cloud vendor is looking to drive in the country.

To be located in Malaysia's capital Kuala Lumpur, the data center will be in one of 11 countries where Google runs such a facility, including Singapore, Indonesia, and South Korea. The vendor currently operates 40 cloud regions and 121 zones worldwide.

Also: Microsoft wants to arm 2.5 million people in Asean with AI skills

When operational, the new data center will run alongside Google's dedicated cloud interconnection sites in Cyberjaya and Kuala Lumpur. It also marks the vendor's largest investment so far in Malaysia, where it has operated for the past 13 years, according to Alphabet CFO Ruth Porat.

"[In particular], the Google data center and cloud region…will empower our manufacturing and service-based industries to leverage AI and other advanced technologies to move up the global value chain," Senator Tengku Datuk Seri Utama Zafrul Aziz said in a statement.

In March, Google launched programs to train Malaysian youth in AI-focused skills, working with the Ministry of Higher Education to provide 161 institutes of higher learning with 500 Google Career Certificate scholarships each. It also offered Google Workspace tools to 445,000 public officers.

The US company added that it had introduced two programs to boost AI literacy among local students and educators, including the Gemini Academy, which aims to help educators use generative AI tools, such as Gemini, responsibly. More than 600 educators have participated in this program since its pilot last November, which now has expanded to include more educational institutions. Google is targeting for 15,000 educators to go through the program by year's end.

Also: Google joins collaborative efforts to build localized large language models

Google has also been providing on-site AI training and materials to students aged 11 to 14. Launched in April, that program aims to train an initial target of 1,000 educators and, in turn, reach 10,000 students across Malaysia, Google said.

How to get ChatGPT to browse the web for free

At its Spring Update event, OpenAI finally addressed one of ChatGPT's biggest pain points — its knowledge cutoff. Now, both paid and free users can have ChatGPT pull their answers from the web through its Browse feature, giving the chatbot the most up-to-date information.

Also: OpenAI just gave free ChatGPT users browsing, data analysis, and more

One of the Browse feature's biggest perks is that ChatGPT can reference the web to provide answers about current events, which the free version was previously unable to do. For example, I asked the chatbot the timely question, "What is the weather in NYC?" ChatGPT found the answer by visiting five websites, as seen below.

Another major perk, also seen in the image above, is that when ChatGPT uses the Browse ability, it includes links to the websites from which it pulled its answer. This is especially important for users seeking to verify the information, especially since generative AI tools such as ChatGPT are prone to hallucinations.

Getting started is easy, but it can be confusing when you first visit the ChatGPT interface because there is no obvious Browse setting that you can simply toggle on. Follow the steps below and you can get started in seconds.

1. Log into ChatGPT

Even though OpenAI made it possible for users to access ChatGPT without logging in, if you want access to certain perks, such as GPT-4o and all of its advanced features — including browse, vision, data analysis, file uploads, and GPTs — you need to sign in to your account.

Also: How to use ChatGPT to make charts and tables with Advanced Data Analysis

Creating an account is easy. You have two options: You can create an OpenAI account from the sign-in page, or log in with your existing Google or Microsoft account. I think the latter is the easiest option.

2. Ask a question on current events

To activate the Browse feature, you'll find that no setting needs to be turned on. It's much easier than that! All you have to do, in OpenAI's words, is "ask a question in the chat that requires the use of the Browse feature."

Also: ChatGPT vs. Copilot: Which AI chatbot is better for you?

In my experience, this means that whenever you ask the chatbot anything about current events, Browse will turn on automatically. For example, when I asked ChatGPT, "Who is currently the President of the United States?" it automatically turned on the Browse feature to index the web for the answer.

That's it! Happy ChatGPT web browsing!

Artificial Intelligence

Gartner’s 7 Predictions for the Future of Australian & Global Cloud Computing

Cloud computing will account for 70% of global enterprise workloads by 2028, up from about 25% now, according to Gartner, and issues like sustainability, AI computing and data sovereignty will play greater roles in how Australian enterprises use and procure cloud vendors.

At the Gartner IT infrastructure, Operations & Cloud Strategies Conference in Sydney, Dennis Smith, a leading cloud computing analyst, told Australian cloud computing executives the cloud had moved from being a technology disrupter to a business disrupter and was now becoming a business essential.

“If you don’t have a cloud strategy that’s solid and aren’t executing on it, you’re going to be putting your business at risk in many ways,” he said. “We’ve gone beyond this being a thing you’re kind of dabbling with to really something that needs to be a part of your much larger IT strategy.”

Gartner’s seven cloud computing trend predictions for Australia and globally until 2028 were:

More than half of current multicloud plans will fail to provide value by 2028.
Cloud-native platforms will be the de facto way of implementing new applications.
Cloud modernisation will see 70% of workloads in cloud environments by 2028.
Industry clouds will be used by more than half of all organisations in the cloud.
Multinationals will need to have a digital sovereignty strategy by 2028.
Sustainability will become a top five procurement criterion for cloud vendors.
AI and machine learning will account for 50% of cloud computing by 2028.

In additional research published to coincide with the Australian cloud conference, Gartner predicted Australian companies would spend AUD $23.3 billion (US $15.4 billion) on public cloud in 2024, up 19.7% from 2023. Spending on software-as-a-service will continue to be the largest category, up 18.3% in 2023 to AUD $11 billion (US $7.2 billion) in spending.

1. More than half of current multicloud plans will fail to provide value by 2028

Multicloud strategies are a top agenda item for clients in 2024, Smith said. However, while saying multicloud was not inherently a bad strategy, 50% or more organisations would not gain the value they were seeking by 2028, often because “they are not always doing multicloud for the right reasons.”

Smith said multicloud did not always provide portability or resilience if applications themselves are not architected and encoded for those advantages. Multicloud may not be cheaper if customers have less price leverage with a cloud vendor or need to spend on talent and tooling to manage the environments.

2. Cloud-native platforms to become the de facto way of implementing new applications

Gartner believes cloud-native platforms will be the default for building new applications by 2028, whether in the public cloud or in on-prem or hybrid environments. Smith described cloud-native as those platforms “enabling developers to get up to speed and develop code quicker.”

“Think of the ability to build applications that are scalable, that are instrumented already, that have a nice tight CI/CD (continuous integration and deployment) pipeline, that give the ability to implement some serverless functions. Maybe a managed Kubernetes offering or other activity that makes it easier for me as a developer to code that application,” he said.

3. Cloud modernisation will see 70% of Australian and global workloads in cloud environments by 2028

Gartner said the focus on modernisation from enterprises and cloud providers, as well as new emerging AI tools for modernisation such as tools discovering legacy systems or refactoring notes, will see a dramatic shift in the proportion of workloads in the cloud, from 25% to 70%.

SEE: The top five advantages of cloud computing.

Australian research firm ADAPT has found highly modernised organisations in Australia already have 67% of their workloads in public clouds and predicted that 55% of workloads overall will be in public clouds by 2025, with larger organisations in particular more committed to cloud strategies. Hyperscalers like Microsoft have been investing in new cloud capacity.

Chart showing Australian organisations are predicted to home 55% of their workloads in public clouds by 2025. — Australian organisations are predicted to home 55% of their workloads in public clouds by 2025. Image: ADAPT Research

“The remaining will be within your existing data centers and such. A key takeaway is that the future for most of you will be hybrid, so do end up planning for that.” Smith added the blanket concept of organisations just moving everything to the cloud without vetting applications is not the right path to take.

4. More than half of all organisations will accelerate with industry cloud platforms

There is a 50% chance or more, according to Gartner, that organisations will utilise an industry cloud platform by 2028. Smith described industry clouds as a combination of an infrastructure platform and SaaS offering, enabling a company to jump-start initiatives in an industry, like manufacturing or retail.

“There are numerous vendors in this space, including the hyperscalers. So do end up anticipating this, particularly if you’re looking at gaining a competitive advantage within your respective industries,” he told delegates at Gartner’s conference.

5. Digital sovereignty issues will require multinationals to develop strategies by 2028

The multinationals Australian cloud professionals work within will need strategies around digital sovereignty by 2028, Gartner argued. Strategies will aim at having more control over technology, data or operations in national locations, or even having the technology disconnected in some manner.

“This is a very hot area among many of my clients in certain parts of the world, where they may be in countries and a bit hesitant, frankly, to use a cloud provider across the ocean, or may have a fear that there may be some national issues that may cause that to be disconnected,” Smith elaborated.

Australia’s own dependence on tech from the U.S. and China has been noted. The Australian government has admitted dependence on the three U.S. hyperscale clouds and was left stranded when Microsoft pulled out of a project that would have provided a sovereign Australian “top secret” cloud.

6. Sustainability will become one of the top five procurement criteria for cloud vendors

Gartner estimated a quarter of organisations are already asking for sustainability information as part of the procurement process for cloud vendors, with particular interest over the last three years. Smith said that would at least double over the next four years to become a top five criteria.

In 2024, Australia released draft legislation that would introduce mandatory climate-related reporting. These requirements would require reporting from larger companies with more than 500 employees, revenues higher than AUD $500 million (US $331 million) or AUD $5 billion (US $3.3 billion) in assets from the 2024/25 financial year, with medium-size and smaller companies to follow in the next two financial years.

Smith noted some cloud vendors are already having to build data centres in adjacent countries if there are energy consumption restrictions in jurisdictions. He said enterprises in the near future would be asking vendors for more transparency and would need to look at the energy consumption of their own data centres.

7. AI and machine learning will account for 50% of cloud computing resource usage

The amount of cloud computing resources directed to AI and machine learning is about 10% today, but this will increase fivefold, though other activity will not decrease. Smith said the industry would move from a “medium sized pizza to a large pizza,” with 50% dedicated to AI and machine learning.

Gartner VP Analyst Michael Warrilow said generative AI is becoming a key driver and differentiator of future cloud demands. “Australian CIOs must determine the best adoption model for their needs, whether to build a model from scratch, or focus on AI capabilities being integrated into the applications they buy.”

Editor’s note: TechRepublic covered Gartner IT infrastructure, Operations & Cloud Strategies Conference remotely.

OpenAI inks deal with ride-sharing operator to develop AI tools

OpenAI has inked a deal to provide its artificial intelligence (AI) technology to ride-sharing operator Grab and help build tools customized for the Singapore-based company's operations.

Touted as OpenAI's first such partnership in Southeast Asia, the collaboration will also see Grab employees access ChatGPT Enterprise in a pilot rollout, according to a joint statement released on Thursday. Grab will deploy the generative AI tool to select employees in the pilot as part of its efforts to use AI tools to improve productivity.

Also: OpenAI just gave free ChatGPT users browsing, data analysis, and more

Grab provides a range of services via its app, including food and grocery delivery, parcel delivery, rides, fintech and payment services, and telemedicine. It operates in eight Southeast Asian markets including Singapore, Indonesia, Malaysia, Thailand, and the Philippines.

Under the partnership, the company will tap OpenAI's visual AI capabilities to incorporate more automation and pull higher data quality from visual images to improve its maps.

These efforts will enable GrabMaps to be updated faster and provide a better experience for customers and drivers in its ride-sharing partner network, according to the two companies.

Grab will also explore how the company can leverage AI to offer customer support chatbots that better understand and resolve issues.

Also: AI-powered headphones would let you listen to just one person in a crowd

In addition, Grab will use AI-powered text and voice capabilities to enhance the accessibility of its services, including for visually impaired and elderly users who may face difficulties navigating its app on the screen.

"[We] believe generative AI has a lot of potential to further transform the way we solve problems for our partners and users," Grab chief product officer Philipp Kandal said in a statement. "Our goal with any new technology has always been to use it to solve real problems, at scale."

Artificial Intelligence

Your Amazon Fire TV is getting a free generative AI upgrade. Here’s how it works

Amazon Fire TV 55-inch 4-Series 4K UHD smart TV

Amazon today introduced a new — unsurprising, even — feature for its Fire TV devices: an AI-powered search that should make finding your next binge-worthy show a breeze. By using a large language modeling (LLM) to process requests and deliver personalized recommendations, the company hopes its AI service will save users time and energy.

Also: AI-powered headphones would let you listen to just one person in a crowd

Why was this necessary? In today's streaming landscape, the abundance of content can be both a blessing and a curse. According to Nielsen's 2023 State of Play report, the average streaming customer spends 10 minutes or more searching for the right show each time they sit down to watch.

This indecision and choice overload can be frustrating, but it's even more frustrating when you know what you want to watch but can't think of the title. The new Fire TV's AI feature has a voice-activated solution.

The new search feature allows viewers to ask Alexa for recommendations using complex and nuanced language, covering topics, plots, characters, genres, actors, and even quotes. An example might be to say: "Alexa, show me the Seinfeld episode where Elaine dances," or "What movie has the line, 'You're killing me, Smalls?'"

Also: 5 ways Amazon can make an AI-powered Alexa subscription worth the cost

Amazon says something more vague and general works, too, including: "Show me movies where people get sucked into a video game."

The system's vast entertainment library includes choices from users' Prime Video and various other streaming subscriptions, so it will be apparent which shows are available or free for you to click into.

The new Fire TV search experience has begun rolling out the English version to customers in the U.S. on Fire TV devices running FOS6 and later. It will be available on all eligible Fire TV devices in the U.S. in the coming weeks.

Featured

Anthropic brings Tool Use for Claude out of beta, promising sophisticated assistants

An increasingly popular trend in generative artificial intelligence is to give AI models "agent" capabilities, the power to tap into external programs such as databases, or a web browser with live search functionality.

OpenAI popularized the notion of AI agents in November when it introduced its "Assistant" API, meant to make it easier for developers to call specific functions for their applications. On Thursday, OpenAI competitor Anthropic made its bid for developers' focus by making generally available what it calls Tool Use for Claude, which is designed "to automate tasks, personalize recommendations, and streamline data analysis by integrating AI with external tools and services."

Also: Anthropic launches a free Claude iOS app and Team, its first enterprise plan

Anthropic debuted Tool Use, also known as function calling, with the introduction of its Claude 3 family of models in March. There's already a fairly extensive set of posted instructions for developers for how to use the API in the beta version.

Today's announcement takes Tool Use out of beta and available through Anthropic's own Anthropic Messages API, the Amazon Bedrock service, and Google's Vertex AI.

Here's how Tool Use is supposed to work. You enter a prompt into Claude, such as, "What is the weather in New York." Claude interprets the prompt to produce an API call to an app that carries out the function, such as a weather app that returns weather data. The output of that app is then sent back to Claude as a message, and the model then formulates it into a natural-language response for you.

Example of a shell script that supplies Claude with a tool definition and gives a user prompt that will be interpreted by Claude to select the tool.

Which app to call, and how to pass parameters, such as the city name, is either a JSON or a Python call that the LLM can formulate.

Anthropic emphasizes that the app that does the work, such as a weather app, is not provided by Anthropic — it's provided by the developer. The LLM does not directly access the app, but rather only passes the request to the app and then receives the resulting data. Developers can either force Claude to use a particular tool, or allow the LLM to select a tool by interpreting the prompt.

Also: How LangChain turns GenAI into a genuinely useful assistant

The three different versions of Claude, called Haiku, Sonnet, and Opus, have different degrees of sophistication in how they form tool requests, Anthropic explains:

Opus is able to handle the most simultaneous tools and is better at catching missing arguments compared to other models. It is more likely to ask for clarification in ambiguous cases where an argument is not explicitly given or when a tool may not be necessary to complete the user request. Haiku defaults to trying to use tools more frequently (even if not relevant to the query) and will infer missing parameters if they are not explicitly given.

That basic construct can be extended to many paradigms, such as database queries for "retrieval-augmented generation," or RAG, a common approach to ground Generative AI in a known good source of data.

Anthropic featured several clients who have been using Tool Use. Online learning assistant StudyFetch used Tool Use to offer students things such as navigating course materials via Claude. A startup called Hebbia used the technology to do things such as extract metadata from long documents and automate "multi-step workflows" for clients in financials services.

Artificial Intelligence

Using AI and Robots to Advance Science

Even though we invented it, humans can be pretty bad at science. We need to eat and sleep, we sometimes let our emotions regulate our behavior, and our bodies are easily and irreparably damaged – all of which can stand in the way of scientific achievement.

Casey Stone
Credit: Argonne National Laboratory

Human researchers will always play a role in science, but recent developments out of Argonne National Laboratory make the case that we should let robots do some of the work. Specifically, Argonne researchers are

working on what they refer to as “autonomous discovery.” The lab hopes to increase productivity in science by relying on physical robots programmed with versatile AI software.

Casey Stone, a Computer Scientist and Bioinformatician at Argonne, recently gave a speech about autonomous discovery at an Argonne Outloud conference.

“Autonomous discovery will help individual scientists conduct more experiments and reach results faster,” Stone said in an interview after her speech. “In complex and large-scale experiments, robotics can run the experiments overnight and the experiments can be parallelized across multiple copies of the same robot. This would free up scientists’ time and allow them to focus on coming up with other creative solutions or focusing their lab time on smaller scale investigations that might lead to new hypotheses.”

The concept of robots doing hard or boring work has enraptured science fiction writers for decades, but actually achieving it is a difficult task. Stone outlined the challenges currently facing researchers as well as the opportunities that autonomous discovery presents.

Software and Hardware Modularity

One of the main challenges facing scientists who want to dive into autonomous discovery is a need for a high amount of modularity in both the software and hardware involved. Stone pointed out that Argonne scientists are working on complex problems that span many areas of experimentation, and as such they need their robots to be as flexible as possible to adapt to the changing needs of an experiment.

For the hardware, Argonne places each robotic instrument on its own cart. Each cart contains all of the computing power and sensors needed to make the instrument work and ensure that it functions as designed. The beauty of this system is that each instrument is self-contained, so the scientists can unhook the cart from the rest of the instruments, roll it away, and roll in another instrument without disrupting the rest of the system.

PF400/300 Sample Handler Robotic Arms
Credit: Precise Automation

This modularity also allows scientists to use more of the instruments they need. If a specific robot is taking longer than others in the process, researchers can hook in more of that same type of robot to the system to parallelize that step and increase speed.

Stone stated that the current cart system is just the first iteration of hardware modality. She spoke about a future laboratory where humans don’t need to roll the instruments into place themselves.

“Instead, the instruments are located on mobile platforms that can roll themselves into formation based on the needs of an experiment,” Stone said. “In a situation like that, we could take advantage of optimization algorithms to arrange the instruments in the optimal way to complete the experiment as fast as possible.”

For software, Stone stated that the code for each instrument is contained in individual sections on Argonne’s AD-SDL GitHub repository. For example, all the code needed to control the PF400 robotic arm can be found here, while all the code needed to control the OT-2 liquid handling robot is here.

Keeping this code separate for each instrument makes it easier to set up robotic labs because the researchers only need the code that is relevant to the setup of the instruments they currently want to use.

The combination of the instrument itself, the associated code from the GitHub repository, and the computers/sensors needed to allow the instrument to function is called a module. Stone stated that a module is a self-contained unit that can be added or removed from the overall robotic laboratory like a Lego brick.

Each module broadcasts certain information to the rest of the system – like what actions it’s able to complete, if it is ready to receive a command, and what resources it has available. Each module can receive commands, execute the command, and then indicate when the command has been completed. Then, a REST API server handles the distribution of experimental actions to the instruments in the correct order. The server waits for each command to complete before sending the next command.

OT-2 liquid handling robot
Credit: Opentrons Labworks

“This way, each instrument functions completely independently, and the server is responsible for integrating them together,” Stone said. “If you remove one instrument and replace it with something else, there are minimal code changes needed to get the system up and running again. “

Stone also noted that scientists do not need to employ the hardware modularity strategy to take advantage of the software modularity. These resources were designed to be as versatile as they are useful, and Argonne researchers are working hard to remove as many roadblocks as possible.

This mentality of sharing resources cuts to the heart of Argonne’s work with autonomous research. All of the software Argonne has developed here is open-source, and Stone underscored the collectivist nature of this work.

“As a national lab, our goal is to make discoveries and to spur innovation, rather than to profit from our scientific discoveries, “Stone said. “We try to make scientific advancements accessible and beneficial to the communities around us. Making our code open-source enables other groups to bring automated discovery into their scientific process, even if they may not have the funds to pay for the more expensive proprietary solutions for scientific instrument integration.”

While this dedication to the advancement of science itself is noble in its own right, Argonne scientists are helping themselves by helping others. By making this code open-source, researchers can develop a collective knowledge base around robotics and instrument integration. Any scientist who uses this code can contribute to the same software stack and build on the discoveries of others.

Humans Still Run the Show

This kind of autonomous research is exciting, but it's important to note here that humans aren’t being written out of the scientific process. Stone stated that humans will still play a crucial role in every step of the research journey.

In Stone’s mind, an autonomous research experiment would begin with a human scientist formulating a research question or hypothesis. Then, the scientist would direct AI to train on relevant data. The researcher would have to check to make sure that the AI output is logical. Additionally, the scientist would perform tasks like fixing instrument errors or supplying more labware to the system.

Once the robotic experimentation is complete, a scientist would check that the data produced is of sufficient quality before the data is passed back to the AI to update the models. Finally, when the autonomous discovery loops have completed many rounds and reached a result, the researcher can further validate the results with manual tests.

A potential area that autonomous discovery could advance is the study of antimicrobial peptides. These are small proteins that help organisms like humans protect themselves from infections by acting like natural antibiotics.

Peptides are made up of a sequence of amino acids, and there are 20 common amino acids. If a scientist wanted to design an antimicrobial peptide with a length of 10 amino acids (which is short for antimicrobial peptides) and there are 20 amino acids to choose from, the scientist would end up with 2010 total possibilities for peptide sequences. This is more than 10 trillion possible sequences to test.

Of course, a knowledgeable scientist would be able to narrow down some of these possibilities, but the fact remains that it would be nearly impossible to run all the experiments necessary to reach an optimal result using traditional methods.

This is where autonomous discovery could be immensely helpful. The scientist could train the AI on large amounts of data related to known antimicrobial peptides and their sequences. Then, the AI would learn patterns in those sequences that might contribute to their effective antimicrobial nature. Thus, the AI software would narrow down the number of sequences to test. After this, the scientist could hand off the physical operation of these experiments to one of the robotic carts described above. If the researcher discovers a physical bottleneck, or if one of the carts isn’t working properly, they can swap hardware in and out as needed.

Humans will always play a crucial role in research. However, our scientific progress over the years has been tied to the tools we use. Versatile AI software with modular robotic hardware both combine to form one of the most revolutionary tools science has ever seen.

As these autonomous discovery systems become more capable, they may one day make leaps of scientific understanding that were previously unimaginable to the human mind alone.

AI-powered headphones would let you listen to just one person in a crowd

Training the AI system to listen to one person's voice

Imagine you're in a crowded room with multiple people talking, and you're trying to listen to just one specific person. That's a challenging situation we've all faced. Now, a team at the University of Washington has created a technology aimed at addressing this challenge.

As reported in a UW news release, the team has designed an AI system that lets someone wearing off-the-shelf headphones listen to just one person in a crowd of people. To enroll a person's voice, you simply look at them once for three to five seconds. The system, known as "Target Speech Hearing," can then block out all other voices and sounds in the area and let you listen just to the person you enrolled. You can even move around and away from the speaker and continue to hear just their voice.

Also: The best headphones for music you can buy: Expert tested

Here's how the system works.

Wearing any pair of headphones outfitted with dual microphones, you tap a button while looking at someone who's speaking. The sound waves from that person's voice hit the microphones on both sides of the headset. That signal is sent to the system's on-board computer, where the embedded AI learns the speaker's voice patterns. The system then picks up the voice and continues to play it back to you. The longer the person speaks, the more the system learns and adds to its training data.

Current headphones and earbuds already offer noise cancellation features and other options to help you better hear specific sounds. Apple's AirPods Pro, for example, provide noise control settings in which you can muffle sounds around you to focus on the audio piping through the earbuds. You'll also find features such as Personalized Volume and Conversation Awareness, both aimed at automatically adjusting the audio volume. An accessibility setting in iOS called Conversation Boost can amplify the conversations of nearby people. Plus, iOS 18 is reportedly gaining a hearing aid mode to help if you have trouble hearing.

The system developed by the UW team promises to expand this type of capability, especially since it's designed to work with any pair of headphones.

Also: The best headphones I've ever listened to are not by Bose or Sony

"We tend to think of AI now as web-based chatbots that answer questions," senior author and UW professor Shyam Gollakota said in a statement. "But in this project, we develop AI to modify the auditory perception of anyone wearing headphones, given their preferences. With our devices, you can now hear a single speaker clearly, even if you are in a noisy environment with lots of other people talking."

So far, the team has tested its system on 21 different people, who rated that the clarity of the enrolled speaker's voice was almost twice as high as unfiltered audio.

Also: The best headphones for 2024: Expert tested

The system has some limitations.

For now, you can enroll just one speaker at a time, and only when there isn't another loud voice coming from the same location. Further, the system works only with headphones, although the team is working to support earbuds and hearing aids. Finally, the system itself isn't commercially available. Rather, the code for the device is available for other developers to examine and use.

To learn more about the system, check out the team's presentation and report delivered on May 14 in Honolulu at the ACM CHI Conference on Human Factors in Computing Systems.