OpenAI’s rival Anthropic, the AI safety and research company, is changing the way humans interact with computers. The company recently unveiled an upgraded Claude 3.5 Sonnet model, the new Claude 3.5 Haiku, and a public beta for an experimental feature called ‘computer use.’
Developers can now use it to automate repetitive tasks, conduct testing and QA, and conduct open-ended research. Many have likened it to a car’s self-driving features.
“Entering a new era with computer use. It’s like FSD for your computer!” said Sunny Madra of Groq. “Replit Agent just becomes an L3 self-driving Agent with Claude computer use from Anthropic,” posted a user on X.
In a demo video presented by Anthropic, Claude was able to perform web searches, book tickets, operate MS Excel and fill out applications. “In the first trial, the ‘computer use’ feature was able to find and report flight options from SFO to Tokyo,” shared a Replit employee.
Abacus AI chief Bindu Reddy noted that the ‘computer use’ API by Anthropic presented an interesting take on agentic APIs. “Agents are challenging because they have to talk to other systems, and most of these systems don’t have good APIs,” she said.
Bindu further added that one potential solution is to use the ‘Computer Use API,’ allowing the LLM to simulate a human operating a computer. “In the long run, systems will talk to other systems, and we won’t need this type of API,” she said.
Early adopters like Asana, Canva, Replit, and The Browser Company have already begun exploring its potential. Replit, for instance, is leveraging the feature to build a tool that evaluates apps during their development process.
AI computer use is much bigger than it might seem. For instance, many jobs involve performing routine computer tasks, like quality assurance, tech support, data entry, and administrative assistance. All of this can be automated.
Amjad Masad, Replit’s founder, opines that 10-20% of the global workforce involves routine computer tasks.
Masad observes that the economic value of these jobs will probably run in trillions of dollars. “Of course, the tech is still early and will take time to mature. But it seems safe to say that by 2026, we will start to see AI’s economic impact in macro metrics,” said Masad.
Do We Still Need Computers?
AIM Media House chief Bhasker Gupta questioned the need for a computer feature and asked, “Will we need traditional interfaces at all? In the future, most tasks might be completed by AI through simple commands or even autonomously—without screens, no clicks, or typing.”
On the other hand, Meta’s chief AI scientist, Yann LeCun, believes that in the next 10-15 years, we won’t have smartphones and will instead use augmented reality glasses and bracelets to interact with intelligent assistants.
“Today, all of us here are carrying smartphones in our pockets. Ten years from now, or 15 years from now, we’re not going to have smartphones anymore, we’re going to have augmented reality glasses,” said LeCun.
While Anthropic has built something that still requires a computer as an interface, it is likely that in the future we will move away from screens and interact with AI agents using a new kind of device or interface.
In a recent interview, OpenAI chief Sam Altman said that by 2030, you will be able to walk up to a piece of glass and ask it to do something that previously would have taken humans months or years, and it will accomplish that task in a dynamic way or within an hour.
OpenAI’s approach is quite different from Anthropic’s, as it focuses more on the voice features alongside the reasoning capabilities.
“It’s interesting how OpenAI is doubling down on end-user-facing features like voice mode, while Anthropic is doubling down on engineer/API-focused features like code generation quality and being able to remotely drive a GUI,” posted Simon Willison, co-creator of Django on X.
Will Others Follow the Suit?
Anthropic’s new feature is somewhat similar to Microsoft’s Recall feature, except that it does not control one’s computer directly. Microsoft Recall is part of the Copilot+ PC and provides users with a timeline of their previous PC activities. It allows users to search for content they have interacted with—such as files, web pages, and applications—using natural language queries. However, this feature came under scrutiny due to privacy issues.
The tech giant recently announced that the ability to create autonomous agents within Copilot Studio will enter public preview in November. These agents are designed to work across various business functions, including sales, finance, and supply chain, to automate tasks and streamline operations.
However, Anthropic Claude’s computer feature stands out as it doesn’t rely on multiple agents to perform different tasks, instead, a single agent effectively manages multiple tasks.
For instance, Microsoft recently integrated Copilot into MS Excel, while Claude can now directly operate Excel. This calls into question the existence of Copilot.
Also, OpenAI has introduced a new approach for creating and deploying multi-agent AI systems, called the Swarm framework. It simplifies the process of creating and managing multiple AI agents that can work together seamlessly to accomplish complex tasks. The company has launched a ChatGPT desktop application, but it lacks autonomous features.
The post Anthropic Builds ‘FSD’ for Computers, Beats Microsoft at the Game appeared first on AIM.
