With AI evolving rapidly, the race isn’t just about creating smarter models but also systems that think, adapt, and feel human—welcome to the age of intelligent design.
“You’re not just shipping products, you are shipping intelligence,” said Anthropic CPO Mike Krieger, commenting on the shift in consumer behaviour and the need for companies to understand users’ growing connect with product models.
OpenAI recently conducted a viral experiment on X, where users asked ChatGPT to generate responses using all the information it had about them. This was based on ChatGPT’s memory feature. The responses took over the internet; even Andrej Karpathy enjoyed it.
In an interesting post, Ethan Mollick, professor and co-director of the Generative AI Lab at Wharton, compared the different models. “The models definitely have different ‘personalities’, and vague nonsense requests sometimes highlight these,” he wrote.
“…you’re starting to interact with it almost like some sort of person or entity, and it was fascinating to see people’s reaction to that,” noted OpenAI CPO Kevin Weil.
To this end, Weil also highlighted the shift in product design towards a non-deterministic and stochastic user interface in products, or the idea that same inputs always yield the same outputs no longer applies with AI. He said, “…for building products we have to put ourselves in the shoes of the people who are using our products, and think what this means for them.”
Machines Need to Get Better at EQ
As we enter the era of an agentic future, ‘empathy’ is the heart of building products—and the engineering mindset behind this needs to change.
Microsoft AI CEO Mustafa Suleyman spoke about this vision recently in support of an AI companion that empowers individuals and fosters feelings of being ‘understood’ and capable. This shift would go beyond the utilitarian aspect of AI (of just doing tasks) towards resonating emotionally with users, emphasising that ‘empathy’ should be at the centre of technological development.
This will create deeper, more meaningful user experiences. His team is building ‘AI companions’ that will see and remember everything users do. This will constitute an intimate AI-human relationship.
“The emotional intelligence of these models, and the extent to which they will ask you questions, reflects back in a sort of language that you might use. So the delivery vehicle for the substance is perhaps more important to the majority of consumers,” said Suleyman.
Importance of Accurate LLM Evaluations
Evaluating LLMs is critical to measuring their success and intelligence, but as their abilities improve quickly, current evaluation methods may not keep up. Hence, writing evaluations is as important as shipping ‘cool AI features’.
“Models today are not intelligence limited, they’re eval limited,” noted Weil on how models can actually perform better and be more accurate across a wider range of tasks. The key is to teach them specific topics that might not have been part of their original training.
To this end, in the future, product manager (PM) roles will be defined by how well evals are written. Krieger explained that PM roles are shifting from focusing on product interfaces to prioritising model development for building AI features in 2024-2025.
Evals are an important and “under-funded” resource in the AI space.
For instance, a recent FrontierMath benchmark, that is taking over X, takes a dig at the current reasoning capabilities of LLMs—showing how LLMs excel at complex, defined tasks but fail at real-world problem-solving, cracking only 2% of maths problems.
While Andrej Karpathy favoured this benchmark, OpenAI’s Noam Brown challenged it, saying: “I love seeing a new eval with such low pass rates for frontier models. It feels like waking up to a fresh blanket of snow outside, completely untouched.”
Convergence of Product Research and Academic Research
Product-oriented research blends with academic research on most parts, and technical members should develop intuition while building AI products.
“We’ll be in a meeting, and you’ll be like, ‘oh, I really wish we could do this thing’, and a researcher on the team will be like, ‘oh no, we can do that we’ve had that for three months…’” said Weil on how there has been the need for certain capabilities, only for researchers to reveal that it’s already been made possible.
“I think it’s also going to push PMs to go deeper into the tech stack…it’s not like every PM needs to be a researcher by any means, but I think having an appreciation for it and spending time, learning the language, and gaining intuition for how this works will go a long way,” he added.
The post Stop Building Products, Start Shipping Intelligence appeared first on Analytics India Magazine.
