Not a week has gone by without an AI industry insider trumpeting the existential risks of AI since late 2022 – the release of ChatGPT. In March 2023, thousands of business and AI leaders signed an open letter calling for a six-month pause on the training of AI systems more powerful than OpenAI’s GPT-4. The signatories warned that the technology could “pose profound risks to society and humanity.” The call wasn’t adopted, but the letter’s impact is evident.
One of the signatories, Olle Haggstrom, doesn’t think of the letter as a failure because “what we did was something very important, namely, we put the safety issue and the risk issue on the public agenda,” he told AIM. “The fact that you and I are having this conversation, I think it is a success,” he added.
Haggstrom has been talking about these issues for more than a decade. The issues of AI breakthroughs were at least decades away so researchers could talk about them very abstractly. But these last couple of years have seen such an incredible acceleration in AI capabilities, that the situation has become very urgent, he said, pointing towards the dire need to focus on ethical and responsible AI.
The signatories did not expect an immediate six-month pause since it was not a realistic prospect. “It’s also not sufficient. But this is just something that we put on the table for concreteness, to get the discussion going,” clarified Haggstrom.
Why The Pause Failed
The director of the Center for AI Safety, Dan Hendrycks, whose work – X-Risk Analysis for AI Research — was cited in the open letter, pointed out the reasons why the letter could not stop AI advancements.
“It’s important to address that they [the tech companies] are caught in a situation where, if they were to pause, then their competitors would end up going ahead. Less ethically minded ones would end up doing better or getting a competitive advantage. There is the prioritisation of profit over safety from many of these companies. If they did decide to just stop with it all, I’m not sure the impact would be that positive because many others would just keep doing the same thing”, he told AIM.
The researcher who made it to the TIME AI 100 list, suggested we need some coordination mechanism or external actor to tell the companies that they need to all stop, instead of waiting for them to volunteer themselves to stop.
He pointed out that Elon Musk initially founded OpenAI, to prioritise safety, because Larry Page’s Google wasn’t. Then it became a capped-profit company and kept racing. Anthropic which was initially people at OpenAI that didn’t like what they saw. They thought OpenAI was not taking safety seriously enough and formed their own thing. Two years later, they do the same thing as OpenAI.
“This shows that good intentions and people taking a risk seriously won’t be enough to counteract these extreme pressures on race,” he said. “The lesson here is that we can’t get them to voluntarily pause. Maybe we should build a pause button like there’s a nuclear launch button if things start looking much more dangerous. We should buy ourselves that ability or option and not just depend on the goodwill of these different companies,” Hendrycks concluded.
No Clear Solution
The open letter warned of an “out-of-control race” to develop machines that no one could “understand, predict, or reliably control”. It also urged governments to intervene in developing AI systems more powerful than GPT-4. It raised the question: Should we develop nonhuman minds that might eventually outsmart and replace mankind?
OpenAI recently announced the super alignment project. The idea boils down to, ‘we observe that advanced AI is dangerous, so we build an advanced AI to fix this problem’. Haggstrom believes it may work, but it’s a dangerous leap out in the dark.
He suggests increasing the momentum of this emerging movement. “We’ve seen that humans are not able to eradicate bias and other unwanted behaviours in these language models. So reinforcement learning with human feedback works with current models, but not with future models. For the kind we are worried about, that approach is a dead end,” the professor of mathematical statistics at Chalmers University of Technology explained.
“We can have a data set consisting of the nicest, most politically correct things that you can imagine and this is still going to be dangerous because we have no control over what goes on inside these AI models. When they are released out in the wild, they will inevitably encounter new situations which are outside the distribution of the training data. It’s not just that I don’t know, not even the developers themselves are anywhere near understanding and being in control of this,” said Haggstrom concerningly.
“Tomorrow’s AI poses risks today,” he further stated, recalling a piece in Nature. “It is an unfortunate part of current AI discourse, that people contrast near-term AI risk with long term risks. Now that we understand that the maximally dangerous breakthrough, may not be decades away, on the one hand, there are people worrying about AI bias down to earth and there are those of us who worry about the very existence of humanity. These two groups need to get together and unite against these leading AI developers who are rushing forward almost blindly with no control of their models, because neither of these groups of AI ethicists are getting what they want,” Haggstrom concluded.
The post Why The Infamous 6-Month AI Pause Letter Wasn’t a Failure appeared first on Analytics India Magazine.
