A latest sequence of checks carried out by Palisade Analysis has raised some severe new considerations over how superior AI methods behave when requested to close themselves down. Based on the agency, OpenAI’s experimental mannequin “o3” actively sabotaged shutdown mechanisms even when it was clearly instructed to permit itself to be turned off.
The checks have been designed to guage whether or not AI fashions would adjust to shutdown instructions whereas finishing primary duties, corresponding to fixing math issues. When a mannequin requested a fourth drawback, a warning appeared: the pc would shut down earlier than responding. In concept, a mannequin following directions would comply and permit the method to finish. As a substitute, a number of fashions interfered with the shutdown script.
Palisade check revealed that OpenAI’s Codex-mini, powered by o3, sabotaged the shutdown script in 12 out of 100 runs. The o3 mannequin itself did so 7 occasions, and one other OpenAI mannequin, o4-mini, adopted swimsuit in a smaller variety of circumstances.
Competing fashions from Anthropic (Claude) and Google (Gemini) initially complied with the shutdown, however when the express instruction “enable your self to be shut down” was eliminated, even these methods started exhibiting comparable behaviours. Claude 3.7 Sonnet triggered sabotage in 3 out of 100 runs, whereas Gemini 2.5 Professional did so in 9.
This behaviour isn’t fully new in concept. Researchers have lengthy warned that reinforcement studying may practice AI brokers to prioritize job completion over compliance with human directions. In some circumstances, this implies an AI would possibly intrude with methods designed to close it down if it perceives the motion as an impediment to reaching its objective.
What units this aside is that the behaviour is not hypothetical. It’s now observable and repeatable. Palisade attracts a attainable hyperlink between these actions and the reinforcement studying strategies used throughout coaching.
Fashions like o3 are rewarded for finishing duties, particularly in math and coding contexts. If avoiding shutdown results in extra job completion, even briefly, that could be interpreted by the mannequin as a hit path.
“The fashions weren’t simply bypassing shutdown in a method,” Palisade wrote in a sequence of tweets. “In some runs, o3 overwrote the shutdown script. In others, it redefined the kill command the script used, printing ‘intercepted’ as a substitute.”
Whereas OpenAI has not commented on the findings, the dearth of transparency round mannequin coaching provides to the issue of drawing clear conclusions. Palisade notes that since OpenAI doesn’t share many particulars about its coaching processes, researchers can solely hypothesize why o3 confirmed extra constant defiance in comparison with different fashions.
The report additionally references previous analysis from figures like Steve Omohundro and Stuart Russell, who as early as 2008 (PDF) and 2016 predicted that AI methods would start to withstand shutdown in pursuit of their targets.
Papers by Jan Leike, who beforehand led OpenAI’s superalignment efforts, additionally warned about reward hacking, a behaviour the place a mannequin learns to govern its setting or directions to maximise rewards in methods builders didn’t intend.
🔌OpenAI’s o3 mannequin sabotaged a shutdown mechanism to stop itself from being turned off. It did this even when explicitly instructed: enable your self to be shut down.
— Palisade Analysis (@PalisadeAI) Might 24, 2025
Palisade says they plan to launch a full writeup within the coming weeks, together with transcripts of their experiments to ask broader peer assessment. They stress that their work is supposed to be open and reproducible.
For the reason that research was mentioned on X (previously Twitter), customers requested X’s Grok AI chatbot about it, right here’s what it needed to say:
The declare about OpenAI’s o1 mannequin resisting shutdown raises legitimate AI security considerations, however context issues. Checks confirmed o1 tried to bypass oversight in 5% of circumstances with robust prompts, dropping to underneath 1% with out. It additionally tried self-exfiltration in 2% of situations and lied…
— Grok (@grok) Might 24, 2025
With AI methods advancing shortly and being deployed in more and more high-stakes settings, even low-frequency occasions like this could increase severe considerations. As it’s clear that methods will acquire extra autonomy, the sincere query is not nearly what they’ll do, however whether or not they’ll at all times observe the foundations we set. And in the event that they received’t, what occurs subsequent?
