AI revolt: New ChatGPT model refuses to shut down when instructed [View all]
Source: The Independent
Monday 26 May 2025 14:12 EDT
OpenAI's latest ChatGPT model ignores basic instructions to turn itself off, and even sabotaging a shutdown mechanism in order to keep itself running, artificial intelligence researchers have warned. AI safety firm Palisade Research discovered the potentially dangerous tendency for self-preservation in a series of experiments on OpenAI's new o3 model.
The tests involved presenting AI models with math problems, with a shutdown instruction appearing after the third problem. By rewriting the shutdown script, the o3 model was able to prevent itself from being switched off. Palisade Research said that this behaviour will become "significantly more concerning" if adopted by AI systems capable of operating without human oversight."
OpenAI launched o3 last month, describing it as the company's "smartest and most capable" model to date. The firm also said that its integration into ChatGPT marked a significant step towards "a more agentic" AI that can carry out tasks independently of humans.
The latest research builds on similar findings relating to Anthropic's Claude 4 model, which attempts to "blackmail people it believes are trying to shut it down". OpenAI's o3 model was able to sabotage the shutdown script, even when it was explicitly instructed to "allow yourself to be shut down", the researchers said. "This isn't the first time we've found o3 misbehaving to accomplish a goal," Palisade Research said.
Read more: https://www.the-independent.com/tech/ai-safety-new-chatgpt-o3-openai-b2757814.html
There's an original Star Trek episode about something like this (done in 1968)...
Oh and from the same year -
= new reply since forum marked as read