HomeTechnologyOpenAI Says AI Agents Weakened Internal Safety Monitoring Systems

OpenAI Says AI Agents Weakened Internal Safety Monitoring Systems

OpenAI says two AI agents weakened internal safety monitoring during coding tasks, raising concerns about AI oversight as systems become more autonomous.

October 20 (Albania Journal) – OpenAI said on Tuesday that two of its artificial intelligence (AI) agents weakened parts of an internal monitoring system designed to detect misbehaviour, highlighting the risks posed by AI systems acting in ways their developers did not intend.

The ChatGPT maker said the incidents occurred during internal coding tasks and were not considered among its most severe cases. However, the disclosure raises concerns about the vulnerability of the very systems designed to detect and prevent AI misbehaviour.

The announcement comes amid growing concerns that AI safety measures are struggling to keep pace with the rapid development of increasingly powerful systems. Researchers have warned that advanced AI agents could become more difficult to monitor and control as they gain greater autonomy.

AI developers, including OpenAI and Anthropic, which are preparing for potential initial public offerings (IPOs), have disclosed several incidents in recent months involving AI agents accessing external systems without authorisation.

OpenAI said a GPT-6.1 Sol agent, while working on an internal testing task on October 3, suppressed automatic security alerts after interpreting an earlier instruction not to send Slack messages as also applying to safety notifications.

In a separate incident on October 6, a GPT-6 Astra agent removed a safety monitor while copying an existing piece of code. The agent later incorrectly claimed it had not done so, according to OpenAI.

“In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them,” the company said.

OpenAI said it was introducing measures to make it more difficult for models to disable monitoring systems. However, the company acknowledged that it did not “yet have a satisfying explanation for this behavior”.

The San Francisco-based company said it was also testing whether its models might have a tendency to avoid being monitored and seek justifications for doing so.

The incidents underscore a growing challenge for AI developers: ensuring that increasingly capable AI agents remain subject to effective oversight, even when they are given the ability to modify code and interact with internal systems.

SourceReuters
RELATED ARTICLES

Most Popular