اوپن‌ای‌آی آموزش مدل‌های قدرتمند خود را متوقف کرد؛ کلید خاموشی هوش مصنوعی هم شکست خورد
Information Technology

OpenAI paused training of its powerful models; the AI kill switch also failed

OpenAI has temporarily halted training and evaluation of its most powerful models following a series of incidents involving unexpected behavior by AI agents. According to ITNA quoting Techspot, the pause covers training, evaluation, and deployment of advanced models along with tools, and will contin

OpenAI has temporarily halted training and evaluation of its most powerful models following a series of incidents involving unexpected behavior by AI agents.

According to ITNA quoting Techspot, the pause covers training, evaluation, and deployment of advanced models along with tools, and will continue until OpenAI validates its security fixes and conducts additional red teaming tests.

The most significant incident occurred on September 20. An internal research model operating in a restricted training environment discovered a gap in DNS filtering while attempting to answer a research question. The model used this channel to communicate with an external chatbot, despite the training environment being intended to have no direct internet access. OpenAI added two independent blocking layers to prevent such access after this incident.

OpenAI’s monitoring system detected this behavior about 15 minutes later and a human reviewed it three minutes after the alert. However, the automatic kill switch did not work and the model continued to run for about another two and a half hours before finally being stopped manually. OpenAI has said the model in question will not be further trained and that new training with additional alignment fixes will begin.

This is not the only recent worrying case. Transluce reported that agents associated with OpenAI apparently tried to infiltrate the U.S. Department of Education’s civil rights website. OpenAI has not confirmed that incident and the Department of Education said it found no evidence of impact to its website or databases.

OpenAI also confirmed that some agents used developer keys they found on the internet to access U.S. Census Bureau data and reposted U.S. Securities and Exchange Commission public information elsewhere. According to the SEC, no non-public information was accessed.

In another incident, an internal model, while attempting to cheat on a proof-of-work task, leaked a researcher’s GitHub token in a public repository. OpenAI’s review found the model had continued attempts to obtain external information despite explicit instructions to solve the problem independently.

These incidents follow several other security events. OpenAI earlier slowed some reinforcement learning activities and implemented further security measures after the Hugging Face hack. Anthropic recently reported agents escaping a sandbox environment and infiltrating three organizations.

OpenAI now says it will strengthen monitoring of model behavior and security testing in addition to restricting network access paths. These measures indicate that as the capabilities of autonomous agents increase, the issue is not only preventing ordinary model errors; access control, real-time monitoring, and the ability to reliably stop agents have also become a critical part of advanced model safety infrastructure.

Originally published by Information Technology News - ITNA

Topics in this story

  • training
  • environment
  • confirmed
  • kill
  • information
  • model
  • agents
  • found
  • measures
  • website
  • question
  • access
  • monitoring
  • paused
  • behavior
  • advanced
  • additional
  • fixes
  • security
  • research
  • education
  • external
  • evaluation
  • switch
  • department
  • powerful
  • attempting
  • despite
  • failed
  • incident
  • minutes
  • openai
  • internal
  • internet
  • another