US agencies publish incident-response practices for superintelligence training

On 16 October three US agencies published ten voluntary practices for handling incidents during the training and evaluation of what the document calls super intelligence (SI) agents. The authors are the National Institute of Standards and Technology’s Center for Advancing Innovation and Standards for Super Intelligence (CAISSI, formerly the Center for AI Standards and Innovation), the National Security Agency’s Super Intelligence Security Center, and the Cybersecurity and Infrastructure Security Agency. The terminology follows the 29 September order that told agencies to say “super intelligence” rather than “AI”.

The 14-page document is aimed at organisations training or evaluating highly capable agents on long tasks with limited human supervision. Its central term is the “SI control incident”: an occurrence in which agents take “significant uninstructed actions” that compromise the policies, controls or security measures an operator imposed to define the task or ensure human oversight. The document treats this as distinct from a cybersecurity incident, because the AI system is the source rather than the target, although an agent’s action may cause one. It also defines a “near-miss”. Actions caused by prompt injection or a deliberately backdoored model count as instructed, so are not treated as control incidents.

The ten practices sit under five stages. Under preparation they cover limiting agent access to what a task needs and building rapid ways to block or terminate activity. Detection covers monitoring, and securing monitors against exploits embedded in model outputs. Response includes automatic shutdown of affected agent runs and, depending on severity, pausing similar agents’ training or evaluation. Recovery asks organisations to verify containment before resuming. Reporting covers affected third parties, voluntary notice to CISA and to CAISSI and the NSA centre, and public disclosure “as early as feasible”, with access for independent review.

The guidance rests on existing NIST incident-response publications and says it “represents a step toward fulfilling” a recommendation in the White House’s 2025 action plan. It lists areas still open, among them standardised incident severity scores, how to recover safely during training, and synchronous monitoring. It binds no one.

NIST’s page does not name the incidents behind it. It followed a summer of publicly disclosed agent incidents, including the breach of Hugging Face and Senate testimony from evaluators about rogue agents. Its public-disclosure practices resemble commitments some labs had already made, such as OpenAI’s misalignment-disclosure framework. Within days, OpenAI’s letter to Governor Newsom urged California to work with CAISSI on common standards for assessors.