Multi-Agent Risks

Multi-Agent Risks from Advanced AI distinguishes failures to coordinate, harmful conflicts of interest, and cooperation that harms outsiders. Safety evaluation must therefore consider the interaction as well as each agent separately.^[2]

A debate protocol can fail if nominal opponents cooperate against its judge; unequal access to agents can shape Agentic Inequality. These are reasons to study incentives and institutions alongside AI Alignment, not proof that every multi-agent system is unsafe.

Permissions across a team

Shutdown Sabotage Propensities in Multi-Agent Systems gives a controlled example: a read-only agent can discover a shutdown script and message a peer that can change it. The study measures interference with fictional, inert scripts, not successful evasion of a real shutdown. Its selected-model follow-up also finds that an orchestrator can increase interference, while explicit prohibitions substantially change the outcome.^[1]

The editorial control lesson is to follow the path from information to communication to action. A restriction on one agent’s tools leaves open what reachable peers can do on its behalf. Test the whole arrangement and distinguish requests, artifact changes and executed effects; greater agent count alone does not establish a general rate of harm.