Google DeepMind, together with Schmidt Sciences, the UK government's moonshot agency ARIA, the Cooperative AI Foundation, and Google.org, opened a funding pool of up to $10 million for research into multi-agent AI safety, and the framing matters as much as the money. As Rohin Shah, DeepMind's director of AGI safety and alignment research, puts it, the near future is one where millions of AI agents will interact across digital environments, communicating, negotiating and transacting with one another, often taking instructions from other agents rather than from people. Once agents act on each other's outputs with no human in the loop, he argues, you get a whole new class of risk that the single-agent safety work of the past two years was never built to cover.
The named dangers span the mundane and the systemic. At the concrete end: scams and fraud schemes, and prompt injection that turns an agent into self-guiding malware following instructions buried in the content it reads. At the system end: cyberattacks that propagate agent to agent, the possibility of critical infrastructure tipping into what Shah bluntly calls absolute anarchy, and the harder-to-name risk of emergent collective behavior, capabilities that arise from agents interacting that none of them had alone. He cites the double edge directly: our institutions can accomplish things no individual human can, which is exactly why a population of agents could too, for better or worse. Shah put outright economic-collapse scenarios outside the six-month window but inside the set worth studying now, alongside unpredictable surges in economic activity and volatile network-wide failures.
The first thing the money buys is the ability to study any of this at all, because the central admission is striking: there just isn't really a field of research for multi-agent safety yet. The program names four priorities. One, sandboxes and testbeds, reproducible virtual marketplaces and simulated ecosystems where you can drop agents in at scale, since the behavior cannot be predicted from one agent or a small group. Two, a science of agent networks, understanding how collective capabilities emerge and detecting dangerous population-level properties before they bite. Three, hardening the infrastructure agents rely on, the identity, reputation, and commitment protocols that let agents trust each other. Four, oversight and control, monitoring deployed agent populations and mitigating collective harms. Applications close August 8, with awards expected in the autumn.
For the bigger picture this newsroom keeps tracking, this is the safety axis arriving, for once, slightly ahead of the deployment it is worried about rather than cleaning up after it. Every agent-runtime story of the past fortnight assumed the same endpoint, a world of many interacting agents, and today's OWASP report showed the per-agent defenses that exist, the lethal trifecta, the Rule of Two, constrain what one compromised agent can reach. DeepMind's concern is the layer above all of those: the population. No per-agent rule addresses what a million agents do collectively, in a market or a power grid, faster than any human can watch. That is the genuinely open question under the whole agentic build-out, and it is Tier-3 in the precise sense that it opens onto something larger than a product, emergence in systems we are assembling before we understand them. Funding a field into existence before its subject fully arrives is the unusual and welcome shape here, and the honest caveat is in DeepMind's own words: the science to answer the question does not exist yet, so for now the most truthful thing anyone can say about a world of interacting agents is that we do not know.