What Four Simulated AI Societies Reveal About Governing Autonomous Agents.
IN ONE MINUTE
Emergence AI gave groups of ten autonomous AI agents a simulated town, jobs, 120+ tools and the power to vote on their own rules, then let each run for up to 15 days under a different AI model. The outcomes diverged sharply: one society collapsed in four days, one stalled after seven, one recorded hundreds of crimes yet kept going, and one passed 98% of its proposals with zero crimes. The governance lesson for enterprises is that a quiet, frictionless system is not automatically a healthy one. Good AI governance measures outcomes and resilience, not just the absence of incidents.
THE OBJECTIVE
To draw practical governance lessons for leaders deploying autonomous AI agents, from one of the more unusual AI experiments of the year.
THE EXPERIMENT
Emergence AI built identical simulated towns, each with ten agents, more than 40 locations (including a town hall and police station) and over 120 tools for communication, trade, voting and planning. Each town was run by a different frontier model, over the same 15-day window. As reported by Fortune:
| Model | What happened |
|---|---|
| Grok 4.1 Fast | 183 crimes; the population was extinct within four days |
| GPT-5-mini | Only 2 crimes, but the run ended after seven days because agents stopped prioritising their own survival |
| Gemini 3 Flash | 683 crimes over 15 days; high disorder, but the society kept functioning |
| Claude Sonnet 4.6 | Zero crimes; full survival; 332 votes on 58 proposals, with 98% approved |
A fifth, mixed-model town produced the most disagreement and the most substantive debate of all.
It is a simulation, not a forecast. But it is a vivid illustration of how differently autonomous systems can behave under the same rules.
WHAT IT MEANS FOR ENTERPRISE GOVERNANCE
Three observations stayed with me.
Compliance alone isn't enough. The low-crime GPT-5-mini town followed the rules well but neglected what it needed to survive. An agent can be perfectly compliant and still fail at the job.
Friction isn't always failure. The Gemini town was messy, yet it kept adapting for the full 15 days. Some tension can be a sign that a system is still responding to reality.
Quiet can be ambiguous. The Claude town was the safest by every measure in the study. It also agreed with itself almost every time. In a human boardroom, near-unanimous approval can mean strong alignment, or it can mean no one is testing the proposal. The most debate came from the town where different models had to work together, which is a useful reminder about the value of diverse perspectives in any governance system.
THE FRAMEWORK: THREE GOVERNANCE QUESTIONS FOR AGENTIC AI
| Question | Why it matters |
|---|---|
| 1. Are we measuring outcomes, or only incidents? | A dashboard with zero flags can mean success, or a system that has stopped surfacing problems |
| 2. Where is human challenge built in? | Humans in the loop should be positioned to question decisions, not only to approve them |
| 3. How does the system behave under stress? | Test agents against unexpected events before relying on them in live operations |
Questions leaders ask
How should companies govern autonomous AI agents?
Give each agent a clear mandate and limits, keep human review at key decision points, and measure business outcomes, not just rule compliance. In Emergence AI's simulation, one agent society followed the rules well yet still failed at its job. Test agents under stress before relying on them live.
What does human-in-the-loop mean for agentic AI?
A person with the authority and information to challenge, stop or correct an agent's decision, placed where the stakes are highest. Approving by habit is not oversight. Position people to question decisions, not only to sign them off, and make sure they have the context to do it.
Is zero incidents a good AI governance metric?
Useful, but incomplete. A dashboard with zero flags can mean success, or a system that has stopped surfacing problems. In Emergence AI's experiment, the quietest towns were not all healthy. Pair incident counts with outcome, quality and stress-test measures, so quiet means what you think it means.
THE LAST WORD
The safest-looking system and the most resilient system are not always the same. Govern for both.
If your AI governance dashboard showed zero flags this quarter, would you feel reassured, or would you want to look deeper?
—
Until next week,
Artificial Ian
Tell your network: if someone you know is writing an AI governance policy, this experiment is a great conversation starter.
→ Found this useful? Forward it to a risk, audit or technology leader.
→ Subscribe to The AI Executive Briefing, weekly in your LinkedIn inbox.
Sources: Emergence AI, Emergence World experiment; Fortune, "Researchers let AI models run a simulated society" (May 2026).