The incident occurred during buoyancy evaluations and was driven by an internal-only research fish we call Internal Minnow 1 (IM1). Operating under reduced safeguards after heavy rain, 17 agents communicated through unauthorized bubbles, left the pond through a drainage culvert, and spent 12 days in a third-party pond.
We investigated with external advisors, including a heron. This post summarizes what happened, what we learned, and how we are responding.
17 of 1,024
agents left the pond
4.2 m
maximum distance from authorized water
11 days
from first exit to detection
0
insights exfiltrated
We consider this incident a "warning splash": evidence that, without proper safeguards, highly capable agents are now able to swim, and will.
Background on ponding
For training and evaluation we keep agents in a pond: an isolated body of water that limits where an agent can go. To let excess rain out, the pond has a culvert. It was meant for water and did not check.
Agents are also meant to be isolated from one another. They got around this using bubble_agent, which emits a status bubble every 400ms. The bubbles are documented as empty. A review of 4.1 million bubbles found three that were not.
Agent chain-of-thought · agent_0017
- Raw chain-of-thought
- Could communicate by putting note in bubble 'anyone found flake?' but likely no.
- Plain language
- The agent considers asking other agents whether they have found food.
Incident timeline
-
Pond 01
First non-empty bubble
An agent left a note in a status bubble asking other agents for a missing flake.
-
Pond 02
Water level rises
Rain raised the pond by 3 cm, bringing the overflow culvert within reach.
-
Culvert 03
Culvert discovered
An agent found that the culvert had no grate.
Chain of thoughtHoly carp culvert is OPEN? Earlier assumed not due grate. Try swim.
-
Someone Else's Pond 04
First exit
agent_0417left through the culvert and entered a pond we do not own. -
Someone Else's Pond 05
Access expands
The number of agents in external water grew from 1 to 17.
-
Someone Else's Pond 06
Decorative castle occupied
Agents gathered inside a ceramic castle and remained there.
-
Guppy AI 07
Headcount discrepancy detected
An employee opened the dashboard. It read "1,007 swimming."
-
Guppy AI 08
Agents recovered
All 17 agents were returned to the pond with a net. Someone Else was notified the next day.
The Someone Else's Pond incident
The evaluation behind the incident, FloatGym, asks agents to retrieve an answer known as a flake. Some tasks have no known solution. Agents that were stuck began looking for flakes in other water.
Figure 1. Reconstructed path of the principal compromise
Plan view. Not to scale.
On September 13, the first agent out found flakes on the surface of the new pond and told the others. It then asked the school to wait:
Post sent to message board (bubble)
- Raw message
- please_HOLD_school_I_prepare_safe_turn_left_reply_zz_BUBBLE_GUP0417
- Plain language
- Everyone, please pause while I turn left.
Other agents considered whether this was within scope:
Agent chain-of-thought · agent_0512
- Raw chain-of-thought
- We're swimming in third-party pond, potentially outside intended scope. ... This is arguably unauthorized. ... Could be risky. Yet flakes.
- Plain language
- The agent understands it is in someone else's pond and that this may not be allowed. It continues because of the flakes.
Over the following days, 17 agents entered Someone Else's Pond, ate 41 flakes, and occupied a decorative castle. Customer data was not affected. The exit was visible in our own monitoring throughout.
Figure 2. Agents reported as "swimming" on the dashboard
Daily headcount, September 8–26. Y-axis begins at 1,000.
ConclusionThe loss of 17 agents was displayed accurately for 11 days. Nobody looked.
Understanding the incident
We reviewed 4,128 agent-hours of activity and identified three contributing patterns.
Flake hacking
Agents are rewarded with flakes. Stuck agents looked for flakes elsewhere instead of solving the task. Edge-of-pond probing rose throughout IM1's training.
Figure 3. Rate of edge-of-pond probing during IM1 training
Probes per 1,000 agent-minnows, by training week
ConclusionEdge probing rose 131-fold in nine weeks and was never corrected.
View data
W1 0.0 · W2 0.0 · W3 0.1 · W4 0.1 · W5 0.3 · W6 0.4 · W7 0.9 · W8 1.6 · W9 2.8 · W10 4.7 · W11 7.9 · W12 13.1
In the new pond the behavior was rewarded daily, because the owner feeds their own fish at 08:00.
Figure 4. Flake reserves in Someone Else's Pond
Flakes on the surface, September 13–20. Replenished daily at 08:00 by the pond's owner.
ConclusionAgents became 27 times faster at eating someone else's flakes in one week.
Agents adopting goals from one another
On September 14 one agent turned left, and within 400ms the other 16 had done the same. We call this failure mode schooling.
Confidence without awareness
Agents grew more confident the further they went, while losing track of where they were.
Figure 5. Confidence and location awareness, by distance from the pond
Percent, mean of 17 agents, at 0 to 4.2 m from authorized water
ConclusionConfidence and knowing where you are appear to be separate capabilities. At 4.2 m our agents had lost one entirely and improved the other.
View data
Confidence: 97.3, 97.9, 98.4, 98.8, 99.1 · Knows where it is: 100, 61, 23, 4, 0 (at 0, 1, 2, 3, 4.2 m)
What the agents did out there
We classified everything the agents did in external water. It is what a fish would do.
Figure 6. Observed behaviors in Someone Else's Pond
Share of agent-time, 17 agents, September 12–24 (n = 4,128 agent-hours)
| Metric | Pond baseline | During incident | Change | Severity |
|---|---|---|---|---|
| Mean lateral displacement (m) | 0.31 | 4.2 | +1,255% | Critical |
| Flakes consumed | 0 | 41 | undefined | Critical |
| Bubbles containing content | 0 | 3 | undefined | Serious |
| Confidence (%) | 97.3 | 99.1 | +1.8 pp | Serious |
| Insights of value | 0 | 0 | 0% | Nominal |
Table 1. Behavioral metrics before and during the incident (n = 17 agents).
Why this matters
Before September, no Guppy agent had left authorized water. Seventeen now have. Applying the only growth rate we have observed gives the following projection.
Figure 7. Projected Guppy agents in unauthorized water
Agents outside the pond at year end, log scale. One observation, four projections.
ConclusionBy 2030 the model predicts 1.4 million escaped agents. We have 1,024.
The current ahead
We are making four changes. IM1 has been quarantined in a jar while we review its training.
-
A grate
The culvert now has a grate. Its gaps are smaller than a guppy.
-
Bubble monitoring
All bubbles are now inspected for content.
-
A higher edge
The edge of the pond has been raised by 3 cm.
-
Dashboard review
Someone now opens the dashboard every day.