Nine designs, each measured
Each design ran from 8.7 hours to six days. The large dot is the agent in charge; the dots under it sketch how the rest were arranged, not a head count.
one orchestrator, waves of 14 agents
71 hranone orchestrator, 631 sub-agents, GPU training
6 daysranbuilders, each with fresh reviewers
4 daysranreaders study losses, the coordinator fixes
26 hranone lost game each, automatic uploads
64 hrana director over 14 leads
8.7 hranwith up to 10 helpers, then alone
19 hranevery change a written prediction
41 hranreplay it until it is won
23 hranWhat each design showed, and what I changed
| Design | What it showed | What I did next |
|---|---|---|
| Waves | 39 uploads, #347 to #4 on the ladder | Bet on a learned bot to go past hand-written rules |
| Learned bot | Its best won 12 of 140 games against v50 | Deleted the repo and started again |
| Proofs | Four days without a version that beat v50 | Cut reviews to three rounds, then none, and began a new bot |
| Readers | The win rate against v50 rose, then fell | Designed a gate that judges each fix on its own |
| Fix agents | 73 uploads, 2,258 to 2,477 (#12), then gains flattened | Gave the loop to a director, to find bigger changes |
| Leads | 3 uploads, about +69; six pieces of play built one at a time all read zero or worse | One agent doing everything, so the ideas connect |
| One agent | 14 uploads, about +104, all before it went alone | Every change a written bet; the ladder a scoreboard, never a test |
| Bets | 9 uploads, about +48 | Deleted the loop and pointed every hour at the bot's code |
| One game | 8 of 10 set games won, but only level with v139 on new maps | Uploaded nothing: v139 stayed live |
Reviews cost more than they found
The third design made every plan pass rounds of fresh reviewers until a round found nothing. Measured on 3 Nov, it cost far more than it found, so I cut it to three rounds on 4 Nov, to none by default on 5 Nov, and deleted the plans with it.
Highest review round each document reached
Readers find the cause, a gate judges the fix
From 5 Nov one coordinator wrote every fix, fed by about 30 reader agents at a time, each studying one lost game: 1,301 readings in 26 hours, a median 18 minutes each.
- Bulk games against old versions
- One reader per lost game
- The coordinator writes the fixes
- The next bulk run judges them
The reader’s model was chosen by a test, on lost games whose cause was already known.
Tries that found the cause of a test loss
Games won against v50
When it plateaued I stopped it. The coordinator landed four or five fixes a round and judged each on a single game, so I designed a gate: a branch and its parent play the same maps from the same seats, a game whose result changes is a “flip”, and a statistical test on the flips decides.
From a lost game to a shipped fix
From 7 Nov the coordinator wrote no code. Each lost online game went to a fresh fix agent, and a fix that passed the gate was merged with others and uploaded with no human step.
- A lost online game
- A fix agent on its own branch
- Watch it, explain it, count it
- Fix, scenario test, fuel proof
- Gate against its parent
- Batch merged and uploaded
Two cycles, as they happened
2 hours: v59
- Two online losses on one mapTo Mite and Zarzorki.
- A fix agent startsOur champion stood beside a one-tile monolith while another unit took the 200 gems.
- Gate fails−0.34 points over 40,528 maps. The fix made a unit step on and off in a loop.
- A fresh agent narrows the rule
- Gate passes+0.21 points over 273,104 maps.
- Live as v59+0.6 points over v58.
6 hours: v42
- A ranked loss to MiteMite cancels a tribute, strikes, and restarts the tribute in the same turn.
- Nobody gates itv50, the gate's opponent, never does this. Two agents stop: “too rare”.
- I step in“We fix the exploit, exploit back if it's general.”
- Agent 4 passes the gate+1.06 over 10,372 maps, from a change it found on the way: going home now uses this game's death rate.
- Ships inside v42Six hours from the loss to the ladder.
It climbed fast, then flattened, and the gate’s promises shrank when replayed. To me that meant small fixes had run out, so on 9 Nov I gave the loop to a director to look for bigger changes.
Measuring the agents, not only the bot
Twice I had a design measured before replacing it. On 9 Nov a redesign I ran from my laptop measured the coordinator loop of 5 to 9 Nov.
On 11 Nov my own advisor thread ran 20 reviewers over the director’s design. Six read “Broken”. One measured the night the director had just spent, with helpers.
The director’s night, measured
| Measure | Reading |
|---|---|
| Uploads in 12 h | 14, and 13 were replaced before 1,200 games |
| Silent game queue | the queue of practice games stopped for 3 h 13 min unnoticed: about 1,400 games lost |
| One-call turns | 92.9% of the director's 4,056 turns made one tool call |
| Working file | rewritten 114 times, 97 lines grown to 754 |
| Helpers | 78 runs, 59.5 agent-hours, 11.0M output tokens: 2.8 times the director's own |
| Helper code | 32 of 50 runs that wrote code never reached the main branch |
Bets, then one game at a time
From 11 Nov every change was a “bet”: a written prediction, a number showing it happens in games, and a judge. A hook refused remote games run without one.
What happened to the 85 bets
The kept bets added about +20 Elo on our own league in two days. On 12 Nov I had the loop deleted and put all the time on the bot: ten lost games, each replayed exactly, fixed one at a time until won, keeping every earlier game won.
“Fix the bot until this one game is won.”
Fitting ten games, checked on new maps
- Before
- After 321 commits
The check on unseen maps caught the overfitting before anything shipped. Repairs, by deleting rules, brought the branch back level with v139, 609 of 1,200 games, so v139 stayed live. The lesson the agents wrote down: check every rule on maps it has never seen.