A game was mostly start-up
The official runner compiles both bots before every game. On 4 Nov that was almost the whole cost of a game.
- Start-up and the first round3.04 CPU-s, 3.0 of it compiling the two bots
- The other 1,169 rounds0.32 CPU-s
Compile once, play many games
The official engine ships as WebAssembly. A translator, wasm2c, turns it into C that compiles to native code; each bot gets the same treatment once, then plays thousands of games. That alone made a game about 9.5× cheaper. My agents’ rework of the engine then took it to about 15×.
CPU-seconds a game
- Official runner
- Our engine
How it got there
- The official engine, compiled to native code7.17 games a CPU-second against the official runner's 0.77: 9.4× cheaper.
- I flag it as the big leverThe plan had sent every game back to the official runner. “Every single test that we will run will take 2.5 times the amount of time… a huge lever.”
- Native becomes every caller's default2,400 games at 0.352 against 3.357 CPU-seconds: 9.5×. “We shouldn't even be able to use the old engine.”
- A faster-compiling runtime, rejectedWasmtime compiled a bot in about 1.5 CPU-s against clang's 16 to 28 s, but ran turns 1.36× slower. “Even 30% is big”, so native stayed.
- Agents hill-climb the engine's own codeTwo climbs cut 21% and 19%. The ratio to the official runner reaches about 15×.
- Build flags and one C call a game0.7914 to 0.7177 worker-seconds a game, 9.3% less.
- No fallbacksFour gates had been quietly playing on the official runner, 3.4 to 4.7 s a game against 0.4 to 1.0. Now a game our engine can't play exactly fails and says why.
Agents climbed the engine against one number
On 5 Nov I asked for “something to hill climb”. A profile of one native game showed where its 0.167 CPU-seconds went. Everything but the bots’ own code was fair game.
- Engine answering the bots' calls
- The bots' own codeoff limits
- Copying call buffers and import glue
- Rules, setup, hashing and output
The biggest find: one engine call copied 0.25 to 1.7 GB a game to read 2 bytes. It now reads the bot’s own buffer. I asked for 70%; with the bots’ code untouchable, about 20% a climb was the ceiling.
What each change cut
Tried and dropped
- ✗ -O3builds identical to -O1
- ✗ Segue, one-file buildabout 1% and 0.3%
- ✗ Profile-guided buildno profiling runtime
- ✗ Shadow casting in C30% slower than wasm2c's
- ✗ Shadow-casting memofailed the exactness check
- ✗ Hand-written portnever played a bot; deleted 6 Nov
Not one game changed
A faster engine is worthless if it plays a different game. Every check compared it with the official engine, and every speedup with the version before it.
| When | Check | Games | Differed |
|---|---|---|---|
| 26 Oct | Whole engine memory hashed every round, against the official module run in V8 | 3,498 games, 3.77M rounds | 0 |
| 26 Oct | Replay and result file, byte for byte, against the official runner | 269 of 280 (11 excluded by rule) | 0 |
| 29 Oct | Replays with fuel metering on | 5,074 of 5,074 | 0 |
| 4 Nov | Outcome, rounds, faults and result file | 2,013 played natively | 0 |
| 4 Nov | The same on 80 generated maps | 2,352 of 2,400 | 0 |
| 4 Nov | Replay, result and both fuel files, byte for byte | 176 | 0 |
| 5 Nov | Every fast path also run in the original engine | 192 + 1,890 on 135 maps | 0 |
| 8 Nov | Memory-hash agreement test, full run | 181 + 7 end scenarios | pass |
Before it landed, each speedup also had to leave every output file of 192 reference games identical.
Then the same method on every tool
Two slow Python tools were rewritten in C++ and checked against the originals: the map generator made the same maps byte for byte, and the fuel proof gave the same answers on 114 of 115 test programs.
Python against the C++ rewrite, seconds
- Python
- C++
On 7 Nov I asked for “a sub one second command” that scores a real, fully simulated scenario. Its rules became the rules of every climb.
- One change
- Score it
- Kept only past the noise
- Every try logged
- Land at the bar
From 9 Nov a different agent designed each eval from the one that climbed it, and a climb under 10% after an hour restarted in a fresh agent.
“They shouldn't have the limiting beliefs from their past fails.”
Every tool climb, before and after
- Before
- After
Every rebuild got cheaper
Every change to the bot meant a rebuild, so builds got the same treatment.
- Before
- After
Where climbing stops working
A climb works when the number is the goal. Aimed at the bot’s play, it moved the score and not the games, so I kept it for tools and deleted the rest.
| Climb | Its score | What games said |
|---|---|---|
| Fuel a turn | 56,731.8 → 54,259.6 | +0.23 Elo, inside −1.26 to +1.71 |
| Turns decided by the fuel limit | 0.7189% → 0.2109% | 50.7% head to head, 32,000 games |
| One farming turn against a deep solve | 3.2879 → 3.2808 | noise; its gate failed |
| Sharpness of the rating model | 2.1005 → 2.0321 | the live version's interval widened, ±32 to ±39 |