What I found
Self-play training stalled in a recognisable way, and across three seeds it made the network worse than where it started. These are the measurements, what they support, and what they do not.
How the investigation went
- July 2026
- One GPU run, one seed per arm. Blending a material count into the search seemed to break the draw cycle, from −21 to +28 Elo. I published that as the headline.
- 14 Nov
- I found the flaw: each arm had been evaluated with its own training search, and nothing was compared with the network it started from. I withdrew the headline, and fixed a draw rule that ended games before anyone had actually repeated.
- 15–16 Nov
- The rerun: three seeds per arm, every network against its own start. Training inside the cycle made the network worse, by 93 Elo against random. The July gain did not replicate.
- 16–17 Nov
- Two more suspects. Scoring drawn games by material taught the value head a little but changed nothing in play (−93 again). Reading the search code turned up a bug: it could not see repetitions of earlier positions in the game. Fixed, with a test that reproduces it.
- Now
- One last run with the fixed search, alone and with contempt. Results below when it finishes.
One run with no reference point can produce a convincing effect that is not there. The habits that caught it, a starting-network control, several seeds and an interval on every number, are now how everything here is measured. The two real bugs, though, were found by reading code, not by spending more GPU time.
The draw cycle
The engine could win material but not convert it. In the baseline run, the untrained network scored a few accidental checkmates in its first two iterations. From iteration 3 onward, all 80 self-play games ended in threefold repetition.
The mechanism is in the value target. When every game is drawn, the outcome
z is 0 for every training position, so the value head learns that every
position is even. A search guided by that value has no reason to prefer a won position over
a repetition, so the next batch of games is drawn too. The value loss falling to about 0.01
is a symptom of this, not a sign of good training. The policy loss falling from 4.5 to 0.8
means less than it appears to as well: the targets are the search's own visit counts, which
sharpen as the network does.
Replay a real self-play game
- Move
- 0
- Last move
- –
- Position seen
- 1×
The first experiment (July, one seed)
One GPU run (Kaggle, 2× T4, 9.8 hours) with three arms. Each arm changes one thing from the
baseline: A scales self-play tenfold; B blends a material count into the search's leaf value,
v = 0.7·vnet + 0.3·material.
| Arm | Self-play games | Sims / move | Material w | Checkmates in self-play | Elo vs random, pooled (95% CI) |
|---|---|---|---|---|---|
| Baseline | 100 | 100 | 0 | 8% | −22 (−38, −7) |
| A · 10× compute | 1,000 | 160 | 0 | 12% | +10 (−8, +28) |
| B · material assist | 100 | 100 | 0.30 | 19% | +21 (+1, +41) |
Elo is pooled over each arm's five 50-game checkpoints against a uniformly random mover (250 games, 50 simulations per move). Pooling mixes networks from different iterations, so read it as the arm's typical level, not its final strength. The per-checkpoint intervals are about ±40 Elo, and every single checkpoint in arm B includes zero.
A confound, and a control
Arm B was evaluated with the same search it trained with, so its moves against the random opponent were also guided by the material heuristic. Its Elo therefore mixes two things: what the network learned, and what the heuristic does on its own. The original write-up attributed all of it to learning.
To separate them, I evaluated freshly initialised networks that had never been trained, under the same settings. The most useful is the exact network all three arms started from (seed 42, 100 games per setting); three other seeds (50 games each) show how much an untrained network varies on its own.
| Network | Material w at evaluation | W / D / L | Elo vs random (95% CI) |
|---|---|---|---|
| Starting network (seed 42) | 0 | 10 / 80 / 10 | 0 (−31, +31) |
| Starting network (seed 42) | 0.30 | 3 / 91 / 6 | −10 (−31, +10) |
| Untrained, seeds 1–3 | 0 | 41 / 98 / 11 | +70 (+39, +103) |
| Untrained, seeds 1–3 | 0.30 | 7 / 137 / 6 | +2 (−14, +19) |
Two things stand out. The heuristic alone does not beat random: it turns an untrained network's games into draws. And untrained networks differ a lot by seed: seeds 1 and 2 scored about +110 against random, seed 3 about −7. A single seed per arm cannot separate arms that are 20 Elo apart.
Measured against the starting network, under the search each arm was evaluated with:
| Arm | Trained (pooled) | Starting network | Change, Elo (95% CI) |
|---|---|---|---|
| Baseline | −22 | 0 | −22 (−56, +12) |
| A · 10× compute | +10 | 0 | +10 (−26, +45) |
| B · material assist | +21 | −10 | +31 (+3, +60) |
A rule that ended games early
The engine stopped a game as soon as a draw could be claimed. In python-chess a repetition claim is valid when the side to move merely has a move that would repeat a position a third time, so games ended as “threefold repetition” before anyone repeated, and the search scored every such position as a dead draw, even for the side that was winning. Every number above used that rule, so the comparisons between them hold, but the draw counts are inflated and the rule itself pushes towards draws.
The rule is now the real one: three actual occurrences, or a fifty-move counter that has reached 100. Under it, the network on this site still ends all six of its recorded self-play games in a genuine threefold repetition, after 27–35 plies (the replay above). The rerun sized the rule's effect: the same seed and arm finished 89% of self-play games without checkmate under the old rule and 82% under the corrected one. The rule inflated the counts; it did not create the cycle.
The value head, measured directly
The explanation is about the value head, so I measured it: 300 roughly level positions, and the same positions with one queen removed so the side to move is up or down a queen.
| Network | Up a queen | Level | Down a queen | Up − down |
|---|---|---|---|---|
| Untrained (seed 42) | −0.113 | −0.113 | −0.113 | 0.000 |
| Early CPU checkpoint (July) | −0.057 | −0.058 | −0.060 | +0.003 |
| Rerun, seed 1 baseline (the best) | +0.111 | +0.068 | +0.051 | +0.060 |
| Rerun, the other five networks | −0.03 to +0.02 | −0.03 to +0.02 | −0.04 to +0.02 | −0.001 to +0.009 |
| This site’s network (seed 1, assisted) | +0.004 | +0.001 | +0.004 | 0.000 |
On a scale from −1 to +1, being a queen up should read close to +1. After ten GPU iterations the best network separates the two by 0.06 and the rest by under 0.01: they have learned almost nothing about material, which is what a value target of zero everywhere predicts.
A sturdier yardstick
Most games against a random mover are drawn, which is why an untrained network can land anywhere from about −7 to +120 against it. The repository now has a second opponent: one that mates when it can and otherwise grabs the most material, looking no further. Untrained networks lose most games to it outright:
| Starting network | Material w | W / D / L | Elo vs greedy (95% CI) |
|---|---|---|---|
| Seed 42 (100 games) | 0 | 0 / 13 / 87 | −463 (−592, −386) |
| Seed 42 (100 games) | 0.30 | 2 / 47 / 51 | −186 (−238, −141) |
| Seeds 1–3 (150 games) | 0 | 3 / 32 / 115 | −335 (−405, −282) |
| Seeds 1–3 (150 games) | 0.30 | 7 / 57 / 86 | −203 (−252, −161) |
That leaves room to measure learning, with mostly decisive games. Seed still matters, so the rerun uses three. The networks from the July run no longer exist, so this applies from the rerun on.
Three seeds per arm
The rerun trained the baseline and the material-assisted arm from three different random initialisations each, under the corrected draw rule (Kaggle T4, about 11.5 hours of training). Every trained network was then measured against its own untrained starting network, under the same evaluation. Pooled over the three seeds:
| Arm | Against | Trained | Starting network | Change, Elo (95% CI) |
|---|---|---|---|---|
| Baseline | Random | −19 | +74 | −93 (−125, −60) |
| Baseline | Greedy | −524 | −369 | −155 (−265, −45) |
| Material assist | Random, its own search | +2 | −3 | +6 (−19, +31) |
| Material assist | Random, network only | −30 | +74 | −104 (−136, −72) |
| Material assist | Greedy, network only | −432 | −369 | −63 (−154, +28) |
Per seed, the baseline moved −177, −112 and +3 against random. Seeds 1 and 2 happened to start strong (+135 and +119) and lost all of it; seed 3 started weak and stayed there. From iteration 3, 89–100% of self-play games in every run ended without checkmate. The data, the summary script and the notebook are in the repository.
Two more suspects
A value target that is not zero. If drawn games teach the value head nothing, give them something to teach: the final material balance, about +0.8 for a queen up. Same three seeds and settings as the rerun. Against its own start the network still lost 93 Elo against random (95% CI −122 to −63), the same as the baseline, and 87–95% of self-play games still ended without checkmate. The value head did learn some material: up a queen minus down a queen rose to +0.04 to +0.14, from 0.00 to 0.06 in the rerun. Knowing that a queen matters was not enough to change how the engine played. (data)
A search that could not see the repetition. The tree search works on copies of the board, and the copies dropped the game's move history. Inside the search, a move that repeated a position for the third time looked like any other move, so the search could not tell it was walking into a draw. The copies now keep the history that matters (everything since the last capture or pawn move), at no measurable cost, and a test reproduces the old behaviour. Every result on this page used the old search.
What the data supports
- The draw cycle is real. From iteration 3, 89–100% of self-play games end without checkmate in every run, under either draw rule, and ten times the games did not escape it in July.
- Training inside it made the network worse. Across three seeds the baseline ended below the random network it started from, against both yardsticks.
- The material assist did not fix training. With the blend in its search the engine held level (+6); its network alone degraded like the baseline's. The July single-seed gain of +31 did not replicate.
- A material value target did not fix it either. −93 against random, the same as the baseline, although its value head learned more about material.
- The draw rule inflated the counts, not the cycle. Same seed, old rule 89% non-decisive, corrected rule 82%.
- The original headline does not hold. “Signal, not compute: −21 → +28” compared networks evaluated with different searches, with no starting-network reference and no intervals.
Limitations
- Three seeds and ten iterations per arm. The intervals are wide, especially against the greedy opponent, and much longer runs could behave differently. The 10× arm was not rerun.
- The GPU run's numbers are transcribed from its captured output; its raw logs and checkpoints were lost with the session. The transcription is in one JSON file, and the figure is rebuilt from it.
- Random is a weak yardstick. It separates a broken engine from a working one, not a good engine from a better one.
- The network you can play on this site is the seed-1 material-assisted network from the rerun, chosen before any results were seen. It is weak, as the tables say.
What next
One run is in progress: the fixed search on its own, and with contempt, where during self-play each side counts a draw as slightly bad for itself, so neither is content to repeat. Three seeds each, every network against its own start (notebook).
After that I am stopping. Each run here is 100 self-play games; AlphaZero trained on about 44 million. At this size an idea can be ruled out cleanly but rarely ruled in, so more variations would mostly add more null results. The one I would try with real compute is the standard fix: move-history planes in the network's input, so it can see a repetition coming.