A chess engine that learns from its own games

An AlphaZero-style engine written from scratch in Python and PyTorch, and a controlled look at why its self-play training stalls.

The Play view: a maple and walnut board in a dark wooden frame, an evaluation bar, the engine's suggested moves and the move list.

Three opponents: the trained network alone, the network inside a small search, and a classical alpha-beta engine.

What the experiments show

Within three iterations, self-play collapses into threefold repetition: the network learns that every position is a draw, so the search has no reason to convert an advantage. Ten times more self-play does not escape it. Across three training seeds, measured against the network each run started from, training inside the cycle made the engine worse: about 93 Elo against a random mover and 155 against a material-grabbing one. Blending a material count into the search only held it level. An earlier single-seed gain for that arm did not replicate. Scoring drawn games by material did not help either, and a bug that hid repetitions from the search has been fixed.

How it is built

Network
A residual CNN with a policy head over AlphaZero's 4,672-move encoding and a value head. 4 blocks, 64 channels.
Search
PUCT Monte-Carlo tree search, with the value head at the leaves instead of rollouts.
Training
Self-play games become (position, visit distribution, result) examples for a combined policy and value loss.
Second engine
A classical alpha-beta searcher with quiescence and piece-square tables, rated about 1390 against strength-limited Stockfish.
Tests
59 tests in CI, aimed at the bugs that fail silently: board mirroring, value sign flips, move-encoding round trips.

Play against the engine

Engine suggests

  1. Press Hint for suggestions.

Moves

No moves yet.
50%
You play White. Click a piece, then its destination, or drag it.

Opponent

A material and capture search in your browser. The network only breaks near-ties.

Settings
Play as
Difficulty
Opponent
Clock

Right-drag to draw an arrow, right-click to mark a square. Keys: N new, F flip, U undo, H hint.

Captured

By you
By the engine

What I found

Self-play training stalled in a recognisable way, and across three seeds it made the network worse than where it started. These are the measurements, what they support, and what they do not.

How the investigation went

July 2026
One GPU run, one seed per arm. Blending a material count into the search seemed to break the draw cycle, from −21 to +28 Elo. I published that as the headline.
14 Nov
I found the flaw: each arm had been evaluated with its own training search, and nothing was compared with the network it started from. I withdrew the headline, and fixed a draw rule that ended games before anyone had actually repeated.
15–16 Nov
The rerun: three seeds per arm, every network against its own start. Training inside the cycle made the network worse, by 93 Elo against random. The July gain did not replicate.
16–17 Nov
Two more suspects. Scoring drawn games by material taught the value head a little but changed nothing in play (−93 again). Reading the search code turned up a bug: it could not see repetitions of earlier positions in the game. Fixed, with a test that reproduces it.
Now
One last run with the fixed search, alone and with contempt. Results below when it finishes.
What I took from it

One run with no reference point can produce a convincing effect that is not there. The habits that caught it, a starting-network control, several seeds and an interval on every number, are now how everything here is measured. The two real bugs, though, were found by reading code, not by spending more GPU time.

The draw cycle

The engine could win material but not convert it. In the baseline run, the untrained network scored a few accidental checkmates in its first two iterations. From iteration 3 onward, all 80 self-play games ended in threefold repetition.

The mechanism is in the value target. When every game is drawn, the outcome z is 0 for every training position, so the value head learns that every position is even. A search guided by that value has no reason to prefer a won position over a repetition, so the next batch of games is drawn too. The value loss falling to about 0.01 is a symptom of this, not a sign of good training. The policy loss falling from 4.5 to 0.8 means less than it appears to as well: the targets are the search's own visit counts, which sharpen as the network does.

The first experiment (July, one seed)

One GPU run (Kaggle, 2× T4, 9.8 hours) with three arms. Each arm changes one thing from the baseline: A scales self-play tenfold; B blends a material count into the search's leaf value, v = 0.7·vnet + 0.3·material.

ArmSelf-play gamesSims / moveMaterial wCheckmates in self-playElo vs random, pooled (95% CI)
Baseline10010008%−22 (−38, −7)
A · 10× compute1,000160012%+10 (−8, +28)
B · material assist1001000.3019%+21 (+1, +41)

Elo is pooled over each arm's five 50-game checkpoints against a uniformly random mover (250 games, 50 simulations per move). Pooling mixes networks from different iterations, so read it as the arm's typical level, not its final strength. The per-checkpoint intervals are about ±40 Elo, and every single checkpoint in arm B includes zero.

Left: share of self-play games not ending in checkmate per iteration; the baseline reaches 100% by iteration 3, the 10x run stays near 90%, the assisted run ends at 70%. Right: Elo against random at each checkpoint with 95% intervals, with reference bands for the starting network.
Left: self-play games that did not end in checkmate. Right: Elo against random at each checkpoint, with 95% intervals; the dotted lines and bands are the starting network, with and without the material blend.

A confound, and a control

Arm B was evaluated with the same search it trained with, so its moves against the random opponent were also guided by the material heuristic. Its Elo therefore mixes two things: what the network learned, and what the heuristic does on its own. The original write-up attributed all of it to learning.

To separate them, I evaluated freshly initialised networks that had never been trained, under the same settings. The most useful is the exact network all three arms started from (seed 42, 100 games per setting); three other seeds (50 games each) show how much an untrained network varies on its own.

NetworkMaterial w at evaluationW / D / LElo vs random (95% CI)
Starting network (seed 42)010 / 80 / 100 (−31, +31)
Starting network (seed 42)0.303 / 91 / 6−10 (−31, +10)
Untrained, seeds 1–3041 / 98 / 11+70 (+39, +103)
Untrained, seeds 1–30.307 / 137 / 6+2 (−14, +19)

Two things stand out. The heuristic alone does not beat random: it turns an untrained network's games into draws. And untrained networks differ a lot by seed: seeds 1 and 2 scored about +110 against random, seed 3 about −7. A single seed per arm cannot separate arms that are 20 Elo apart.

Measured against the starting network, under the search each arm was evaluated with:

ArmTrained (pooled)Starting networkChange, Elo (95% CI)
Baseline−220−22 (−56, +12)
A · 10× compute+100+10 (−26, +45)
B · material assist+21−10+31 (+3, +60)

A rule that ended games early

The engine stopped a game as soon as a draw could be claimed. In python-chess a repetition claim is valid when the side to move merely has a move that would repeat a position a third time, so games ended as “threefold repetition” before anyone repeated, and the search scored every such position as a dead draw, even for the side that was winning. Every number above used that rule, so the comparisons between them hold, but the draw counts are inflated and the rule itself pushes towards draws.

The rule is now the real one: three actual occurrences, or a fifty-move counter that has reached 100. Under it, the network on this site still ends all six of its recorded self-play games in a genuine threefold repetition, after 27–35 plies (the replay above). The rerun sized the rule's effect: the same seed and arm finished 89% of self-play games without checkmate under the old rule and 82% under the corrected one. The rule inflated the counts; it did not create the cycle.

The value head, measured directly

The explanation is about the value head, so I measured it: 300 roughly level positions, and the same positions with one queen removed so the side to move is up or down a queen.

NetworkUp a queenLevelDown a queenUp − down
Untrained (seed 42)−0.113−0.113−0.1130.000
Early CPU checkpoint (July)−0.057−0.058−0.060+0.003
Rerun, seed 1 baseline (the best)+0.111+0.068+0.051+0.060
Rerun, the other five networks−0.03 to +0.02−0.03 to +0.02−0.04 to +0.02−0.001 to +0.009
This site’s network (seed 1, assisted)+0.004+0.001+0.0040.000

On a scale from −1 to +1, being a queen up should read close to +1. After ten GPU iterations the best network separates the two by 0.06 and the rest by under 0.01: they have learned almost nothing about material, which is what a value target of zero everywhere predicts.

A sturdier yardstick

Most games against a random mover are drawn, which is why an untrained network can land anywhere from about −7 to +120 against it. The repository now has a second opponent: one that mates when it can and otherwise grabs the most material, looking no further. Untrained networks lose most games to it outright:

Starting networkMaterial wW / D / LElo vs greedy (95% CI)
Seed 42 (100 games)00 / 13 / 87−463 (−592, −386)
Seed 42 (100 games)0.302 / 47 / 51−186 (−238, −141)
Seeds 1–3 (150 games)03 / 32 / 115−335 (−405, −282)
Seeds 1–3 (150 games)0.307 / 57 / 86−203 (−252, −161)

That leaves room to measure learning, with mostly decisive games. Seed still matters, so the rerun uses three. The networks from the July run no longer exist, so this applies from the rerun on.

Three seeds per arm

The rerun trained the baseline and the material-assisted arm from three different random initialisations each, under the corrected draw rule (Kaggle T4, about 11.5 hours of training). Every trained network was then measured against its own untrained starting network, under the same evaluation. Pooled over the three seeds:

ArmAgainstTrainedStarting networkChange, Elo (95% CI)
BaselineRandom−19+74−93 (−125, −60)
BaselineGreedy−524−369−155 (−265, −45)
Material assistRandom, its own search+2−3+6 (−19, +31)
Material assistRandom, network only−30+74−104 (−136, −72)
Material assistGreedy, network only−432−369−63 (−154, +28)

Per seed, the baseline moved −177, −112 and +3 against random. Seeds 1 and 2 happened to start strong (+135 and +119) and lost all of it; seed 3 started weak and stayed there. From iteration 3, 89–100% of self-play games in every run ended without checkmate. The data, the summary script and the notebook are in the repository.

Two more suspects

A value target that is not zero. If drawn games teach the value head nothing, give them something to teach: the final material balance, about +0.8 for a queen up. Same three seeds and settings as the rerun. Against its own start the network still lost 93 Elo against random (95% CI −122 to −63), the same as the baseline, and 87–95% of self-play games still ended without checkmate. The value head did learn some material: up a queen minus down a queen rose to +0.04 to +0.14, from 0.00 to 0.06 in the rerun. Knowing that a queen matters was not enough to change how the engine played. (data)

A search that could not see the repetition. The tree search works on copies of the board, and the copies dropped the game's move history. Inside the search, a move that repeated a position for the third time looked like any other move, so the search could not tell it was walking into a draw. The copies now keep the history that matters (everything since the last capture or pawn move), at no measurable cost, and a test reproduces the old behaviour. Every result on this page used the old search.

What the data supports

  • The draw cycle is real. From iteration 3, 89–100% of self-play games end without checkmate in every run, under either draw rule, and ten times the games did not escape it in July.
  • Training inside it made the network worse. Across three seeds the baseline ended below the random network it started from, against both yardsticks.
  • The material assist did not fix training. With the blend in its search the engine held level (+6); its network alone degraded like the baseline's. The July single-seed gain of +31 did not replicate.
  • A material value target did not fix it either. −93 against random, the same as the baseline, although its value head learned more about material.
  • The draw rule inflated the counts, not the cycle. Same seed, old rule 89% non-decisive, corrected rule 82%.
  • The original headline does not hold. “Signal, not compute: −21 → +28” compared networks evaluated with different searches, with no starting-network reference and no intervals.

Limitations

  • Three seeds and ten iterations per arm. The intervals are wide, especially against the greedy opponent, and much longer runs could behave differently. The 10× arm was not rerun.
  • The GPU run's numbers are transcribed from its captured output; its raw logs and checkpoints were lost with the session. The transcription is in one JSON file, and the figure is rebuilt from it.
  • Random is a weak yardstick. It separates a broken engine from a working one, not a good engine from a better one.
  • The network you can play on this site is the seed-1 material-assisted network from the rerun, chosen before any results were seen. It is weak, as the tables say.

What next

One run is in progress: the fixed search on its own, and with contempt, where during self-play each side counts a draw as slightly bad for itself, so neither is content to repeat. Three seeds each, every network against its own start (notebook).

After that I am stopping. Each run here is 100 self-play games; AlphaZero trained on about 44 million. At this size an idea can be ruled out cleanly but rarely ruled in, so more variations would mostly add more null results. The one I would try with real compute is the standard fix: move-history planes in the network's input, so it can see a repetition coming.

How it works

The three parts of AlphaZero (a network, a search and a self-play loop), implemented from scratch. python-chess supplies only the rules.

The network

A position becomes an 18 × 8 × 8 tensor: twelve piece planes, four castling planes, en passant and the fifty-move counter. It is always drawn from the side to move, so the network never has to learn the game twice, once per colour. A small residual CNN (4 blocks × 64 channels) maps it to two outputs:

  • a policy: logits over all 64 × 73 = 4,672 moves in AlphaZero's encoding (56 sliding moves, 8 knight moves and 9 under-promotions per square), masked to the legal ones;
  • a value in [−1, 1]: the expected result for the side to move.

The search

Each move runs a fixed number of simulations down a tree. At every node the search picks the child that maximises

Q(s,a) + cpuct · P(s,a) · √N(s) / (1 + N(s,a))

The first term is the average value found below that move. The second is the network's prior, discounted as the move gets visited. Leaves are scored by the value head, not by random playouts, and each value flips sign as it is backed up a ply. The visit counts at the root are the search's verdict and, during training, its teaching signal.

Self-play and training

  1. Play a game against itself, choosing moves by search, with Dirichlet noise at the root and sampling for the first 15 plies.
  2. Store every position with the root visit distribution π, then fill in the result z when the game ends.
  3. Train on a replay buffer of recent positions, minimising
L = (z − v)² − π · log p + λ‖θ‖²

A better network makes a better search, whose games make better targets. That loop is the whole method; the Findings page shows what happens when its value target degenerates.

Which engine you are playing

Network
The value head alone, one ply deep, run in your browser with ONNX Runtime. It is the seed-1 material-assisted network from the GPU rerun, chosen in advance, and it plays weakly.
Hybrid
A material and capture search in your browser. The network's value only breaks near-ties (worth at most 0.4 pawns), so this mostly plays like the search.
Classical
Alpha-beta with quiescence, move ordering, piece-square tables and an endgame term, on the server. At depth 2 it scored 14–8–8 against Stockfish limited to UCI_Elo 1320: about 1390 (95% CI 1284–1512) on Stockfish's scale, not a FIDE rating.

Before the browser trusts the network, it checks its own board encoding against golden values exported from Python. A mismatch would feed the network garbage without any error.

What I wrote and what I used

Wrote
The network, the move and board encodings, the search, the self-play and training loop, the evaluation and experiment tooling, the classical engine and this site.
Used
PyTorch, python-chess and chess.js for the rules, ONNX Runtime Web and FastAPI. The algorithm is DeepMind's.