A reinforcement-learning project
A chess engine that teaches itself
An AlphaZero-style reinforcement-learning system, built from scratch in Python & PyTorch — a neural network (policy + value) guided by Monte-Carlo Tree Search that learns purely by playing itself. This site showcases that project: read how it learns, watch it self-play, and play a game yourself.
Honest note: the star is the self-taught RL network, trained from scratch. You can play it raw — but a laptop can't do enough self-play to make it strong, so on its own it's weak. So the default opponent, “Neural net + search”, pairs the network with a small tactical search so it plays a genuinely solid game — switch to the raw net anytime to meet the learning project on its own. See How it works.
Headline result — the “draw cycle”
The engine kept drawing even when it was winning. A controlled
experiment showed this wasn’t mainly a compute limit: giving self-play a better signal
(material_weight = 0.30) flipped its Elo-vs-random from −21 to +28
at the same compute.
Neural network
A residual CNN reads the board and outputs a move policy and a value (who's winning) — no hand-written chess knowledge.
Search (MCTS)
Monte-Carlo Tree Search with the PUCT rule looks ahead, turning the network's hunches into much stronger moves.
Self-play learning
The engine is its own opponent and its own teacher: games become training data, which sharpens the next generation.
Play it now
Face the self-taught network (paired with a quick tactical search so it's a fair fight), the raw network on its own, or a strong classical engine — right in your browser.
How this is different from other chess engines
Most chess engines are pre-built black boxes. This one is about understanding how an engine learns and thinks.
It learns, it isn't told
Engines like Stockfish run on decades of human-tuned chess knowledge. This one starts from random and teaches itself — the AlphaZero idea — discovering good play from its own games.
It shows its thinking
See the live evaluation, win probability and the moves it's considering, drawn as arrows on the board. Most engines never let you see why.
Built from scratch to be understood
The network, the search and the self-play loop are all written from scratch and explained in plain English on the How it works page. It's a learning tool, not just a player.
Watch it play itself
Hit “Watch self-play” and the engine reasons through a whole game on its own — exactly the kind of self-play it learns from.
Engine recommends
Moves
Game
Tip: right-click + drag to draw an arrow; right-click a square to mark it. Keys: N new · F flip · U undo · H hint.
Material
What I found
The most interesting part wasn’t building the engine — it was diagnosing why it wouldn’t win. This page is that experiment and its result.
The “draw cycle”
My self-taught engine could win material but not convert it — it shuffled winning positions into draws by repetition, and drew even a random-move opponent about half the time. The cause: in self-play almost every game ended in a draw, so the network’s value head only ever learned “the position is even.” It never learned that being up a queen is good, so the search never tried to convert — a self-reinforcing trap.
The experiment
The comfortable explanation was “it just needs more compute.” Rather than assume, I tested it — two controlled runs on a GPU (Kaggle, 2×T4, ~9.8 h), each holding the code fixed and changing exactly one thing:
- A — more compute. Scaled self-play 10× (100 → 1,000 games).
- B — a better signal, same compute. Blended a small material term into self-play leaf
evaluation (
material_weight = 0.30) so games ended decisively.
| Run | Games | Sims/move | Material w | Non-decisive | Elo vs random |
|---|---|---|---|---|---|
| baseline | 100 | 100 | 0.00 | 100% | −21 |
| scaled (10×) | 1 000 | 160 | 0.00 | 92% | −7 |
| assisted | 100 | 100 | 0.30 | 70% | +28 |
The result
The draw cycle was primarily a self-play signal problem, not a compute ceiling. Scaling
compute 10× barely moved the plateau (−21 → −7); a better self-play signal at the same compute
flipped Elo from −21 to +28 and cut non-decisive games from 100% to 70%.
material_weight = 0.30 is now the engine’s default.
The real lesson wasn’t about chess: the most convincing-sounding explanation — the one that happens to require nothing of you — is exactly the one worth testing first.
How the engine works
The same three ideas that powered DeepMind's AlphaZero — implemented from scratch to be read and understood, not just run.
The simple version (30 seconds)
Nobody teaches the engine chess strategy. It only knows the rules. It gets good through one repeating loop:
- Guess. A neural network looks at the board and guesses the best moves and who's winning.
- Think. A search tries out those moves many moves deep to find something better than the first guess.
- Play itself. It plays thousands of games against itself using that search.
- Learn. Whoever won, it nudges the network toward the moves that led to wins — so next time the guess is already smarter.
Repeat millions of times and the engine bootstraps itself from random moves to real chess — with zero human strategy added. Everything below is just that loop, in detail.
1 · One network, two heads — the "guess"
A position is encoded as an 18 × 8 × 8 stack of planes (piece locations, castling rights, …), always shown from the side-to-move's perspective. A residual convolutional network maps it to:
- a policy — a probability over all
8×8×73 = 4672possible moves ("which moves look promising?"), and - a value in [−1, 1] — "who is winning?"
2 · Search that thinks ahead (MCTS + PUCT) — the "think"
Each move, hundreds of simulations descend a search tree, balancing what looks good now against what's worth exploring, using the PUCT rule:
Exploitation (Q, the average value found) plus exploration (the network's
prior P, damped by how often a move was tried). The most-visited move is the
search's verdict — far stronger than the raw network.
3 · The self-play loop — "play itself" & "learn"
The training objective combines both heads (plus weight decay):
value error (MSE to the game result z) + policy cross-entropy to the MCTS
target π. Minimising both is what makes a single network good at both
halves of the search.
4 · A real training run
Loss from an actual self-play run — the policy loss drops sharply as the network learns to imitate the search:
Honest note: reaching strong play needs far more (GPU) self-play than a laptop CPU allows — the learning machinery is correct, the limit is compute. So on the Play page the default opponent pairs this network with a small in-browser tactical search (so it never simply hangs a piece); you can switch to the raw self-taught network to see it play on its own, or to a strong classical alpha-beta engine for a tough game. The network stays the from-scratch learning project — the search just makes it a fair opponent today.