Descent
Watch a machine learn, as a landscape. A live in-browser tour of gradient descent — perceptron to transformer — with the real loss surface it rolls down. No server, all math.
Every model that learns is doing the same quiet thing: rolling downhill.
Give a network a task and a way to measure its error, and that error becomes a surface stretched over every possible setting of its weights — a landscape with peaks of confident wrongness and valleys where it has understood. Learning is just descent: read the slope, take a step downhill, repeat.
Below, that surface is real. Each chapter trains an actual model in your browser — no server, no video, just arithmetic — and traces the path it takes down its own loss. On the left, the landscape rotates so you can see its shape; on the right, watch what the model is learning to do as it descends.
The story runs from 1958 to 2017, but it is one idea the whole way down. Drag the landscape to turn it.
One neuron, one line. Rosenblatt's perceptron learns AND — a single line splits the true corner from the rest. Its loss surface is a smooth convex bowl, so descent slides straight to the floor.
A different kind of problem
Every task so far had an answer you could draw — a line, then a curve — separating dots by where they sit. But some questions aren’t about position at all. Ask “do these two match?” and no boundary over fixed features can help: the answer lives in the relationship between the inputs. Reaching it took a new idea — one the field didn’t land until 2017.
Learning to compare. This chapter asks a relationship — do the first and last tokens match? — which no boundary over fixed features can express. So the model gets a new move: attention. Scrub forward and watch each position learn where to look, until the two ends are linked.