Topic 6

Why hidden layers: XOR

Now the full loop, live. The points below follow the XOR pattern: positive where x₁ and x₂ have the same sign, negative where they differ. Each frame runs one epoch, a pass over all the training points in batches of 10, and redraws the network's output across the plane: orange where it says +1, teal where it says −1.

Predict first

With no hidden layer, the network is one neuron reading x₁ and x₂ directly. After 200 epochs, roughly what share of the test points does it classify correctly?

About 50%: no better than a coin flipAbout 75%: three quadrants out of fourAbout 90%100%

Can a network with no hidden layer learn XOR?

XOR · 400 points, half for testing · learning rate 0.03 · batch 10 · seeded

Epoch 0 · train 0.601 · test 0.706

Test accuracy

51%

Parameters

Training lossTest loss

Positive points where x₁ and x₂ have the same sign, negative where they differ. A single line can't split them; one hidden layer can, and so can the right input.Open the full playground

Two lessons in one picture. A hidden layer of neurons, each bending its own straight line through an activation function, can combine them into shapes no single line could make. And the right input can do the same job: the network can learn the feature x₁x₂ itself, or you can hand it over. Deep learning's advantage is that it learns useful features from raw data instead of needing someone to design them.

Quick check

A network with no hidden layer has to separate points inside a circle from points around it, and it's only given x₁ and x₂. What would let it succeed without adding a hidden layer?

A larger learning rateTraining for more epochsAdding x₁² and x₂² as inputsMore training points