Topic 1
A model is a function with knobs
Every model in this course, from this lesson's tiny network to a large language model, is a function. Numbers go in, numbers come out, and what happens in between is set by a list of adjustable numbers called parameters. Training is the search for parameter values that make the function do what you want.
Hands-On Large Language Models describes neural networks as interconnected layers of nodes, where each connection has a weight; those weights are the model's parameters. The building block is one neuron: it multiplies each input by a weight, adds them up with one extra number called the bias, and passes the total through an activation function that bends it.
Scroll sideways to see the whole diagram
Two inputs: a point's coordinates.
Text version
- Two inputs: a point's coordinates.
- Multiply each input by its weight (0.5 and 1.5 here), add them up, add the bias (0.1). The weights and the bias are the neuron's parameters: the numbers training adjusts.
- An activation function bends the result. Without the bend, stacking neurons only ever gives another straight line; with it, layers can build curves.
Stack neurons into layers and you have a network. Each neuron in a layer takes every output of the layer before, so the parameters add up quickly.
Predict first
The activation function matters more than it looks. Without it, a neuron is a weighted sum, and a weighted sum of weighted sums is still just a weighted sum: however many layers you stack, the network can only draw straight lines. The bend is what lets layers build curves, which you'll see happen in the XOR lab.