Deep Black BoxOpen up the black box of AI one problem at a time, with your own hands. 21 problems in all.
Question 4

More knobs, more shapes you can draw

Knobs = the number of sliders on the screen (unchanged since Problem 1)
KnobsShape of the lineBends
2โ—โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ—0
3โ—โ”€โ”€โ”€โ—โ”€โ”€โ”€โ—1
4โ—โ”€โ”€โ—โ”€โ”€โ—โ”€โ”€โ—2
5โ—โ”€โ—โ”€โ—โ”€โ—โ”€โ—3
Black dots are the data, green is the line, and the orange dots are the bends you can move
โ€“points = error  / 2 knobs
0goal 3.0 ↑30
Move the sliders until the line sits on the data.
The measure-and-move from Problem 2, now with these 2 knobs
Press a button.
What to take away
More knobs = more shapes you can draw. The arithmetic is the same multiply and add as in Problem 1; there is simply more of it.
How many knobs, and how far they reach
KnobsError at the best heights
214.280 bends. Straight, so it cannot make a hill
34.641 bend. Still not enough
42.052 bends. Reaches the goal
50.793 bends. Almost exactly on the data
Turning them by hand runs out around here. The best height for one knob depends on where the others are, so going through them one by one never finishes.
Where this pays off
When you cannot get there, ask whether you have enough knobs.
Real models bend the line in the same way (ReLU). Stack the stages vertically and it becomes “deep” learning — depth gets measured in Chapter 2. → the principles of AI