Deep Black BoxOpen up the black box of AI one problem at a time, with your own hands. 21 problems in all.
Question 6
Add more inputs
What we are about to build
hours studied · hours slept · attendance · student number → multiply and add → score
Columns
Formula
Knobs
1
score = a × studied + d
2
2
score = a × studied + b × slept + d
3
3
score = a × studied + b × slept + c × attendance + d
4
4
… + e × student number + d
5
d is the same thing as b in Problem 1 (the score for someone with zeros everywhere).
Choose the columns to use
Each tick adds one knob.
A new knob starts at 0 (that is, doing nothing yet).
Move it and watch which part of the picture rises.
Moving one knob on its own can make the error worse. That is fine.
Move a knob.
prediction = a × studied + b × slept + c × attendance + d
The marker here is “being on the right” (more hours studied). Raise a and
the further right someone is, the more they rise (â).
prediction = a × studied + b × slept + c × attendance + d
The marker here is “a darker dot” (slept well). Raise b and
the darker someone is, the more they rise (â). â short on sleep (4.0 h on average)
â average (7.1 h)
â slept well (9.5 h)
prediction = a × studied + b × slept + c × attendance + d
The marker here is “a bigger dot” (attended more). Raise c and
the bigger someone is, the more they rise (â). S = rarely there (0.56 on average)
M = average (0.74) L = there most of the time (0.90)
You can move the knobs. But setting them by hand is all but impossible
What you see
Combinations
2 knobs Problem 1
They split cleanly: a is the slope, b is the height.
So you could take them in turn: b first, then a
100 × 100 10 thousand
4 knobs here
Move the sleep or attendance knob and the shape of the picture does not change;
everyone’s prediction just goes up or down. There is nothing to take in turn
100×100×100×100 100 million
the walking-downhill method from Problem 2
What to take away
This arrangement — one multiplier for every column — is a fully connected layer.
The multipliers are called weights, and d is the bias.
Where this pays off
This is the first answer you submit on Kaggle. Every column you add brings one more knob.
When adding a column does not lower the error, that column carries no information about
what you are predicting (the score, here).