Deep Black BoxOpen up the black box of AI one problem at a time, with your own hands. 21 problems in all.
Question 19
The finer you split, the training score always goes up
In Problem 18 it was sex ร class = 6 groups. Now we keep adding columns.
Age takes 88 different values, so as it stands the groups come out far too fine. So we cut it every 10 years
(0-9, 10-19, …, blank). Fare we cut into four equal parts from the cheapest up. Where those cuts go is decided by a person. There is no right answer.
Split a group and you do the same thing again inside each part
— count how many lived and died in the band, and make the bigger side that band’s answer.
So the more you split, the fewer you miss. Even the 144 that came out level split by age into bands that lean one way.
Number of groups the 891 fall into: 1
— the bar below is all 891. The vertical lines are where the groups divide
โ
green = got right
/ white = missed (each group is answered with the bigger side)
right โmissed โ
โright = the training score (scored on the 891 whose answers you know)
500goal 891891
Move the slider.
What to take away
The finer you split, the higher the training score always goes. Taken to the limit it reaches 100%.
So if you pick whatever has the highest training score, you will always end up picking the one that learned them by heart.The training score is no measure of good or bad.
The same shape as Problem 11
Feed in the student number and the training set came out perfect while the test set fell apart. This is the same thing.
The name for it is overfitting. So where should you stop
— the measure for that comes up in the next problem.