Deep Black BoxOpen up the black box of AI one problem at a time, with your own hands. 21 problems in all.
before

From here on, it is your turn to do the work

Chapters 1 to 4 were the principles of AI. You turned knobs, brought the error down, and saw words turned into numbers. From here on you put those principles to work on real, publicly available data.

Kaggle and the Titanic

Kaggle is a place where the same data and the same question are public, anyone can answer it their own way, and a score comes back. People all over the world work the same problem at the same time. Problems whose answer is not settled yet sit there too.

The Titanic is the problem placed at its entrance. Almost everyone who starts on Kaggle touches it first. From the records of 891 people, you work out who survived.
What you do here scores there, just as it is.

All you are handed is two tables

The first five people of the training table (the actual train.csv from Kaggle). The green column Survived is whether they lived. 1 for survived, 0 for died. That one column is the only thing missing from the scoring table.

What you get to see is 891 people (training, numbers 1 to 891).
What gets scored is 418 people (scoring, numbers 892 to 1309). Not one number overlaps.

Real data comes with blanks

There are places where no record survives. In maths you can say “not defined there” and move on, but the 418 people being scored have blanks too, and you have to give an answer for every one of them.
So something has to go in the gap. What goes in is your decision. This course filled the age blanks with 28 (the middle value of the 891 ages).
What you hand in
418 rows — just a file with a 0 or a 1 for each person. “70% chance of surviving” will not be accepted. The score is how many of the 418 match what actually happened. How you got there makes no difference to the score.
Question 18

When a column is missing, no change to the rule gets you past it

First, split on sex alone

Using the Sex column, the 891 people split into 314 female and 577 male.
Everyone in the same group can only be given the same answer (all you are using is the one thing — male or female).
For each of the two groups, decide survived or died.
What you get right is the count on the side you chose. What is the largest number you can reach?
โ€“right (of 891)
0goal 701891
Decide an answer for both groups.
โ€“

Next, add the cabin class

The Pclass column has three values — 1st, 2nd and 3rd. Two sexes ร— three classes = 6 groups.
1st is the most expensive cabin; the bigger the number, the cheaper the room.
The groups went from 2 to 6. What happens to how many you get right?
(against the number you just reached — up, the same, or down)
Press one of the three.
The vertical line is 0.5 (half). A dot to the right means survivors outnumber the rest.
โ—‡ is exactly half (either answer gets the same count)
Decide survived or died for each of the six groups too. The goal is the largest number these six can reach (โ€“).
โ€“right (of 891)
500goal โ€“760
Decide an answer for each of the six groups.
โ€“