Leo has been fascinated for some time by the probabilities of things. He’s telling us all the time about the probability that, for example, the light will turn red before we get to it, or a meteor will destroy the earth tomorrow. Of course, he’s totally making up the numbers, although he does know that a probability lies between 0 and 1 (inclusive), and that when something can’t possibly happen, it’s p=0.0, and when it either is sure to happen, or has already happened, it’s p=1.0 He also knows the general principle that p of n equally-likely events is 1/n, and he sort of get the p of complex things generally in the right range. For example, he puts the probability of the meteor at 0.0000…lots of zeros…and then a 1.
Given that he’s fascinated by probabilities, my main principle of teaching, “Catch ’em while they’re interested!”, suggests taking this to the next level, so today we started into Bayesian probability updating. In order to do this, I had to devise a particularly simple example, and biomedical tests (a typical example) isn’t really gonna work for him. So I invented a little card game, that goes like this.
Take N ordered cards I’ll start with 3 for these examples, so say J Q K of the same suit. (Could be 1 2 3, but I don’t want to get all confused with the numbers, so actually, I’m just gonna call them A B C for this explanation.) Shuffle them and lay them face down. Now, the question (hypothesis, in Bayesian terminology) is whether they have been laid out in order: ABC. p(H) at this point is obviously 1/6 (could be any of ABC ACB BAC BCA CAB CBA). Okay, so we turn over the first card. In the easy case, it’s not an A. Intuitively, p(ABC|first<>A) should be zero, and indeed, if we compute P(first<>A|ABC) = 0, and the whole equation just obviously becomes zero, regardless of the other terms.
Yes, I know, you’ve forgotten Bayes rule….Here you go: P(H|D) = (P(H)*P(D|H))/P(D)
Okay, so suppose first = A. What the new P(ABC|first=A)? Now we have to ask about P(D) and P(D|H). P(D|H) = 1…it had to be an A if the hypothesis was true. Okay, so what’s P(D). Well, that’s just 1/3rd since there are three letters. And P(H) was 1/6th (six possible sequences, as above). So you end up with:
P(ABC|first=A) = (1/6 * 1)/(1/3) = 3/6 = 1/2. So there’s now a 50:50 chance that it’s the right sequence, and this accords with intuition.
And so on.
It’s useful to go to very large numbers, and that’s easy to do using cards by just asking exactly the same question as above, but use all the cards. So P(ABCDEF…) is very very small from a shuffle. And P(D) is 1/52 the first time through, so although getting an Ace of Spades (assuming that’s what you are counting as the first ordered card) multiplies the probability 52 times, it’s still very very small.
So, now I have to start engaging Leo when he generates random probabilities, for example of meteorites hitting the earth tomorrow, to reenforce his Bayesian thinking.