September 29, 2015 sam fisher

The Hot Hand Fallacy Is Safe

There has been a lot of talk related to a Wall Street Journal Article on a recent research paper by Joshua B. Miller and Adam Sanjurjo. After taking a closer look, I have identified the exact flaw in their math that led to the wrong answer, and I will illustrate how the correct math — combined with the very approach they used — leads to the correct answer of 0.5, not 0.4.

Before diving in to the flaw of the paper, let’s first solve this problem the easy way, by counting. Take a close look at the chart below, which is the main chart for the argument of the paper:

Screen Shot 2015-09-29 at 10.02.45 AM

In the 16 equally likely cases, there are 24 total heads among the first three flips. In 12 cases, heads is then followed by tails, and in 12 cases, heads is followed by heads. Of course, 12/24 = 0.5, the probability we expected all along.

So how did the authors get it wrong (and come up with the 0.40 number?) The answers starts with the chart. The key math being done is tricky, and it is done behind the scenes. All of the numbers you see on the screen are correct, except for the most important line: “Expected Value.” First, let’s see how they get to 0.4.

The authors weighted the probabilities in the rightmost column by the number of rows in each group (3, 6, 4, and 1, respectively), divided by the total number of rows with at least one H (14). Add the sum-product together (0*3/14 + (1/3)*(6/14) + (2/3)*(4/14) + (1)*(1/14) = 17/42 = 0.4048 (which the authors write as 0.40).

This is incorrect, because it fails to weight the actual variable in question for the expectation we are summing over. We need to weight each probability by the number of H’s on the first three tosses of each row in its corresponding group. There are 24 such H’s (3, 9, 9, and 3, respectively, among the groups). The new sum-product (0*3/24 + (1/3)*(9/24) + (2/3)*(9/24) + (1)*(3/24) = 36/72 = 0.5) proves to us that the answer is what we all expected all along, 0.5.

Why do we need to weight by the number of H’s? As we are calculating the probability of an H occurring after another H, if it happens more than once in a row, that row needs to be weighted more, as the event occurs multiple times.

You probably remember dealing with a similar problem in school when calculating rate of speed. Consider the following: “If you average 40 mph on your 10 mile drive to work and 25 mph on the drive back, what is your average speed?” Using the authors’ method, you would get (40+25)/2 = 32.5 mph. But that is incorrect, because your drive to work takes much less time. The correct answer requires a few steps (and is 30.8 mph).

I do not know why the authors chose to do their weighting in the way they did, but it is mathematically invalid for the expectation they claim to be producing. Have no fears, the Hot Hand Fallacy is alive and well.

— Sam Fisher, CEO Right Call Consulting

Email: Sam “at” rightcallconsulting “dot” com