Yesterday I posted an MadMath blog about polling margin of error, asserting that the following claim was invalid: "If Candidates A and B differ by a number less than the margin of error, you can't be sure who is really ahead."
Well, it turns out that was a mistake on my part. As I groggily woke up this morning (the time where most of my best thinking occurs), I realized I'd made a mistake with a hidden assumption that the percentage of people supporting Candidates A and B were independent... when obviously (on reflection) they're not; in the simplest example every vote for A is a vote taken away from B. You can take A's support and directly compute B's support.
So if I do a proper hypothesis test with this understanding (H0: pA = 0.5 versus HA: pA > 0.5), with polling size n=100 and 55% polled support for A (as an example), you get a P-value of P = 0.1587 (significantly higher than the limit of alpha = 0.05 at the 95% confidence level), showing indeed that we cannot reject the null hypothesis.
In short, it turns out that the statement "A and B are within the margin of error, so we can't be sure exactly who is ahead", is actually correct at the same level of confidence as the margin of error was reported. In fact, to extend that result, there will be a window even if A and B are beyond the margin of error where you still can't pass a hypothesis test to conclude who is really ahead. (Visually, intervals formed by the margins-of-error overlap a little bit too much.)
Mea culpa. I removed the erroneous post from yesterday and left this one.
Subscribe to:
Post Comments (Atom)
Indeed, as Wikipedia points out, "The margin of error for the difference between two percentages is larger than the margins of error for each of these percentages..."
ReplyDeletehttp://en.wikipedia.org/wiki/Margin_of_error