: Inside out…How to read a poll…
Howard Dean is leading in “the polls.” No…wait…it’s Clark. Yes, Clark is the leader.
While keeping track of the polls is certainly an excellent service that PollingReport.com provides, much of the information about the Democratic candidates tells us nothing. Or, rather, the information tells us one thing only: There’s a wad of people tied for the top spot, i.e. no clear front runner.
Interpreting such polls is not difficult to do as long as you remember a few basics (re: John Allen Paulos):
1- Random sample and sample size: To give the truest indication of what people think, a sample should be randomly chosen from among a given population. The method of reaching the sample can affect its randomness. For example, if you call people on the telephone you are selecting for people willing to answer a stranger’s call. Caller ID and screening have added interesting glitches to this method of polling. Next, the size of the sample should be large. How can you tell? That’s difficult to say. But we do know this: a random sample of 500 people from among the entire population of the U.S. returns more predictive results that 50 polled from a population of 2,800. Odd, but true.
2- Interpretation errors: Even if a sample is random and large, the data is open to Type I and Type II errors. One commits a Type I error when a true hypothesis is rejected. One commits a Type II error when a false hypothesis is accepted. The only way to know if a journalist committed these errors is to review the survey data, including how the sample was selected, how large it is, how surveyed, and the content of the questions. Without all of this information, interpreting a poll is mere guessing.
3- Confidence intervals: Should we believe the data? If a poll shows Clark ahead of Dean by 3 points should we be confident that Clark indeed leads? The confidence interval is expressed in a margin of error figure. Any poll that does not give you the MoE is worthless. Any results that fall within the MoE should be considered ties. In other words, if Clark leads Dean by 3 points in a poll with a +/- 5 MoE, then they are tied–no leader.
4- Nonsense data: Any poll in which respondents self-select (e.g. most internet or call-in polls) has a MoE too large to be taken seriously. Such polls may, however, have entertainment value.
Second to last line of bullet point #1. It would be 5 out of 2800 supposing that you meant 500 out of 2.8 million. Figured it was just a typo but the actual proportion adds weight to your claim.
Joshua…that’s no typo