The Pitfalls of Early Polling
 |
James Joyner | Sunday, February 29, 2004 |
Robert Prather has a lengthy excerpt from a subscribers-only article in WSJ that, in a nutshell, says not to pay much attention to presidential approval ratings at this point in the cycle.
Not only are approval ratings not very helpful, neither is a candidate’s standing in opinion polls that ask people for whom they plan to vote. In 1968 Hubert Humphrey led Richard Nixon until July; in 1980 Mr. Carter led Mr. Reagan until June and then again from August through most of October; in 1988, Michael Dukakis led the elder Bush until August; in 1992 the elder Bush led Mr. Clinton into June. All of these “leaders” lost.
Both Robert and the author of the piece he excerpts argue that the main reason for this is that most people just aren’t focused on the election this early. And that’s certainly true. That’s the explanation I’ve always given for this phenomenon.
I was listening to a pollster –I think John Zogby, but maybe not– on a radio talk show recently and he had a different explanation, though: The polls are indeed accurate snapshots of public sentiment at the moment but the candidate that’s trailing in the polls adjusts his campaign on the basis of his polling to more effectively target his message. He used the analogy of going on a diet, arguing that, if it’s February and your goal is to lose 50 pounds by November, you might want to step on a scale periodically to track your progress. That seems plausible as well.
A third explanation is that all the cases above are the exciting outlyers that we remember–elections where there was a late surge by a candidate. Presumably, the early polls were accurate in 1964, 1972, 1976, 1984, 1996, and 2000 or they’d have been mentioned, too.
About the Author: James Joyner is the publisher of Outside the Beltway and the managing editor of the Atlantic Council. He's a former Army officer, Desert Storm vet, and college professor with a PhD in political science from The University of Alabama. He lives just outside the Beltway in Alexandria, Virginia.
Follow James on FriendFeed | Twitter | Digg
- None Found
- John McCain’s Acceptance Speech
- OTB Latenight - Madness
- Caption Contest Winners
- Rich Work More than the ‘Working Class’
- Why Don’t Politicians Write Their Own Speeches?
- Palin’s Speech Reax
- Caption Contest
- Obama’s Legislative Accomplishments (Updated)
- Sarah Palin’s Convention Speech
- Sarah Palin Video (R-Rated)
- The Politburo Diktat linked with Communist Blogs
- Kalblog linked with Pessimism
- BoiFromTroy linked with Soyuz Blogyetski Sotsialistichiski Respublik
So at least 4 of 10 early polls were not indicative of the ultimate winner. That's not much better than a coin flip.
Well, we all know about the polls in 2000. They weren't worth anything at the END, much less in February.
Interesting to look at the elections you mention, however. '64 would have been right after Kennedy's assassination - hardly a typical occurrence. 1976 would have been after Watergate against Ford, who pardoned Nixon - hardly a typical occurrence. 1984 - Mondale won all of what? Minnesota? i could have polled that election.
Another thing that you didn't mention is that the incumbent, who's usually unchallenged, doesn't begin campaigning until after these early polls are taken.
Polling doesn't claim to have predictive powers. All the polls from February can tell us is, within the margin of error, what the people think now. To the extent they think anything, of course.
And all elections are outliers--they're anomalous events.
Actually polls don't tell us within the margin of error what the people think. The margin of error is the standard deviation of a probability distribution. That is to say that there is a 68% chance that what people think now is within the margin of error.
There is still a 32% percent chance that what people think now IS NOT within the margin of error.
If you wanted a 90% threshold on the certainty of a poll you'd probably have a margin of error of something like 10 or 15 points. But generally speaking, we focus on the expectation number of the poll which actually fairly low probability of being exactly right.
Bottom line is that the margin of error is not a boundary. It's a metric of certainty.
Most reputable polls are at the .95 level, or roughly three standard deviations.
Mondale should have been included, actually. He, too, was ahead in the polls all the way through 1983 (even John Glenn was ahead of Reagan - by as much as 17 points - in 1983) and the race was a statstical dead heat at this point in the 1984 cycle. Also, Dole was ahead of Clinton at various points in the 1996 cycle, including a 3-point lead in January.
IOW, every incumbent in the last 25 years has been behind at some point towards the end of their first term. Presidents' approval ratings have a tendency to crater in year 3, actually (Reagan's 1983 numbers, for instance, were terrible - IIRC, he hit ~21% approval at one point in the summer that year; then tax reform kicked in, the economy started rockin', and, well, we know what happened in Nov. '84...).
Sorry James. You're right about the margin of error thing. I had assumed that it was just standard deviation because polls are so often wrong or there is a descrepency between polls taken at the same time. Gallup apparently does claim to be 95% accurate. But looking more closely at what they are claiming, they are claiming that the margin of error due to sampling error is 3%.
Thats the error in estimating the probability of flipping heads after 100 test flips for instance.
(from Gallup)
"In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls."
Weighting the data for instance or not getting data from people who won't participate are major issues.
There is probably no way to find the true margin of error since the accuracy of any specific poll can rarely be verified.
See more here: http://www.pollingreport.com/sampling.htm
Right. All the MOE claims to show is sampling error. Still, polls taken very close to the election replicate the election rather closely virtually all the time.
Comments are Closed
