Online Polling and Margin of Error
 |
James Joyner | Friday, June 1, 2007 |
Mark Blumenthal has an interesting discussion of the accuracy of opt-in, online surveys as compared to those of traditional, telephone-based polls. While he takes issue with those who use the term “margin of error” with respect to the former, he nonetheless argues that they are becoming much more reliable.
I share Blumenthal’s concerns about opt-in surveys, and not just because my wife’s livelihood is tied to a polling firm that relies on traditional methodologies. Still, he’s right that survey research increasingly relies on weighting to control for demographic skews inherent in land-line telephone surveys and growing non-response rates. At some point, telephone surveys become essentially opt-in, too.
It wouldn’t surprise me if online polls caught up to and even supplanted telephone surveys in the not-too-distant future. My expectation is that firms, such as the one that employs my wife, will simply adapt. The art of interpreting polling data is only marginally tied to the technical means by which it is collected.
About the Author: James Joyner is the publisher of Outside the Beltway and the managing editor of the Atlantic Council. He's a former Army officer, Desert Storm vet, and college professor with a PhD in political science from The University of Alabama. He lives just outside the Beltway in Alexandria, Virginia.
Follow James on FriendFeed | Twitter | Digg
- None Found
- Caption Contest Winners
- The Political Economy of the U.S. Mortgage Default Crisis
- The Democrats and the Shock Doctrine
- Obama and Europe
- Supreme Court to Decide on Obama Citizenship
- International Implications of the Drop in Oil Prices (Updated)
- Obama’s Bipartisan Cabinet
- More Votes Found - Franken Hurt
- Caption Contest
- OTB Latenight - Paul Simon
I suspect that you are right-the firms will try to adapt to using online polls, and there was a time when telephone surveys were a bust, because telephone use wasn't saturated enough.
I think the one thing online pollsters have to be wary of, and work to combat is poll stacking, where a people of certain opinions decide to vote, revote, and revote some more. With telephone polls, the pollster calls the person, and they can't keep calling back to support their opinion with more votes.
How can you get a random sample with an online survey?
just_me: Online research surveys are not like the CNN front-page poll. They usually provide a passcode to targeted respondents, who can then take the survey once and only once.
Triumph: Tying into the above, telephone and internet polling usually get a list of potential respondents from a sampling agency. These lists can be tailored to a specific demographics breakout to match the demographics of the target market. This isn't usually an opt-in scenario, since most of the data probably comes from purchased user lists from other companies they did business with in the past.
I have worked in this industry before, and what online polling lacks in accuracy per response, they make up in volume of responses. A 1000 respondent online poll can be as accurate as a 500 respondent telephone poll. Also, having worked in the industry, I wouldn't trust the accuracy of any telephone poll where I wasn't able to monitor the interviewers and get the raw results. Frankly I'm surprised that James' wife's firm hasn't already moved into online polling, how can they compete on cost? Or do they sell based on that nebulous "quality" attribute?
Comments are Closed
