Polls Coming Under Scrutiny

Don’t Ask Me: As Fewer Cooperate on Polls, Criticism and Questions Mount (WaPo)

It’s tough being a pollster these days, even in Minnesota. “It’s the rhetoric of mean-spiritedness, and it’s just gotten worse and worse and worse,” says Daves, a mild-mannered North Carolina native. Across the country, other pollsters are hearing similarly smash-mouth attacks. “The polls are wrong. They are all over the map like diarrhea,” rages filmmaker Michael Moore, who took time out from bashing Bush to bash the polls in a posting on his Web site. “You are being snookered if you believe any of these polls.”

Last month, the independent liberal group MoveOn.org attacked the venerable Gallup Organization in a full-page advertisement in the New York Times. “Why Does America’s Top Pollster Keep Getting It Wrong?” the headline fumed. The advertisement attacked the way Gallup defines likely voters for including too many Republicans and took a swipe at George Gallup Jr., son of the poll’s founder and an evangelical Christian who no longer plays a role in the company.

“Never saw anything like it,” says veteran pollster Andrew Kohut. “People have their guns drawn to a greater extent than usual and they’re shooting at us.” This year’s election-fueled furies are only the latest woes to befall the polling profession, already under siege and, some fear, on its way to extinction.

The combination of the closeness and bitterness of the race, the different likely voter screens used by various pollsters, and the natural tendency to want to shoot the messenger is a volatile one, apparently.

Cell phones, Caller ID and increasingly elaborate call screening technologies make it harder than ever to reach a random sample of Americans. Prompted by the popularity of do-not-call lists, a few state legislatures are considering laws that would lump pollsters in with telemarketers and bar them from calling people at home. Costs are soaring as cooperation rates remain at or near record lows. In some surveys, less than one in five calls produces a completed interview — raising doubts whether such polls accurately reflect the views of the public or merely report the opinions of stay-at-home Americans who are too bored, too infirm or too lonely to hang up.

Rick Klaastad, 41, of Springfield, Ore., had been working as a telephone interviewer on opinion polls for about a year when he dialed a number and an elderly man answered the phone. “He declined to be interviewed,” recalls Klaastad, who works for TNS of Horsham, Pa., the research firm that conducts the Washington Post-ABC News surveys. “But apparently he has some electronic device attached to the phone and the next thing I hear was this loud, painful sound. It was about as bad as it gets — imagine turning an FM stereo up full blast with a really good set of speakers. It bruised my eardrum.” Not all Americans say no so rudely — or painfully. But far more refuse to participate in surveys today than a few decades ago. In the 1960s, it was common for two-thirds of those contacted to complete a telephone survey. But participation dropped steeply through the 1980s and early 1990s, when it appears to have leveled off.

No surveys are immune. “Phone surveys are suffering, but so are response rates to mail surveys and even mall intercept surveys” in which people are interviewed while shopping, says Tom Smith, director of the General Social Survey at the University of Chicago, the best source of data on social trends in the United States. “All of the dominoes are being knocked down because the whole table is being shaken.” Currently cooperation rates hover at about 38 percent for the big national media surveys conducted over several days, but can dip down into the teens for surveys completed in a single night, says Jon Krosnick, a psychologist at Stanford University who has completed a groundbreaking study of response rates. Even exit polls are feeling the pinch. In each of the past three presidential elections the proportion of people who agree to be interviewed after leaving the voting booth has dropped — from 60 percent in 1992 to 55 percent in 1996 to 51 percent in 2000.

Interesting stuff, more of which is at the link (it’s a very long piece).

All I want to know is where do I get one of those neat phone blaster thingies?

FILED UNDER: Campaign 2004, Public Opinion Polls
James Joyner
About James Joyner
James Joyner is a Security Studies professor at Marine Corps University's Command and Staff College and a nonresident senior fellow at the Scowcroft Center for Strategy and Security at the Atlantic Council. He's a former Army officer and Desert Storm vet. Views expressed here are his own. Follow James on Twitter @DrJJoyner.

Comments

  1. Paul says:

    The polls are wrong. They are all over the map like diarrhea,” … “You are being snookered if you believe any of these polls.”

    Spoken like a man who’s behind in the polls.