Pollster Transparency

Honest pollsters should deposit their raw data with the Roper Center to improve transparency.

Regarding the DailyKos/Research 2000 poll scandal (in which Kos seems to have acted scrupulously and quickly and his pollster not so much), Charles Franklin has a smart suggestion:

There is one element of disclosure that has not been pushed, but which could significantly and easily reduce the chance of “pollsters” making up their data. Every media firm, including DailyKos, should write into their contracts the requirement that the raw data and complete questionnaire be deposited within two months with the Roper Center Polling Archive at the University of Connecticut. Two months is long enough that there is little remaining news value, but rapid enough that meaningful vetting and analysis is possible. By forcing this disclosure, by contract, the sponsors of polling would gain credibility for their polls while insisting that their pollsters live up to the standards of disclosure by AAPOR as well as making the raw data available for subsequent scrutiny.

Most major media polls already deposit their raw data with the Roper Center (including, Gallup, ABC/Post, CBS/NYT, NBC/WSJ, Pew, Time, Newsweek), though not necessarily as quickly as two months. Their example should encourage others to also deposit their data.

But most importantly, it is in the interest of the sponsors of polling to protect their reputation by requiring full disclosure and deposit of the data. Such practice would enhance the value of their polls, not diminish it.

It’s hard to argue with this.  Anything we can do to differentiate honest pollsters from the scam artists is in the interest of not only the honest pollsters but the general public.

As always in these cases, I should note my disclosure that my wife, Kimberly, is the chief operations officer of Public Opinion Strategies, a survey research firm.   Indeed, one of the consistent things that befuddles the partners is how often they’re underbid for national surveys despite a very lean operation.  Doing real polling is expensive because it’s so labor intensive to do all the calls.   Presumably, though, it’s much cheaper if you skip that step.

FILED UNDER: General
James Joyner
About James Joyner
James Joyner is Professor and Department Head of Security Studies at Marine Corps University's Command and Staff College and a nonresident senior fellow at the Scowcroft Center for Strategy and Security at the Atlantic Council. He's a former Army officer and Desert Storm vet. Views expressed here are his own. Follow James on Twitter @DrJJoyner.

Comments

  1. Doing real polling is expensive because it’s so labor intensive to do all the calls.

    I would assume that the firms that do robo-polling (Rasmussen, PPP, SurveyUSA) have much lower overhead, then.

  2. James Joyner says:

    One would think. I’m not sure whether we know yet whether there’s a decline in accuracy, though. I certainly hang up instantly when it’s a robo-voice on the other end.

  3. This Guy says:

    Anyone think to ask if Roper could handle such a task?

    Seems like the research market-place would take care of bad-actors when bad data leads to bad results.