Outside the Beltway

Pollster Transparency

Honest pollsters should deposit their raw data with the Roper Center to improve transparency.

James Joyner · Wednesday, June 30, 2010 · 3 comments

Regarding the DailyKos/Research 2000 poll scandal (in which Kos seems to have acted scrupulously and quickly and his pollster not so much), Charles Franklin has a smart suggestion:

There is one element of disclosure that has not been pushed, but which could significantly and easily reduce the chance of “pollsters” making up their data. Every media firm, including DailyKos, should write into their contracts the requirement that the raw data and complete questionnaire be deposited within two months with the Roper Center Polling Archive at the University of Connecticut. Two months is long enough that there is little remaining news value, but rapid enough that meaningful vetting and analysis is possible. By forcing this disclosure, by contract, the sponsors of polling would gain credibility for their polls while insisting that their pollsters live up to the standards of disclosure by AAPOR as well as making the raw data available for subsequent scrutiny.

Most major media polls already deposit their raw data with the Roper Center (including, Gallup, ABC/Post, CBS/NYT, NBC/WSJ, Pew, Time, Newsweek), though not necessarily as quickly as two months. Their example should encourage others to also deposit their data.

But most importantly, it is in the interest of the sponsors of polling to protect their reputation by requiring full disclosure and deposit of the data. Such practice would enhance the value of their polls, not diminish it.

It’s hard to argue with this.  Anything we can do to differentiate honest pollsters from the scam artists is in the interest of not only the honest pollsters but the general public.

As always in these cases, I should note my disclosure that my wife, Kimberly, is the chief operations officer of Public Opinion Strategies, a survey research firm.   Indeed, one of the consistent things that befuddles the partners is how often they’re underbid for national surveys despite a very lean operation.  Doing real polling is expensive because it’s so labor intensive to do all the calls.   Presumably, though, it’s much cheaper if you skip that step.