Rasmussen Poll Overestimated Republican Vote
Rasmussen polls were biased toward Republicans by 3 to 4 points. Rigged results? Or screening error?
Nate Silver‘s analysis confirmed what was already widely thought:
Rasmussen polls quite consistently turned out to overstate the standing of Republicans tonight. Of the roughly 100 polls released by Rasmussen or its subsidiary Pulse Opinion Research in the final 21 days of the campaign, roughly 70 to 75 percent overestimated the performance of Republican candidates, and on average they were biased against Democrats by 3 to 4 points.
Oliver Willis, however, jumps to the wrong conclusion in “Rasmussen’s Rigged Polls For The 2010 Elections.”
There’s no evidence Rasumussen’s polls are “rigged.” And it would be demonstrably stupid for Rasmussen to “rig” his polls, which are intended for public consumption.
I’ve got no dog in this fight. Indeed, my wife is COO of a competing polling firm (see Disclosures). But it’s far more likely that, rather than intentionally delivering biased results and thus hurting his reputation, he’s simply applying a bad likely voter screen that consistently over-estimates Republican turnout. Just like Newsweek and others consistently over-sampled Democrats.
When it comes to very close races, polling is more art than science.
For example, in Nevada, ALL the polls had Angle up. The margin was narrow across the board, with Rasmussen/POR showing a 3 point lead and Mason-Dixon showing a 4 point lead. Reid won by 5.6 points!
In Washington, where it looks as though incumbent Democrat Patty Murray will eek out a close victory over Dino Rossi, Rasmussen actually over-polling her margin at +2. He was actually on the high end, with PPP giving Rossi a +2 margin and McClatchy/Marist giving Murray a +1.
Part of this can be explained by the vagaries of polling: You can never be quite sure that your response sample will match up with those who show up to vote. Or that they’ll vote the way they say they’ll vote. That’s especially true in a race like Nevada’s, where few voters actually wanted either of the two candidates and were having to hold their nose to vote for their least bad choice.
In “normal” elections — and it’s been a while since we’ve had one of those — likely voter screening was comparatively easy. People who voted last election are likely to vote in this election and those who didn’t vote last time are unlikely to vote this time. And it makes sense to weight the responses based on past performance, which typically means bumping up the Republican, white, and older demos.
But there’s a lot of uncertainty right now in the screen. We’ve had angry, energized electorates in three straight cycles. It’s just been different groups who have been angry and energized.
UPDATE: With a lot of caveats, Mark Blumenthal answers the question, “How Did The Polls Do?”
How did the final polls measuring the national “generic” vote preference compare to the ultimate Republican margin that will likely fall somewhere between six and seven percentage points? As the table below shows, the telephone surveys conducted by the Pew Research Center, Ipsos/Reuters, NBC/Wall Street Journal, CBS/New York Times and the two Internet-based surveys by YouGov/Polimetrix and Zogby Interactive all produced margins that are very close to the likely final result.
So, yes, Rasmussen was among the most biased, along with Fox. Both have reputations as Republican-leaning. But — surprise! — the venerable Gallup poll was off even more. In the same direction!
I should note that my wife’s firm, Public Opinion Strategies, makes up half of the NBC/WSJ poll, got the likely final result exactly right.
UPDATE 2: Actually, another — not mutually exclusive — explanation exists. It turns out that Rasmussen’s results, which are robocalled, don’t include cell phones. Neither do the Fox numbers. That alone could account for a 3-4 point Republican skew.
Yes, NBC/WSJ includes cell calls in their methodology. Oddly, so does Gallup — which really makes their numbers bad.
NOTE: This was originally published as a “Quick Picks” snippet, which I almost immediately decided to expand into a longer analysis.
That’s only if you assume Rasmussen’s lean is accidental.
“And it would be demonstrably stupid for Rasmussen to “rig” his polls, which are intended for public consumption.”
Why would it be stupid? Is there any evidence that Rasmussen is going to lose any business because his polls had a GOP bias? Rasmussen has been telling Republicans what they want to hear, on a daily basis, with approval polls, and issue polls – always skewing things to the right, not only under the guise that “likely voters” are the only people whose opinions should count on any matter of public policy, but even further, as we see today. His polls have a rightward bias even beyond likely voters.
This has been his ticket to fame and fortune. Its like all the propagandists who milk the rightwing community for millions of bucks by churning out these loony attack books – the Coulter model. Tell the right what they want to hear – its an easy way to make a buck.
If he tried his best to simply report accurately the opinions of the public, then he would just be an undersized competitor to the Pews, and Gallups and other established firms.
“But it’s far more likely that, rather than intentionally delivering biased results and thus hurting his reputation, he’s simply applying a bad likely voter screen that consistently over-estimates Republican turnout.”
When Rasmussen does this all of the time, it is probably not an accident. As noted by Tano, it guarantees them business. What would he gain by removing that “bad likely voter screen”?
Steve
@Tano and @Steve:
I get why they would produce biased public policy polls, which can be used for propaganda purposes for those paying for them. So, charges that they intentionally skewed their questions on the Gulf oil spill ring true.
But I don’t understand what business model there is in getting a reputation for providing bad numbers on horserace contests.
One of the local complaints (Illinois) about Rasmussen was that he wasn’t including third parties, or not all of them or at least not consitently. The third parties probably had a larger role in the outcomes here than would have been expected.
“But I don’t understand what business model there is in getting a reputation for providing bad numbers on horserace contests.”
First off, they are not shockingly bad. Who, besides Nate Silver (new to the scene) is even going to quantify the bias? And even if someone does. Ras would respond by just pounding home the examples where his numbers did well. Gotta get the presidential right, especially in the last week.
And who knows, maybe he provides better numbers to private companies or campaigns, and his public stuff is just intended as propaganda.
“But I don’t understand what business model there is in getting a reputation for providing bad numbers on horserace contests.”
None if you are really bad, but if you consistently tilt just a bit to one side, you can be sure that the side to which you tilt will always hire you. Everyone wants numbers favorable to their cause. You can use them to show that you are still in races. It encourages donations. The numbers will be attacked by the opposition, letting your side demonstrate the “liberal bias” in the media. On non-horse race issues, like views on taxes it will really pay off well. Of course this all falls apart if you are too far off. Overall, this is a great business model.
Just think about it. No one on the left believes Rasmussen numbers. Everyone on the right believes them. How much money can you make selling to 50% of the population with little competition?
Steve
“Rig the Polls”?
…is this still in question?
“No one on the left believes Rasmussen numbers. Everyone on the right believes them. ”
Rasmussen has Obama’s approval at 48% today.
Gallup has it at 43%.
Rasmussen and CNN/ORC don’t use their polls to provide information. They use their polls to drive the narrative. CNN/ORC uses their overnight junk polls solely for spin. Please check their pdfs linked here.
CNN/ORC is good enough to provide their crosstabs with MOE, and it is incontestable that their sample skews HEAVILY towards over 50 years, conservative, rural sample. You see there on page 8 of that pdf that don’t get enough 18-34 year-olds to report, and barely enough 35-49 year olds.
That’s fine, it’s their business model, and one can very conceivably get samples like that from robocalling in the evening and excluding cell phones. I’m not accusing them of dishonesty. It’s the dumb journos who run to press with this kind of junk poll that’s at fault here. They have no standards at all. But these pollsters know full well what biases are introduced by robocall. It’s just that their aim is to drive the conversation, not to provide information. In fact, here is an excellent study that quantifies that: AAPOR | Cell Phone Task Force: Table of Contents
I gave Rasmussen the benefit of the doubt for a while–assumed they were using a flawed likely voter screen–until it became clear that they were NOT polling races strategically in order to keep their con going (the senate race in MA was an obvious one but there were others). When do media manipulators ever admit to their motives and tactics? One reason standards of objectivity have declined so much, I think, is because there is this tacit agreement not to call out bias unless someone confesses (which offenders never do).
I don’t pay attention to polls if I know they were robot calling. What type of person is most likely to take the time to answer? I think they are older, whiter and more conservative (who are usually more intense about politics, it’s a matter of motive), than the public at large or even than likely voters. Talking to real people is the only way to get a good picture, in my opinion.