Are The Polls Biased Against Mitt Romney?

Many on the right are heavily invested in the argument that current polling is intentionally biased against Mitt Romney. Their argument is not very plausible.

Faced with a slew of polls that continue to show their candidate trailing, Republicans and pundits on the right have stepped up an argument that I’ve actually been hearing for months now, the argument that the polls are biased toward Democrats:

It has become a recurring refrain among some Republican pundits and observers each time a new poll shows President Obama or downballot Democrats doing well: Check the party composition.

Critics allege that pollsters are interviewing too many Democrats — and too few Republicans or independents — and artificially inflating the Democratic candidates’ performance. Pollsters counter that the results they are finding reflect slight changes in public sentiment — and, moreover, adjusting their polls to match arbitrary party-identification targets would be unscientific.

Unlike race, gender or age, all demographic traits for which pollsters weight their samples, party identification is considered an attitude that pollsters say they should be measuring. When party identification numbers change, it’s an indication of deeper political change that a poll can spot.

“If a pollster weights by party ID, they are substituting their own judgment as to what the electorate is going to look like. It’s not scientific,” said Doug Schwartz, the director of the Quinnipiac University Polling Institute, which doesn’t weight its surveys by party identification.

The debate has been amplified as the pace of public polling has accelerated after party conventions. Pollsters are finding diverging results, with consumers of political media left to decide which surveys better reflect the reality on the ground — or to accept the polls most favorable to their partisan leanings. New, less expensive methods for taking polls have led to a proliferation of surveys with varying results, so both sides have ample data to fit their desired narrative.

Gripes about the party-ID composition of poll samples are certainly not new: Eight years ago,Democrats were claiming polls showing a surge in Republican identification did not accurately reflect the makeup of the electorate.

Now, it’s Republicans making the case their voters are undersampled.

I’ve always been of the general opinion that when one side or the other in a campaign starts complaining about the polls it’s a sign that they are losing and they know it. In fact, in that regard, it’s interesting that you generally don’t hear the Romney campaign pushing back on the latest round of poll numbers outside of a few comments you might hear from a low-level surrogate like Bay Buchanan on a Sunday morning talk show. If the campaign could truly make a good case that the media polling was wrong, after all, it would be fairly easy to do so by strategically leaking some of their own internal polling at the state and national level. If there’s a discrepancy between the two then perhaps that would suggest that there’s a problem with the media polling. After all, one would assume that a campaign’s internal polls, which it uses to make all kinds of strategic decisions, would need to be as accurate as possible. The fact that we’re not seeing any polls leaked from Team Romney is, possibly a suggestion that they are essentially mirroring the state polls. Indeed, several recent reports have quoted campaign insiders as saying that their internal polling actually had Romney doing worse in state’s such as Ohio than the media polling that we’ve been seeing.  Romney’s supporters, of course, don’t have access to that internal polling, so in some sense creating the meme that the polling is biased is a way for them to convince themselves that hope is not lost.

Some conservatives, though, have gone even further than that, they’ve taken the public poll data and remade it in a way that appears to show Romney leading by what would amount to an Electoral College landslide:

Dean Chambers, a blogger on Examiner.com who writes from his home in Duffield, Virginia, took that complaint a step further — producing wide Romney leads far beyond what the Republican’s campaign or Republican pollsters have suggested is the case.

He created the site unskewedpolls.com, retooling national polling data this July after reading an ABC News/Washington Post poll that “just didn’t look right.” Looking at the internal data, Chambers saw that the polling unit had sampled more Democrats than Republicans.

“There’s no way they can justify that sample,” Chambers, 44, told BuzzFeed.

Since July, Chambers has re-weighted national polling data from organizations like Gallup, ARG, and the three networks, to fit the Rasmussen Reports partisan trends. Chambers has published 30 “unskewed” polls on his website and on examiner.com, a national network that pays independent bloggers on a wide range of subject by traffic. In the last month, Chambers’s tooled polls have Romney up by seven or more points.

Rasmussen’s most recent partisan breakdown shows that 37.6 percent of Americans consider themselves Republicans, 33.3 percent Democrats, and 29.2 percent Independents.

“A lot of people have a hard time believing it,” said Chambers, “but the simple way to explain it is, if Republicans make up 37 percent, and Democrats make up 33 percent, anyone can see that there’s no justification for having a poll with a sample size of 50 percent Democrats.”

The crucial part of this, of course, is that it assumes that Rasmussen’s turnout model, which differs from the one used by pretty much every other pollster out there is the correct one. As we learned in 2010, that isn’t necessarily the case, and it led Nate Silver to determine that Rasmussen was among the least accurate pollster of the entire 2010 election cycle. Whether this applies to Rasmussen’s numbers this year is something we won’t really know for sure until Election Day, although the fact that it is so inconsistent with everyone else out there is a good reason to doubt it’s accuracy. More importantly, Rasmussen himself rejects what Chambers is doing at his website, pointing out in the article that “you cannot compare partisan weighting from one polling firm to another.” That hasn’t stopped the site from becoming immensely popular on the right, though. It went viral when it was linked by Governor Rick Perry, the Drudge Report, and PJ Tatler. While some on the right have pointed out the rather obvious logical and statistical flaws in Chambers’ approach, expect UnskewedPolls.com to become quite the thing on the right over the next six weeks, especially if the actual polls continue to look bad for Romney.

Those who advocate the importance of looking at partisan breakdown in polls, including conservatives such as Ed Morrissey and Jim Geraghty, argue that it is important to examine the assumptions that pollsters use constructing their likely voter screens. This is most assuredly true not just for Party ID, but for pretty much any other demographic factor. A poll that is too heavily weighted toward women, for example, would tend to overstate President Obama’s support because of the Gender Gap, while a poll that undersampled African Americans in a state where they are significant part of the voting population would tend to overstate Governor Romney’s support. The same is true of polls that over or under sample Republicans or Democrats. That’s why it’s always a good idea to look beyond the topline numbers and take a look at the crosstabs of a polls. In a race that has tended to be close, as this Presidential race has, a poll that shows something atypical is likely to be doing so because of sampling errors rather than because there’s been some major change in the state of the race. It’s also a good reason why single poll results, while they may grab a headline on a given day, aren’t nearly as important as the direction the polls are moving in and the trends in the race. That’s why sites like RealClearPolitics are so helpful, because they aren’t likely to be skewed, to borrow a word, by a single poll result, and they are much better at showing trends.

It’s a huge leap from that argument, though, to the argument that all of the polling being done today is intentionally biased against Mitt Romney because of the Party ID demographics.

The Washington Post’s Chris Cillizza and National Journal’s Steve Shephard both have excellent posts today explaining why much of the furor over Party ID on the right is misplaced which I recommend to anyone interested in the subject, but Time’s James Poniewozik distills the biggest problem with the “biased polling” argument that the right is making right now:

[C]onsider what the “polls are biased” meme asks us to believe. That dozens of national and local polls are deliberately skewing their results to find a greater proportion of Democratic voters than there are. (Not only that, but the private campaign polling that Republicans themselves say bodes poorly for Romney is also biased for Obama!) And/or: voters themselves are—from guilt? political correctness? peer pressure?—reporting themselves as identifying Democratic more than they actually do. Evidently they didn’t do this in 2008, at the apex of Obama-mania-else the pre-election polling would have shown Obama with a 20% blowout lead-but they’re in the tank now, in concert-all of them.

Independent polling firms. Educational institutions. Local media. National media. The Wall Street Journal and Fox News—that’s right, Rupert Murdoch’s media properties are in there lifting away to give Obama a second term! What’s more, they’re all knowingly, erroneously reporting results that not only can be, but by definition will be, refuted by vote returns. Media outlets that rely on polls for credibility, pollsters that rely on accuracy for their very businesses’ existence-they’re ready to blow it all in a few weeks just to give Obama four more years.

Except—aha!—there is of course a way around this argument. Suppose by some chance the results of these “skewed” polls prove right on election day. How do we know that the skewing didn’t influence the result? That a drumbeat of positive polls for Obama didn’t brainwash voters into jumping the bandwagon, dry up campaign donations, depress Republican turnout? You call it an accurate poll? I call it a self-fulfilling prophecy! The game is rigged, my friend!

In the end, the “biased polling” argument is just a sub-set of the “biased media” argument that has been a meme among conservatives for decades, but it’s one that requires you to believe that all of these institutions, most of which are not even part of the media and are engaged in the business of polling for both academic and commercial purposes, are engaged in a vast conspiracy to cook the polling books to make it appear that the President is leading Mitt Romney in order to suppress the vote. With regard to the pollsters in particular, this strikes me as a fairly absurd argument. Isn’t it in their interests to get their polling as accurate as possible, especially since many of them engage in polling that they charge all types of clients, from politicians to businesses to non-profits, lots of money for? If they get a reputation for bad polling, that’s going to cut into their revenue stream. Poniewozik is also correct to point out that its the kind of thinking that provides conservatives with a ready made excuse if, as will probably be the case, the late polling in this race ends up being fairly close to the final results of the election. Romney lost the election, they can tell themselves, because “they” misled the public into thinking the President was going to win so, since the President won, that proves there was a conspiracy. These are the kinds of stories that ideologues tell themselves so they don’t have to admit that they lost because of an inferior candidate and ideas that the public rejected.

None of this is to say that polls are always correct, of course.  As I noted above, a poll whose demographics are out of whack is going to give crazy results, and when you run into a poll like that it’s best to skeptical about the results unless and until they are confirmed by other pollsters. For example, according to every single poll taken since just after the Republican National Convention, not including the Daily Tracking Polls which are a different matter entirely, have shown President Obama leading, with the exception of one CNN/ORC poll that showed the race tied. If tomorrow someone were to come out with a national poll showing Romney leading by 5 points, the best reaction to that poll would be to treat it skeptically, especially since little has happened that would justify such a large shift in public opinion. So, skepticism about individual poll results is a good idea. What the right is trying to do, though, is come up with an argument that rejects every single poll that doesn’t favor their candidate. That’s not skepticism, that’s partisan spin.

FILED UNDER: 2012 Election, Public Opinion Polls, US Politics, , , , , , , , , , , , , , , , , , , , , , , , ,
Doug Mataconis
About Doug Mataconis
Doug Mataconis held a B.A. in Political Science from Rutgers University and J.D. from George Mason University School of Law. He joined the staff of OTB in May 2010 and contributed a staggering 16,483 posts before his retirement in January 2020. He passed far too young in July 2021.

Comments

  1. Gustopher says:

    I think it is more likely that America is biased against Mitt Romney.

    If it wasn’t for his massive wealth, I doubt he could survive here.

  2. slimslowslider says:

    The only poll I believe is Jan.

  3. michael reynolds says:

    At this point Republicans are just trying to stave off panic. They’re trying to protect down-ballot races.

  4. Rafer Janders says:

    Are The Polls Biased Against Mitt Romney?

    This is the wrong question. The question is, is reality biased against Mitt Romney?

  5. bookdragon says:

    Fox News is biased toward Obama?

    http://www.foxnews.com/politics/elections/2012-presidential-race

    Seriously?

  6. Ron Beasley says:

    If there is one thing the right is good at it’s conspiracy theory’s.

  7. EddieInCA says:

    This is just a natural progression to what has been going on since 2001: Many on the right are creating their own reality.

    Science? We don’t need no stinkin’ science!
    College Education? Not necessary. Real Murkins don’t need an elite education!
    Global Warming? What Global Warming?
    Dinosaurs and Jesus walked the earth at the same time.
    Mitt Romney is likeable.
    Rush Limbaugh is smart.
    Sarah Palin is electable.

    And so on….

  8. C. Clavin says:

    Well, what else are they going to say?
    They are putting on a big push…a three day bus-trip thru Ohio. If they admit that they’re 8 points behind then they look a lot like Don Quixote…if they pretend it’s close Jan et al will still be be damp with excitement and maybe even donate some money.
    You have to look at the overall trend and the overall trend is clear.
    Anything can still happen…but right now it is about down-party races. If the election were held today…a la Christie…it might be a lot better for Republicans.

    Speaking of Bay Buchanan…did you see her get torched by Scarborough on MTP?

  9. Jeremy R. says:

    @Doug:

    In fact, in that regard, it’s interesting that you generally don’t hear the Romney campaign pushing back on the latest round of poll numbers outside of a few comments you might hear from a low-level surrogate like Bay Buchanan on a Sunday morning talk show.

    That’s generally been true, but I think they may have reconsidered for the sake of keeping enthusiasm and donations up:

    http://thehill.com/homenews/campaign/251413-gop-takes-aim-at-skewed-polls

    “I don’t think [the polls] reflect the composition of what 2012 is going to look like,” Romney pollster Neil Newhouse said in an interview.

    But Newhouse argues the RCP average, which is designed to neutralize outlying polls, is just an average of skewed polls that have a built-in bias for Obama.

    “Averaging bad polls that include skewed samples will get you a skewed result,” Newhouse told The Hill.

    “You had an extraordinary 2008 turnout among rural evangelicals, but Republicans stayed home in larger numbers,” Newhouse said. “This time you have a more enthused and energetic Republican electorate, and because of that, you’re not going to see the margins go up, you’ll see it narrow. So instead of a 7 [percentage-point advantage for Democrats], I anticipate something smaller than that.”

    The Right had a similar mantra last time around (in ’08). Instead of arguing the polls were wrong because of the sample/weighting, back then it was that they were incorrect because of the “Bradley/Wilder effect.”

  10. Fiona says:

    That’s not skepticism, that’s partisan spin.

    Partisan spin? Yeah, I guess you could call it that. I call it pulling stuff out of their collective ass. They’re creating a post-election narrative to blend with their other narratives, like voter fraud and the Black Panthers, to offer a contrafactual explanation of why their candidate lost. Not that facts matter to their audience.

  11. OzarkHillbilly says:

    Are The Polls Biased Against Mitt Romney?

    No, America is.

  12. OzarkHillbilly says:

    Somebody with a LOT of time on their hands needs to start a blog named StickingTheirHeadsInTheSand.com. They would have a dozen entries a day.

  13. grumpy realist says:

    I think it’s “let’s not let the marks keep any money that they might otherwise donate to us!”

    But yeah, the rest….Obama winning will send the screeching of the monkeys up to 11. I can’t wait to see what La Taitz says.

  14. J-Dub says:

    Mitt was so angry at the polls that he threw them all out the plane window.

  15. Just Me says:

    I don’t think the polls are biased although I suspect some of the races are going to end up closer than the polls, and while I intend to vote for Romney, I am pretty certain barring Obama committing a murder between now and election day, Obama will win reelection. Shoot I think there are still people who would vote for him if he did commit a murder.

    I do think the media is biased-or at least the media is extra, extra kind to Obama where they would have reamed any republican a new one over some of the recent events. Is there anyone who really thinks if it had been Bush president when an ambassador was murdered in the streets of Libya the media would have been focused on what the democratic candidate said about the failure of the state department?

    It isn’t so much that the media is picking on Romney, so much as it is they are letting everything with Obama slide where they wouldn’t with almost any other candidate or sitting president.

  16. PogueMahone says:

    Republicans and pundits on the right have stepped up an argument that I’ve actually been hearing for months now…

    Months?
    That’s all I heard about the polls back in ’08.

    My favorite was this doozy from Wizbang’s DJ Drummond in October 2008…

    The polls are wrong this year, very wrong. I have been saying this for months, and I have backed up my claim with both statistical and anecdotal support.

    Drummond goes on to claim that because the major polling firms were geographically located in and around the NY area, they are liberally biased and subject to “group think.”

    LOL!!

    Well, he’s at it again.

    ACCORDING TO GALLUP, ROMNEY SHOULD WIN

    Like a lot of folks, I follow opinion polls. Unlike a lot of people, I remember what I learned in my Statistics classes. This means that I try not to make the most common mistakes which happen when folks read and discuss polls. I have examined historical trends and the relevant vectors of this election in theGalluppoll, and find significant evidence that the Republicans’ ticket of Romney & Ryan will defeat the Democrats’ ticket of Obama & Biden.

    This election is awesome!!

  17. C. Clavin says:

    J-Dub wins the thread.

  18. Me Me Me says:

    If someone was really concerned about polls being skewed, they’d insist that they always be correctly weighted for GENDER. In presidential polls women should out number men by 5 to 10 points.

  19. Nikki says:

    Their heads really are exploding as they watch it all slip away…

  20. Smooth Jazz says:

    “As we learned in 2010, that isn’t necessarily the case, and it led Nate Silver to determine that Rasmussen was among the least accurate pollster of the entire 2010 election cycle.”

    Blah Blah Blah. Cherry Pick all you want. But if you’re going to quote a left wing hack such as Nate Silver, please point out to your readers that Rasmussen WAS THE MOST ACCURATE POLLSTER for the 2004 & 2008 Presidential elections while all those rigged polls you like to quote from NY Times/CBS/NBC/CNN/Marist/Pew, etc etc etc were all far down the accuracy list in 2004 & 2008.

  21. Barbara Carson says:

    @Just Me: Shoot you would vote for Mitt Witt even if he made the airlines require windows that opened. Yea another silver foot in mouth moment.

  22. Smooth Jazz says:

    “At this point Republicans are just trying to stave off panic. They’re trying to protect down-ballot races”

    At this point, you should lay off the Angel Dust. Juiced polls showing Dems out voting Repubs by 10% +/- are a fantasy. American ain’t that much in Love with the Dems to be sure. I doubt many down ballot Repubs have much to worry about. I think turnout this year reverts to a Dem 2% + norm ala 2004 & 2008. I doubt it will be the Dem 8%+ when Obama won 4 years ago. I think the Dems will be VERY LUCKY to hold on to the Senate. Forgot the house. Americans are not that much into Obama I assure you.

  23. An Interested Party says:

    Americans are not that much into Obama I assure you.

    Hmm…perhaps, but they certainly are that much against Romney I assure you…

  24. al-Ameda says:

    What a surprise – everything is biased against Republicans. Who knew?

  25. PJ says:

    @Smooth Jazz:

    Cherry Pick all you want.

    Have you looked in a mirror recently?
    Which group do the cherry picking here?
    The group trusting every pollster other than Rasmussen, or the group only trusting Rasmussen?

  26. mattb says:

    @Smooth Jazz @PJ:
    The discussion on whether or not Rasmussen is the most accurate poll misses the larger issue. The thing is, Rasmussen is also showing Obama in the lead, with a significant electoral college advantage. And therefore, people are arguing that Rasmussen is just another poll in the bag for Obama and not reflective of reality.

    So basically, after arguing for years that Rasmussen is the only poll that’s accurate, even that isn’t good enough anymore.

  27. PJ says:

    @mattb:

    So basically, after arguing for years that Rasmussen is the only poll that’s accurate, even that isn’t good enough anymore.

    That’s why the next step is to weigh Rasmussen polls after what they believe is true in their own hearts or gut.

    Or they can do what Townhall did, that is argue that a totally unscientific online poll at CNBC is a sign that people really agree with Romney about the 47%.
    Somthing like a poll by Erick son of Erick over at RedState which will show that Romney would win a larger share of the votes than Saddam Hussein or Fidel Castro in their sham elections.

    About Rasmussen, I’m starting to wonder if they will come around, so that they won’t lose all their credibility or if they will just stop polling due to some “unforeseen event”.

    I’m guessing Rasmussen will start to show a real gap between Obama and Romney after the debates, and they will credit it due to Romney tanking in the debates.
    Or perhaps they will say it’s due to voters being influenced by polls from other pollsters…

    But it will be Smooth Jazz’ next talking point.
    Obama stole the election by having his liberal pollsters influence voters…

  28. Mandy Evans says:

    @bookdragon:

    Fox is biased??
    Really??!!??
    Next you’ll be telling me the Pope is Catholic

  29. David Urbina says:

    This study shows Ram polls as the most accurate historically – http://www.youstand.com/news/10112/presidential-election-polls-300