Are The Polls Biased Against Mitt Romney?
Faced with a slew of polls that continue to show their candidate trailing, Republicans and pundits on the right have stepped up an argument that I’ve actually been hearing for months now, the argument that the polls are biased toward Democrats:
It has become a recurring refrain among some Republican pundits and observers each time a new poll shows President Obama or downballot Democrats doing well: Check the party composition.
Critics allege that pollsters are interviewing too many Democrats — and too few Republicans or independents — and artificially inflating the Democratic candidates’ performance. Pollsters counter that the results they are finding reflect slight changes in public sentiment — and, moreover, adjusting their polls to match arbitrary party-identification targets would be unscientific.
Unlike race, gender or age, all demographic traits for which pollsters weight their samples, party identification is considered an attitude that pollsters say they should be measuring. When party identification numbers change, it’s an indication of deeper political change that a poll can spot.
“If a pollster weights by party ID, they are substituting their own judgment as to what the electorate is going to look like. It’s not scientific,” said Doug Schwartz, the director of the Quinnipiac University Polling Institute, which doesn’t weight its surveys by party identification.
The debate has been amplified as the pace of public polling has accelerated after party conventions. Pollsters are finding diverging results, with consumers of political media left to decide which surveys better reflect the reality on the ground — or to accept the polls most favorable to their partisan leanings. New, less expensive methods for taking polls have led to a proliferation of surveys with varying results, so both sides have ample data to fit their desired narrative.
Gripes about the party-ID composition of poll samples are certainly not new: Eight years ago,Democrats were claiming polls showing a surge in Republican identification did not accurately reflect the makeup of the electorate.
Now, it’s Republicans making the case their voters are undersampled.
I’ve always been of the general opinion that when one side or the other in a campaign starts complaining about the polls it’s a sign that they are losing and they know it. In fact, in that regard, it’s interesting that you generally don’t hear the Romney campaign pushing back on the latest round of poll numbers outside of a few comments you might hear from a low-level surrogate like Bay Buchanan on a Sunday morning talk show. If the campaign could truly make a good case that the media polling was wrong, after all, it would be fairly easy to do so by strategically leaking some of their own internal polling at the state and national level. If there’s a discrepancy between the two then perhaps that would suggest that there’s a problem with the media polling. After all, one would assume that a campaign’s internal polls, which it uses to make all kinds of strategic decisions, would need to be as accurate as possible. The fact that we’re not seeing any polls leaked from Team Romney is, possibly a suggestion that they are essentially mirroring the state polls. Indeed, several recent reports have quoted campaign insiders as saying that their internal polling actually had Romney doing worse in state’s such as Ohio than the media polling that we’ve been seeing. Romney’s supporters, of course, don’t have access to that internal polling, so in some sense creating the meme that the polling is biased is a way for them to convince themselves that hope is not lost.
Some conservatives, though, have gone even further than that, they’ve taken the public poll data and remade it in a way that appears to show Romney leading by what would amount to an Electoral College landslide:
Dean Chambers, a blogger on Examiner.com who writes from his home in Duffield, Virginia, took that complaint a step further — producing wide Romney leads far beyond what the Republican’s campaign or Republican pollsters have suggested is the case.
He created the site unskewedpolls.com, retooling national polling data this July after reading an ABC News/Washington Post poll that “just didn’t look right.” Looking at the internal data, Chambers saw that the polling unit had sampled more Democrats than Republicans.
“There’s no way they can justify that sample,” Chambers, 44, told BuzzFeed.
Since July, Chambers has re-weighted national polling data from organizations like Gallup, ARG, and the three networks, to fit the Rasmussen Reports partisan trends. Chambers has published 30 “unskewed” polls on his website and on examiner.com, a national network that pays independent bloggers on a wide range of subject by traffic. In the last month, Chambers’s tooled polls have Romney up by seven or more points.
Rasmussen’s most recent partisan breakdown shows that 37.6 percent of Americans consider themselves Republicans, 33.3 percent Democrats, and 29.2 percent Independents.
“A lot of people have a hard time believing it,” said Chambers, “but the simple way to explain it is, if Republicans make up 37 percent, and Democrats make up 33 percent, anyone can see that there’s no justification for having a poll with a sample size of 50 percent Democrats.”
The crucial part of this, of course, is that it assumes that Rasmussen’s turnout model, which differs from the one used by pretty much every other pollster out there is the correct one. As we learned in 2010, that isn’t necessarily the case, and it led Nate Silver to determine that Rasmussen was among the least accurate pollster of the entire 2010 election cycle. Whether this applies to Rasmussen’s numbers this year is something we won’t really know for sure until Election Day, although the fact that it is so inconsistent with everyone else out there is a good reason to doubt it’s accuracy. More importantly, Rasmussen himself rejects what Chambers is doing at his website, pointing out in the article that “you cannot compare partisan weighting from one polling firm to another.” That hasn’t stopped the site from becoming immensely popular on the right, though. It went viral when it was linked by Governor Rick Perry, the Drudge Report, and PJ Tatler. While some on the right have pointed out the rather obvious logical and statistical flaws in Chambers’ approach, expect UnskewedPolls.com to become quite the thing on the right over the next six weeks, especially if the actual polls continue to look bad for Romney.
Those who advocate the importance of looking at partisan breakdown in polls, including conservatives such as Ed Morrissey and Jim Geraghty, argue that it is important to examine the assumptions that pollsters use constructing their likely voter screens. This is most assuredly true not just for Party ID, but for pretty much any other demographic factor. A poll that is too heavily weighted toward women, for example, would tend to overstate President Obama’s support because of the Gender Gap, while a poll that undersampled African Americans in a state where they are significant part of the voting population would tend to overstate Governor Romney’s support. The same is true of polls that over or under sample Republicans or Democrats. That’s why it’s always a good idea to look beyond the topline numbers and take a look at the crosstabs of a polls. In a race that has tended to be close, as this Presidential race has, a poll that shows something atypical is likely to be doing so because of sampling errors rather than because there’s been some major change in the state of the race. It’s also a good reason why single poll results, while they may grab a headline on a given day, aren’t nearly as important as the direction the polls are moving in and the trends in the race. That’s why sites like RealClearPolitics are so helpful, because they aren’t likely to be skewed, to borrow a word, by a single poll result, and they are much better at showing trends.
It’s a huge leap from that argument, though, to the argument that all of the polling being done today is intentionally biased against Mitt Romney because of the Party ID demographics.
The Washington Post’s Chris Cillizza and National Journal’s Steve Shephard both have excellent posts today explaining why much of the furor over Party ID on the right is misplaced which I recommend to anyone interested in the subject, but Time’s James Poniewozik distills the biggest problem with the “biased polling” argument that the right is making right now:
[C]onsider what the “polls are biased” meme asks us to believe. That dozens of national and local polls are deliberately skewing their results to find a greater proportion of Democratic voters than there are. (Not only that, but the private campaign polling that Republicans themselves say bodes poorly for Romney is also biased for Obama!) And/or: voters themselves are—from guilt? political correctness? peer pressure?—reporting themselves as identifying Democratic more than they actually do. Evidently they didn’t do this in 2008, at the apex of Obama-mania-else the pre-election polling would have shown Obama with a 20% blowout lead-but they’re in the tank now, in concert-all of them.
Independent polling firms. Educational institutions. Local media. National media. The Wall Street Journal and Fox News—that’s right, Rupert Murdoch’s media properties are in there lifting away to give Obama a second term! What’s more, they’re all knowingly, erroneously reporting results that not only can be, but by definition will be, refuted by vote returns. Media outlets that rely on polls for credibility, pollsters that rely on accuracy for their very businesses’ existence-they’re ready to blow it all in a few weeks just to give Obama four more years.
Except—aha!—there is of course a way around this argument. Suppose by some chance the results of these “skewed” polls prove right on election day. How do we know that the skewing didn’t influence the result? That a drumbeat of positive polls for Obama didn’t brainwash voters into jumping the bandwagon, dry up campaign donations, depress Republican turnout? You call it an accurate poll? I call it a self-fulfilling prophecy! The game is rigged, my friend!
In the end, the “biased polling” argument is just a sub-set of the “biased media” argument that has been a meme among conservatives for decades, but it’s one that requires you to believe that all of these institutions, most of which are not even part of the media and are engaged in the business of polling for both academic and commercial purposes, are engaged in a vast conspiracy to cook the polling books to make it appear that the President is leading Mitt Romney in order to suppress the vote. With regard to the pollsters in particular, this strikes me as a fairly absurd argument. Isn’t it in their interests to get their polling as accurate as possible, especially since many of them engage in polling that they charge all types of clients, from politicians to businesses to non-profits, lots of money for? If they get a reputation for bad polling, that’s going to cut into their revenue stream. Poniewozik is also correct to point out that its the kind of thinking that provides conservatives with a ready made excuse if, as will probably be the case, the late polling in this race ends up being fairly close to the final results of the election. Romney lost the election, they can tell themselves, because “they” misled the public into thinking the President was going to win so, since the President won, that proves there was a conspiracy. These are the kinds of stories that ideologues tell themselves so they don’t have to admit that they lost because of an inferior candidate and ideas that the public rejected.
None of this is to say that polls are always correct, of course. As I noted above, a poll whose demographics are out of whack is going to give crazy results, and when you run into a poll like that it’s best to skeptical about the results unless and until they are confirmed by other pollsters. For example, according to every single poll taken since just after the Republican National Convention, not including the Daily Tracking Polls which are a different matter entirely, have shown President Obama leading, with the exception of one CNN/ORC poll that showed the race tied. If tomorrow someone were to come out with a national poll showing Romney leading by 5 points, the best reaction to that poll would be to treat it skeptically, especially since little has happened that would justify such a large shift in public opinion. So, skepticism about individual poll results is a good idea. What the right is trying to do, though, is come up with an argument that rejects every single poll that doesn’t favor their candidate. That’s not skepticism, that’s partisan spin.