Rasmussen Bias Redux

Rasmussen's sample is biased because they're polling on the cheap -- using robocalls, which by law can't dial cell phones, and otherwise cutting corners -- rather than because of some agenda to propagandize for the GOP. The end result, however, is the same: Polls that can't be trusted.

Nate Silver offers more analysis on the Rasmussen polls, which consistently skewed Republican:

Rasmussen’s polls have come under heavy criticism throughout this election cycle, including from FiveThirtyEight. We have critiqued the firm for its cavalier attitude toward polling convention. Rasmussen, for instance, generally conducts all of its interviews during a single, 4-hour window; speaks with the first person it reaches on the phone rather than using a random selection process; does not call cellphones; does not call back respondents whom it misses initially; and uses a computer script rather than live interviewers to conduct its surveys. These are cost-saving measures which contribute to very low response rates and may lead to biased samples.

Rasmussen also weights their surveys based on preordained assumptions about the party identification of voters in each state, a relatively unusual practice that many polling firms consider dubious since party identification (unlike characteristics like age and gender) is often quite fluid.

Rasmussen’s polls — after a poor debut in 2000 in which they picked the wrong winner in 7 key states in that year’s Presidential race — nevertheless had performed quite strongly in in 2004 and 2006. And they were about average in 2008. But their polls were poor this year.

This lends more credence to the view I expressed Wednesday: Rasmussen’s sample is biased because they’re polling on the cheap — using robocalls, which by law can’t dial cell phones, and otherwise cutting corners — rather than because of some agenda to propagandize for the GOP.   The end result, however, is the same:  Polls that can’t be trusted.

FILED UNDER: 2010 Election, Public Opinion Polls, US Politics, ,
James Joyner
About James Joyner
James Joyner is Professor and Department Head of Security Studies at Marine Corps University's Command and Staff College. He's a former Army officer and Desert Storm veteran. Views expressed here are his own. Follow James on Twitter @DrJJoyner.

Comments

  1. Brett says:

    I figured as much, although Silver does qualify by saying that they did fairly well in predicting the 2004 election. Have things changed that much since then in terms of respondents and cell phone usage, or did Rasmussen start doing much more of what is described above?

  2. Brillo says:

    Cell-phone-only households have gone from about 5% in 2005, to a bit over 25% in 2010. [http://www.pollster.com/blogs/2009-05-06_NCHS1-thumb-550×412.png] Additionally, this group is much more heavily Democratic than households with both cell phones and landlines, or those with just landlines.

  3. Elvis Elvisberg says:

    Your reason is plausible enough, that he’s just doing it on the cheap. But he’s also heading on the NR fundraising cruise. Maybe he’s doing his job poorly on purpose, because he knows it’s a way of currying favor with people who like to be lied to.

  4. James says:

    The end result, however, is the same: Polls that can’t be trusted.

    Seems there is a awful lot of money spent on Polling.
    How can this be if the results can’t be trusted?

    Bias: Must be some bias in this White Paper?

  5. Tano says:

    I don’ see the sense in your logic James. By explaining how the biased methodology is implemented, you do not thereby demonstrate that there is no political motivation driving the choices of methodology.

    Why does Ras weight by party ID, if it is not an accepted practice. Why does he weigh categories by their historical likeliness to vote, even when not doing horse-race polling near an election – so that none of his polls ever report the opinions of a sampling across the entire population? Why does he not implement some work-around or correction for the robo-call problem? Do other robo-callers have the same skew in their results?

    And of course, Rasmussen makes no effort at all to hide his political leanings – he has carved out a niche for himself as the pollster for the conservative team. He routinely propagandizes for them in his analytical articles and speeches. Does any other supposedly independent pollster do that?

    I don’t understand why you seem so quick to absolve him of being something it seems very clear that he is.

  6. James Joyner says:

    Why does Ras weight by party ID, if it is not an accepted practice.

    It’s a pretty standard practice. It’s just that others also factor other things into the screen.

    Why does he weigh categories by their historical likeliness to vote, even when not doing horse-race polling near an election – so that none of his polls ever report the opinions of a sampling across the entire population?

    Presumably, because his customers are interested in the next election rather than some pure form of “public opinion.”

    Why does he not implement some work-around or correction for the robo-call problem? Do other robo-callers have the same skew in their results?

    He’s the only major pollster relying on robo-calls, I think. Most use robo-dialing but have humans (mostly from places out West with neutral accents) do the polling.

    As to the rest, I don’t deny that he’s a Republican partisan. And he rose to national prominence by doing polls for Fox News. But his results were actually pretty good in the early days. Indeed, he and Zogby — both touted by Fox in the early days — were among the most reliable pollsters. Both of their methodologies seem to have become outmoded and they don’t appear to have adjusted for them.

    Beyond that, my tendency is to look for explanations other than venality when they plausibly exist. And they do here.