Gallup Won’t Be Polling The 2016 Presidential Primary Races

One of the nation's preeminent polling firms is sitting out the 2016 primaries.

polling-stick-figures-1

Gallup is getting out of the political polling business, at least when it comes to the Republican and Democratic primary races in 2016:

Gallup has been the country’s gold standard for horse-race election polling ever since its legendary founder, George Gallup, predicted Franklin Roosevelt’s landslide reelection in 1936.

But after a bruising 2012 cycle, in which its polls were farther off than most of its competitors, Gallup told POLITICO it isn’t planning any polls for the presidential primary horse race this cycle. And, even following an internal probe into what went wrong last time around, Gallup won’t commit to tracking the general election next year.

It’s a stunning move for an organization that built its reputation on predicting the winners of presidential elections. But it comes at a time of unusual tumult in the polling world. Other top-level brands like the nonprofit Pew Research Center have yet to poll the horse race, and still others have expressed concern about the accuracy of polling at a time when fewer people are reachable or willing to talk to pollsters.

Gallup had vowed to examine its methods closely after 2012. And after a lengthy post-mortem, Gallup editor-in-chief Frank Newport promised to be ready “when the next presidential election” arrived. But so far, Gallup hasn’t been willing to put its methods to the test.

Newport told POLITICO that Gallup has shifted its resources into understanding issues facing voters — and won’t be following the primary horse races, other than asking about how Americans feel about the individual candidates.

“We believe to put our time and money and brainpower into understanding the issues and priorities is where we can most have an impact,” he said.

But it’s a far cry from this time four years ago, when Gallup had already conducted 11 different surveys of Republicans’ presidential preferences.

Its horse-race polls have been missed this time around, because the number of candidates on the Republican side and the ways in which news organizations have attempted to winnow the field for debates have made polls more consequential than they’ve ever been.

“In this case, the problem is both cause and effect,” said Cliff Zukin, a Rutgers professor and the former president of the American Association for Public Opinion Research. “The difficulty in doing this well has caused major players to not participate. That means there’s even less legitimacy because people who know how to do this right aren’t doing it.”

Gallup’s reputation is greater than that of any other polling operation, though its track record was never flawless. It was among the outfits that missed Harry Truman’s victory over Thomas Dewey in 1948, but 12 years later it won plaudits for nailing John F. Kennedy’s razor-thin win over Richard Nixon.

In 2012, many national polls underestimated President Barack Obama’s standing leading up to election, but Gallup’s failure was especially visible because the Obama campaign had pushed back publicly against Gallup’s surveys. When Gallup, in mid-October of that year, released a poll showing Obama and Mitt Romney tied in the swing states, the Obama campaign — led by Joel Benenson, Obama’s lead pollster — went so far as to question Gallup’s methodology in a public memorandum.

Gallup’s final survey showed Romney leading Obama by 1 point — 4.9 points off from the final result, in which Obama prevailed by 3.9 points. It also misidentified the winner. That led to a lengthy and expensive effort by Gallup to retool its methodology, a process the pollster described back in 2013 as aimed at the next presidential election.

Gallup concluded that major parts of its methodology — using live interviewers to creli

“That is certainly one of the advantages that an election provides, and that is an external standard,” he said.

Gallup’s decision here is understandable for several reasons. The extent to which its horse race polling was so off in 2012 was something that made what was once the nation’s most respected polling authority into a target of criticism from campaigns and pundits alike. Given the fact that political polling is only a small part of Gallup’s business, and certainly not the part that makes money for the company, their revenue comes from the private polling that they do for businesses and advocacy groups who have long relied on Gallup to guide them on everything from business decisions to political strategies regarding an issue they are concerned about, and the 2012 polling that Gallup did proved to be quite embarrassing. That final Gallup poll that showed Romney ahead of President Obama by one point was just the final straw in a series of polls that seemed to be far more favorable toward the Republican ticket than most of the other polling. That’s why Galllup ended up being ranked along with Rasmussen as being among the least reliable pollsters of the 2012 cycle. That’s not exactly great public relations that makes money by selling its surveys to clients.

Gallup made an effort to explain what had gone wrong with its polling models in 2012, and promised that it would correct those errors going forward, but it seems that the taint of the inaccurate 2012 polling never went away. Combine that with the fact that this year’s election cycle seems to be presenting a dilemma to some pollsters thanks to the rise of Donald Trump and Bernie Sanders, and it’s easy to see why Gallup might want to at least sit the primary races out before diving into the 2016 General Election.

FILED UNDER: 2012 Election, 2016 Election, Public Opinion Polls, US Politics, , , , , , , , , , , , ,
Doug Mataconis
About Doug Mataconis
Doug Mataconis held a B.A. in Political Science from Rutgers University and J.D. from George Mason University School of Law. He joined the staff of OTB in May 2010 and contributed a staggering 16,483 posts before his retirement in January 2020. He passed far too young in July 2021.

Comments

  1. DrDaveT says:

    Gallup concluded that major parts of its methodology — using live interviewers to creli

    Cut-and-paste fail?

    Not sure what that paragraph was supposed to say — I followed the link and couldn’t find any content that matched the lead-in.

  2. Neil Hudelson says:

    @DrDaveT:

    Here’s the full paragraph:

    “Gallup concluded that major parts of its methodology — using live interviewers to call land lines and cellphones, while screening out people who hadn’t voted in recent elections — were still the preferred means to conduct election polls. That review continued through 2014, when Gallup conducted polls of the national generic congressional ballot for internal use, which Newport and his colleagues “analyzed very carefully,” he said Tuesday.

    Read more: http://www.politico.com/story/2015/10/gallup-poll-2016-pollsters-214493#ixzz3nuYFijML

  3. Pinky says:

    Would it be too much to hope that this is the beginning of the end of our belief in tracking polls?

  4. Ron Beasley says:

    I don’t think I am alone when I simply hang up on pollsters. You can’t get reliable results when people refuse to participate.

  5. Pinky says:

    @Ron Beasley: Pollsters are reporting an 8% response rate. Tell me that’s not garbage data.

    http://www.nytimes.com/2015/06/21/opinion/sunday/whats-the-matter-with-polling.html?_r=0

  6. DrDaveT says:

    @Pinky:

    Pollsters are reporting an 8% response rate. Tell me that’s not garbage data.

    That’s a sufficiently self-selected sample that there’s probably a PhD in it for someone who takes the time to analyze all the ways that people who choose to respond to polls differ from normal people.

  7. CSK says:

    @Pinky:

    That was interesting; thanks for the link. I wonder how this factors into Trump’s success in the polls? I can’t, offhand, connect any dots, but I find it interesting that while Trump is the most popular Republican candidate in a contest with his rivals, he’s the only Republican who loses in a head-to-head match-up with Hillary Clinton. Something doesn’t make sense, unless the people who are boosting Trump in the polls actually support the Democratic ticket.

    Anybody?

  8. DrDaveT says:

    @Pinky: OK, now that I’ve read the article, I have two slightly more substantive responses:

    1) I don’t necessarily believe the author when he says that the 8% sample is still representative. He’s a professional pollster — he has a vested interest in having people believe that.

    2) Having political polls become wildly inaccurate across the board strikes me as an unmixed good. It uniformly reduces the incentives for politicians to pay attention to polling, which has no downside that I can see.

  9. Pinky says:

    @CSK: I’d assume that if the average person hangs up on pollsters, then the people who don’t hang up on them are the, well, non-average. The crank who wants the pollster to get a piece of his mind. He’d be more likely to support eccentric candidates, say, Sanders or Trump. But that seems to go against the bit in the article that people overestimate their passion in terms of willingness to actually vote.

  10. Pinky says:

    @DrDaveT:

    I don’t necessarily believe the author when he says that the 8% sample is still representative. He’s a professional pollster — he has a vested interest in having people believe that.

    Huh? The author wrote the piece in order to question the polling profession. He’s not acting out of a vested interest at all. He warns about the representativeness of polls despite not having any statistical evidence to back his caution.

    Having political polls become wildly inaccurate across the board strikes me as an unmixed good. It uniformly reduces the incentives for politicians to pay attention to polling, which has no downside that I can see.

    I agree that we’d be better off without polls. But that would be a very different environment from the one we’re headed toward (or maybe already in), with lots of inaccurate polls. It’s unrealistic to expect political advisors to not try to glean information from even weak polls. Gallup may leave the field, but “Click to show your support!” isn’t going anywhere.

  11. Tony W says:

    During the 2012 race Gallup was way off base, and seemed to be ‘aspirational’ in the way Rasmussen is. I naturally assumed Roger Ailes had got to them and they had become part of the Right Wing Entertainment Complex – and I promptly ignored their results, as I do Rasmussen.

    It’s interesting to hear that they were actually trying to do a good job, and just incompetent at doing so. It kinda makes sense, to the degree they are an old-school polling house with loads of data about how dialing people’s home telephone numbers is super-accurate, but I guess I assumed somebody under 50 worked there and told them about cell phones and the internet.

  12. DrDaveT says:

    @Pinky:

    Huh? The author wrote the piece in order to question the polling profession.

    Not quite — read it a bit more closely. He refers to “…we pollsters…” early on, and the author credits say:

    Cliff Zukin is a professor of public policy and political science at Rutgers University and a past president of the American Association for Public Opinion Research.

    He’s the head of a professional society for pollsters, and bases his professional research on surveys. He’s responding to widespread “what the heck is wrong with polling?” articles with a factual, reasonable response that is nevertheless intended to defuse skepticism.

  13. Just 'nutha ig'rant cracker says:

    @Pinky: Haven’t read the article, but in education research, a common enthememe in critical pieces is “all those other guys don’t know what they are talking about.”

  14. DrDaveT says:

    @Pinky:

    But that would be a very different environment from the one we’re headed toward (or maybe already in), with lots of inaccurate polls. It’s unrealistic to expect political advisors to not try to glean information from even weak polls.

    Ah, but that’s an evolutionary process, in the Darwinian sense. There are selective pressures against candidates, parties, and advisers who rely on bad poll data.

  15. CSK says:

    @Pinky:

    I don’t disagree, but none of this explains why people who would support Trump in the primaries would vote against him in the general. If I support someone in the primary, and he or she wins, I support them in the general.

    Look at it from the reverse. It’s like someone saying, “Well, I’m supporting Hillary Clinton against Sanders and Biden, but if she wins the primary, I’m supporting the Republican.”

    Doesn’t make sense.

  16. Pinky says:

    Tony, Dave, and Cracker – conspiracy theorists?

    Tony assumes that evil conservatives “got” to Gallup, as if the right wouldn’t have an interest in good polling data. Dave slights the author of the article about what’s wrong with polls, accusing him of covering up what’s wrong with polls, just because he’s a pollster. Cracker didn’t read the article and I guess is responding to what Dave said. Are you guys so locked inside your ideological bubble that the New York Expletive Times is too scary to read? This was a sound article, matching a lot of other work I’ve read on the weaknesses of polling data.

  17. Pinky says:

    @CSK: I’d need to see the specific poll you’re referring to, but my guess would be that they show Trump leading among likely Republican voters, and Clinton beating Trump among all likely voters.

  18. DrDaveT says:

    @Pinky:

    Dave slights the author of the article about what’s wrong with polls, accusing him of covering up what’s wrong with polls, just because he’s a pollster.

    OK, I’m confused. A pollster (not hiding the fact that he’s a pollster) writes an article called “What’s the Matter with Polling?”. In it, he admits (as you noted earlier) that response rates are dropping like rocks — 8% in 2014, and probably still falling. He admits that nobody knows how to grab a random sample of Americans, and that the best known approximations are too expensive to actually use. He admits that recent polls of major elections have been much more wrong than usual. No cover-up in sight — he’s being completely above-board about that.

    The only place I suggested he might be sugar-coating the facts is in his assertion that the 8% is still a representative sample of Americans, which he bases on an unelaborated assertion that the American Community Survey somehow validates them. I’m a data scientist by profession; I’d want to know exactly what it is about the ACS that he thinks shows that the 8% really is representative. It would have to be awfully strong evidence to outweigh a self-selected 8% subsample.

    As for the apparent conflict of interest, it couldn’t really be more obvious. If an economist were talking about why econometric models should be trusted even when they’ve been wrong recently, or a sports tout were talking about why his picks are good despite his recent history of getting it wrong, nobody would bat an eye at someone noting the clear incentive to spin. How is this any different?

    You don’t have to be a conspiracy theorist to suspect that realtors want to sell houses, or that brokers want you to trade.

  19. Just 'nutha ig'rant cracker says:

    @Pinky: In terms of polling data, some of my students 10 years ago did some research that ended up making me quite cynical about the whole polling industry, so I did not read the article because I really don’t have much interest in the topic. I wrote what I did in response to your assertion that because he was a pollster, he would have no vested interested in being critical of polling. I was just noting that the scenario you present does not always match my experience reading educational research and presenting a model for how the vested interest displays itself.

    In general, I believe that Americans are too feckless and incompetent to conspire to do anything. In addition, conspiracies require more trust than most Americans are willing to give to anyone or anything.

  20. Tony W says:

    @Pinky:

    as if the right wouldn’t have an interest in good polling data

    Why would the right embrace this form of science any more than they do other forms? Case in point, we saw Karl Rove blown away after 2012 because he refused to acknowledge reality. Romney allegedly didn’t prepare a concession speech because the polls he acknowledged were the ones showing him winning.

    BTW: it’s not like it can’t be done – Nate Silver seems to have it figured out.

    Republicans regularly make up their own facts on a broad range of issues, and polling is not exempt.

  21. Pinky says:

    @Tony W: Dude, you’re the guy who just said that you thought Gallup was conspiring against you. That puts you on the level of anti-vaxxers. Don’t talk to me about science.

    @DrDaveT: Other dude, I don’t mean this in a snotty way, but weren’t you the guy a couple of weeks ago who asked for help on percentages? But to the point above, questioning stats because of who provides them is also anti-science.

  22. DrDaveT says:

    @Pinky:

    weren’t you the guy a couple of weeks ago who asked for help on percentages?

    Cute. As noted at the time, the problem in that case wasn’t with my arithmetic.

    But to the point above, questioning stats because of who provides them is also anti-science.

    Which particular stat(s) are you alleging that I am questioning?

    If you can’t distinguish between disputing facts and disputing interpretation of those facts, I can’t help you. Disputing interpretation isn’t anti-science — it is science.

  23. Pinky says:

    @DrDaveT: Casting aspersions without looking at the underlying data isn’t science.

  24. DrDaveT says:

    @Pinky:

    Casting aspersions without looking at the underlying data isn’t science.

    You still haven’t said which data I’m allegedly not looking at. Be specific.

    As an aside, do you believe that it is “casting aspersions” on a car salesman to suggest that his characterization of a given car’s quality and value might not be unbiased?