Polls Are Imperfect Criteria For Determining Debate Eligibility, But They’re The Best We Have

Candidates who have been excluded from tomorrow's Fox Business Network are complaining, but their complaints ignore the fact that polling is the best objective criteria we have to determine debate eligibility.

Fox Business Debate Preview

On Thursday, Fox Business Network announced a lineup for the next Republican debate that pushed Chris Christie and Mike Huckabee down to the undercard debate and kept Lindsey Graham, George Pataki, and Jim Gilmore out of both debates completely because they failed to meet the network’s previously announced criteria. Not surprisingly, that decision is being criticized both by those excluded and by others:

 Fox Business Network is under fire for its selection of polls used to determine the lineup of next week’s GOP debate, after bumping two candidates off the main stage and excluding a sitting U.S. senator and a former three-term governor of New York altogether.

The networks and the polls they choose to use have had outsize impact on the presidential race, with critics saying they are artificially splitting the unwieldy Republican field into top-tier and lower-tier candidates, and making a big statement with whom they completely freeze out of the events.

Fox Business is no exception. New Jersey Gov. Chris Christie and former Arkansas Gov. Mike Huckabee for the first time were downgraded to the undercard debate next Tuesday after failing to get an average of 2.5 percent in four polls selected by the network. And South Carolina Sen. Lindsey Graham and former New York Gov. George Pataki were for the first time excluded because they weren’t able to register 1 percent support in any of the four polls.

At the crux of this most recent controversy is which polls Fox Business used. Of the four surveys, two have the campaigns and other observers crying foul: NBC News/Wall Street Journal and Investor’s Business Daily/TIPP. They say that had Fox Business used alternative and — in some people’s view, more credible — polls, Huckabee would still be on the main stage, and Graham would still be suiting up for the event.

Graham’s campaign on Thursday said it was “sincerely disappointed” — and on Friday added that it had complained about the polls used to decide the participants on each stage.

“I don’t think ‘contest’ is the right word, but we have raised our concerns about the NBC/WSJ poll’s inclusion and I hope they will see this as something similar to reviewing a bad call that is easily reversible,” Graham campaign manager Christian Ferry told POLITICO on Friday.

Fox Business pushed back on any criticism, stressing that it accepted the four most recent live-interview polls, without discrimination. But the network’s criteria for inclusion — like those if its affiliated cable-news station, Fox News Channel — were less than clear when the network rolled them out last month. In order to make the main stage debate, the channel said, candidates must achieve “2.5 percent or higher in an average of the four most recent national polls conducted through November 4th, as recognized by FOX Business to participate in the 9PM/ET debate.”

Which polls are “recognized” by Fox Business? “Such polling must be conducted by major, nationally-recognized organizations that use standard methodological techniques (i.e., live interviewers, random digit-dial sampling techniques and include both landlines and cell phones),” the network said.

But an analysis of the polls that were — and were not — included shows some of the candidates may have legitimate cause to quibble with Fox Business’ decisions. If either or both the NBC or IBD polls were excluded, Huckabee would have made the main stage, and Graham would be in the undercard event.

There’s an extended discussion at the linked article about some of the differences between the polls that were used to determine who got into which debate, or if they got into a debate at all that’s worth reading if you’re interested in the subject. The short version of the analysis, though, is one that we’ve discussed before, namely the fact that one should approach any set of poll averages with caution in no small part because different pollsters conduct their polls in different ways. Questions are worded differently, sample sizes may or may not be weighted differently, and pollsters may approach respondents who don’t seem committed to any candidate firmly differently. On that last point, for example, it’s becoming more frequent for pollsters to ask respondents for a second choice after asking who they support, or to ask if they are firmly committed to their candidate or see themselves possibly changing their minds before the time to vote arrives. Other pollsters may instruct their poll takers to push noncommittal respondents to indicate which candidate they might be leaning toward. Because of all that, taking the top line numbers from a group of polls, adding them together and then averaging them out can sometimes be like mixing apples and oranges.  This isn’t necessarily a big deal when you’re just looking at the poll averages to get an idea of where the trend lines are going over a period of time, but it becomes somewhat of a bigger deal when the averages are used to decide who does and doesn’t get invited to a debate.

This problem with the way debate eligibility is being determined leads Dan Balz to wonder why polls should be used to determine debate eligibility at all:

If the stage cannot accommodate everyone, what is the right number for a debate and how is that determined? Are eight candidates really better than 10? Wouldn’t four be better than eight? In that case, why not Trump, Carson, Rubio and Sen. Ted Cruz of Texas — the two leaders and the two candidates who are drawing attention as their poll numbers increase?

Is it better to use national polls or those from states like Iowa and New Hampshire, where the candidates are spending the most time? The stratification of the GOP field looks somewhat different in the state polls than in the national polls, and it varies between the two states. Another question: Should the big field be split in half, randomly, for each debate night, providing a different mix and therefore less predictable exchanges among the participants?

For those sponsoring debates, there is a natural tendency to seek ways to limit participation — and not just for logistical reasons. Network sponsors are looking for the kind of on-stage chemistry that will produce political drama and fireworks, high ratings and big revenues. That argues for including fewer candidates but comes at a cost of not giving the public a look at as many as possible.

It’s easy to use polls as a proxy for the public to rationalize cutting out a few candidates with dismal numbers. But on this, polls are imperfect and the cut points arbitrary. The first contests are still almost three months away, in a cycle in which conventional wisdom has taken a beating. Who should be picking winners and losers at this point?

Balz does raise some fair points here. For one thing, relying solely on national polling in this context doesn’t necessarily make sense given the fact that primaries are a state by state affair. A candidate who is doing well among Republicans in a state like Iowa or New Hampshire may not see that success reflected nationally, and thus would be penalized notwithstanding the fact that they could be a factor in early races. One way that this could be handled would be for debate criteria to include both national and state polls in the relevant primary states, which at this stage of the race would be early primary and caucus states like Iowa and New Hampshire. If you look right now at the national poll average compared to those from Iowa and New Hampshire, there isn’t much of a difference, but there are a few significant ones. Chris Christie has been moving up in recent national polling in the Granite State to the point where he is doing better than Rand Paul and, in the most recent poll, outperformed Carly Fiorina, Ted Cruz and Jeb Bush. In Iowa, Louisiana Governor Bobby Jindal is doing substantially better than he is anywhere else in the country and currently stands in seventh place ahead of Huckabee, Christie, and Rand Paul. Mike Huckabee is also doing better in Iowa than he is nationally. If Fox Business applied its 2.5% cutoff to the state polling as well as the national polls, then Christie, Jindal and Huckabee would have a strong case for inclusion in the main debate. Doing this, of course, would mean more people on the main stage which is not exactly the best way to improve debate quality, but it’s arguably a fairer way to use polling to determine debate eligibility at this point in the race.

There have been some suggestions that polling shouldn’t be used to debate eligibility at all, but then the question becomes what criteria you do use. If the networks were willing to use some kind of multi-debate format that featured every candidate in a field as large as the one the GOP has this year, perhaps that would solve some of the problems, but even you have to decide who gets invited. If you look at the FEC records for the people who have actually filed a declaration of candidacy for the Republican nomination, the list goes beyond just the fifteen people we’re familiar with, seventeen if you count Rick Perry and Scott Walker who have already dropped out. If you aren’t going to use polling to determine who gets invited, then how do you tell any of those candidates that they’re not invited? It’s worth keeping in mind in this regard that Federal Election Commission regulations require that debate sponsors establish “pre-established objective criteria to determine which candidates may participate in a debate.” Polls may be imperfect, but the reputable ones are generally objective, and I’m not sure what other objective criteria a debate sponsor could use to determine who should and should not be invited to a debate.

FILED UNDER: 2016 Election, Public Opinion Polls, US Politics, , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Doug Mataconis
About Doug Mataconis
Doug Mataconis held a B.A. in Political Science from Rutgers University and J.D. from George Mason University School of Law. He joined the staff of OTB in May 2010 and contributed a staggering 16,483 posts before his retirement in January 2020. He passed far too young in July 2021.


  1. C. Clavin says:

    What an amazing line-up of gravitas.

  2. CrustyDem says:

    Considering what a clusterf*ck the field is and the extreme volatility in the polls in advance of the primaries, I’d think that a random selection of the 5 candidates over 2% doing 2 separate 90 minute debates would be ideal. Any more than 5 on a stage is silly, and dragging the whole thing out for a 2 1/2-3 hr debate is terrible..

  3. CrustyDem says:

    @CrustyDem: Whoops, meant 10 candidates over 2%, two random pairings of 5 for 90 minutes each.. Ugh, Monday..

  4. michael reynolds says:

    There is a better mechanism, it’s called a political party.

    The GOP should be deciding who does and who does not get to the debate and if they exclude the wrong people they’ll hear about it from their voters. Allowing the media to turn this into a national primary and running roughshod over a selection process meant to move from state to state, is not okay. The media should not be picking GOP candidates, the party should.

  5. Ron Beasley says:

    The “debates” especially on the Republican side have become rather silly exercises but the networks get lots of eyeballs and add dollars. Perhaps one solution is to require the candidates to pay for the privilege of participating since they are little more than campaign ads anyway. If a candidate can’t afford it he/she is out.

  6. C. Clavin says:

    Can you imagine these folks in the group that met in Philadelphia in 1775?

  7. Mu says:

    The real problem is they didn’t put a cap in. If they had said from the beginning “we invite the top 8 with 2.5% or more support minimum” none of these arguments would happen.

  8. Gustopher says:

    How about making them take a 4 hour written test — no phones, no aides, no outside help of any kind — where they must do such things as:

    – Identify which countries/regions in the middle east are Sunni and Shia (partial credit given for getting it consistently backwards)

    – Identify real and fake quotes from the founding fathers

    – Explain the Laffer curve correctly (scored) and then where we are on it (unscored, hard to verify)

    – Calculate marginal tax rate.

    I know poll tests are unconstitutional, but no one has really considered debate tests. And, then we will have chopped the field down considerably by saying anyone who cannot get a B doesn’t get to take the stage.

  9. stonetools says:


    Or simply have them take an arithmetics test. From what I understand,based on their tax and budget plans, you would immediately start with a field of only 2 or 3.
    Right now, those smoke filled rooms are starting to look pretty good.

  10. John Peabody says:

    This is true if you want an entertaining, commercially viable product. If you wanted a true debate, you would include all candidates, with the same question to all, and equal time for all. This is not palatable for the commercial enterprises producing the debates, so we have reality-show rules

    That’s not a problem if you are watching “Survivor” or “The Biggest Loser”.

  11. SC_Birdflyte says:

    @stonetools: My thoughts exactly.

  12. Grewgills says:

    @John Peabody:
    I say go all the way. Lock them all in a house for the next couple of months and see what happens.

  13. Moosebreath says:


    Heh — that reminds me of one of Mark Twain’s essays, where he was able within 1 day to teach a variety of animals who normally attack each other to live together in harmony, but when he tried the same with humans of different religions, he returned to find that they had a dispute and carried it to a higher court.

  14. dmichael says:

    Doug appears to be saying that although it is not perfect, use of polls to determine debate eligibility is objective. For a completely different view, see this article: http://www.newyorker.com/magazine/2015/11/16/politics-and-the-new-machine?mbid=nl_151110_Daily_Letter&CNDID=38981805&spMailingID=8238268&spUserID=MTExMjc3MjkzNDU4S0&spJobID=800982802&spReportId=ODAwOTgyODAyS0

  15. Hello there! This post couldn’t be written much better! Going thgrough this article reminds me of mmy previous roommate!
    He always keptt preaching about this. I aam going to send this post
    to him. Pretty sure he’ll have a good read. I appreciate you for sharing!