A Brief PSA on Polling Error
It isn't unidirectional.
I have noticed that almost every time I hear someone talk about potential polling error going into next week the underlying assumption is almost always that the error will favor Trump.
Democrats think this out of anxiety (and 2016 PTSD).
Republicans think this out of hope (and, often, the belief that the polls are simply biased against them–and ongoing theme going back several cycles).
Now, it is certainly true that there are variables in the current election that create some additional uncertainty for polling (and hence the hedging that James Joyner wrote about this morning).
Some examples include:
- The largest usage of mail-in voting in our history.
- Voter turnout that appears to be headed for a possible record.
- The effects of the pandemic.
There is also the simple fact that polling, by definition, can has some amount of error.
But something to remember: it is true that when the dust settles we might find out that the polling underestimated support for Trump. But it is also possible that we could find out that there was a systematic error that underestimated support for Biden.
Simply put: just remember that errors can run both directions.
It seems like a lot of people need to hear this, as they seem to be running around thinking that the option are: the polls are accurate or there is polling errors that favor Trump when, in fact, the options are threefold:
- The polling is accurate.
- There is error favoring Trump.
- There is error favoring Biden.
There can be be little doubt that there are several unusual factors in this race. But, by the same token, we have almost four full years of presidential approval polling that provides evidence that the current presidential polls are likely accurate. We have pollsters who are spooked by 2016 and really, really, want to get this right and therefore are being cautious (and, again, 2016 was not a bad year for polling accuracy, despite conventional wisdom). We also have decades of pretty solid polling practices that has produced the current procedures that pollsters use.
So, yes, error is possible (even big errors are possible, although not likely).
But, again, errors can run both ways.
Also, the odds of a huge error is not that high.
Just some things to think about as we march towards the conclusion of this campaign season.
Yeah. My intellect is on board with you. I just wish the rest of me was.
I mean, we might not know who won by this time next week. We see a Supreme Court really champing at the bit to not allow votes to be counted, because why would that be good for democracy? I mean, just because some piker like that Gischer fellow thinks that counting every vote has a very strong priority in a system based on the sovereignty of the voter, why should the high-flying intellects of people like Brett Kavanaugh, that avatar of impartial wisdom, pay that any notice?
And of course, I’ve had to endure lots of people telling me not to worry so much. I wish I knew how to do that.
@Jay L Gischer: The courts, alas, are another matter.
In my more optimistic moments note that if the polling is accurate, the margins will be too large for the courts to matter. But all of my moments are not optimistic ones.
Plus polls all have a margin-of-error and usually operate at a 95% confidence level, not 100%. The precise poll percentages that get published give a false sense of precision – you really have to consider the entire spread.
In 2016 most polls were actually in the margin of error, so it wasn’t as bad as people have made it out to be.
It would be great if the polls are underestimating Biden’s support, and he wound up with close to 400 EVs to trump’s well under 200.
If we lived in a rational universe, trump would lose by Carter/Mondale levels.
@Steven L. Taylor:
As I worried on James’ polling post, this year, yes, looks like we’ll have safety margins. But in 2022 and 2024 we’ll have smaller margins (mid-term, revision to the norm, and absence of Trump) but we’ll have essentially the same Court. We really need a new voting rights act.
I will say again that polling wasn’t that inaccurate in 2016. Final polls had Hillary winning the popular vote 47-44 and she won 46-44.
Also, even states that were wrong – like Wisconsin – accurately predicted Hillary’s votes. WI final polls were 46-40 for Hillary. Trump won 46.1-46. This was due to large numbers of undecideds that broke for Trump in the wake of the Comey letter.
Neither of those conditions apply this year. There are very few undecideds and the Hunter Biden October Surprise is such a turd that all but the hackiest right wing outlets are reluctant to push it.
Get out and vote, and don’t relax until Biden is sworn in , but I feel safe in predicting a Biden victory with close to 400 EVs.
@Kathy: If we lived in a rational universe Hillary Clinton would be cruising to reelection, having put together a widely respected response to the Corona Virus.
@gVOR08: Before someone else points it out, if EDIT were working today that would be reversion, not revision.
@gVOR08:
Here’s something new for all of us to worry about: the election won’t be close enough for the Court to steal it for Pessimus Minimus. But one or two Senate races may get stolen with the COurt’s help, giving the GOP control.
IMO, the Republican establishment would prefer that to four more years of Orange Insanity
@gVOR08:
I think the one we have is fine. The piece the Supreme Court struck down a few years back can easily be re-instituted by a Democratic President and Congress—they just have to apply it to the country as a whole or at least to any states meeting (reasonably current) (Condition X).
Ben Garrison says the pollsters are LIARS!
the Great Red Wave of 2020
@James Joyner:
Have you seen people routinely waiting in line for 12 hours to vote?
That’s not fine. Wouldn’t even be fine if it was Republicans. A right that you cannot reasonably exercise isn’t a right.
Someone correct me if I’m wrong but isn’t it usually the case in our elections when more people vote, that favors Democrats? And if we really are headed for record voter turnout…
I’m feeling pretty good about the vote. Nate Silver has it 89 to 11 in his simulations. Trump could win, sure, but if he were getting traction we’d be seeing polls tighten and they just aren’t. A week ago Nate had it at 87 to 13.
This year is not remotely like 2016. Hillary never held onto a seven to ten point margin as Biden’s done for months now.
Voter intimidation and disenfranchisement efforts may get you a point or two but not enough. There’s no Comey letter. Everyone’s hip to the Russian bullshit. And Trump isn’t going to suddenly become a human being.
So I guess we’re going to finally get a chance to turn the country into a socialist hellscape where suburbs are on fire and buildings have no windows.
In fact, I’m optimistic enough that I have a bottle of Dom in the wine fridge.
Yes, I have a wine fridge. It came with the place.
@Gustopher: I bet you’ve heard that in theory there is no difference between theory and practice. In practice there is.
In theory, slavery and Jim Crow and the Civil Rights movement are historical facts. In practice they are present facts.
@Teve:
The Great Red Wave? Russia?
@PJ: A week from now he’s gonna be drawing cartoons about how George Soros paid Hunter Biden to use terrorist emails to hack the voting machines something something something, AOC buying T-bone steaks with food stamps.
@Teve: He did the same cartoon in 2018.
I feel like the p0lls are more likely to err in Trumps favor because of people lying to the pollsters. No Biden supporter will lie about it, but not everyone wants to admit they are a racist to a pollster.
Dave Wasserman tweet on Nov. 3, 2016:
“Five days from Election Day, it’s clear who has the momentum. And it’s not Hillary Clinton. This thing is close.”
Tweet today:
“Five days from Election Day, it’s clear who’s the favorite. And it’s not Donald Trump.”
@J-Dub:
I addressed that argument the other day. Short answer: It’s BS. There’s no evidence it happened last time, and no reason to think it’s happening now.
I’ll be honest.
I just can’t bring myself to take any of the scenarios where a president who has spent the vast majority of his term at 45% or lower approval, has always trailed his opponent and is currently within kissing cousins of losing by a double digit popular vote margin and wins said election, serious. Neither do I believe any of the imagined court machinations will be successful enough to overturn such a vote, especially when the state level polling is so against him, and it is backed up by state level polling showing that his fellow party members are scratching and clawing for their political lives in such stalwart red states like KS, SC, IA, GA.
And I haven’t even mentioned Texas which just might have a blue House next year.
So…
I acknowledge the fact that the future is unknowable, that it is in fact possible trump might actually dive thru a second electoral needle at 60 mph, but that is not the way I bet. I put several hundreds of dollars on the bet that not only would he lose but he would cost the GOP the Senate and even more house seats.
FTR, I. Don’t. Gamble. I make bets, yes, but I don’t make bets I think I might lose. And I very rarely lose.
@J-Dub: Kylopod’s response is good, but I have a shorter take: Trump supporters, as far as I’ve seen, are loud and proud, except in heavily blue states. But a heavily blue state, such as CA, where I live, is not going to go to Trump because Trump voters are undercounted.
Meanwhile, Trumpists in places like OH and PA love to wear their MAGA hats and yard signs and bumperstickers, etc. They aren’t shy about supporting Trump, not in the least.
@Teve: In this age of Trump, I’m inclined to think that quite a few people engage in boasting contests, you know, who can lie the biggest and the bestest!
@Jay L Gischer: Plus I doubt Trumpers think of themselves as racist. In fact I bet they occasionally start sentences with “I’m not a racist, but…” 😀
@Michael Reynolds:
I have (my last) bottle of Mumm from a visit to Rheims with my father sat waiting in my wine chiller cabinet.
I think Dad would have approved.
(And mother even more so, except she never did like champagne that much)
Will save it for Friday though. 🙂
270 to-Win now showing PA back over the 5% margin, and their polling map showing 279 for Biden for all states (and split votes) at or better than 5%.
Latest Economist central prediction Biden 350, with 95% confidence of Biden victory.
@Teve: I guessed wrong. I was thinking a muscular Trump in baggies (but only slightly) on a board shouting “Kowabunga, Dude!”
(I think he needs a better idea man, but since the job is only going to last for another 3 or 4 days…)
@Kylopod: Okay, I see why he went with a power boat this time.
@Jay L Gischer:
It’s a rehash of “the polls are skewed” from 2012. Back then the meme was that people were hesitant to admit they were voting against a black man. This time it’s their hesitant to admit they are voting for a racist.
The polls weren’t skewed then.
IMO, as has been said before, Trump bumbled into a perfect storm and thinks it was his brilliance that carried the day. Therefore anything he does is brilliant and right and proper and a winner! But perfect storms, or black swans, don’t come along too often.
He should look at scandals instead. Bill Clinton, and all those associated closely with him, like Gore and Hillary, got splattered with Bill’s many scandals. Bush the younger lacked them for the most part, as did Obama, and now Biden. trump has scandals enough, especially those involving corruption and misrule, to make Bill’s time in office seem boring by comparison.
He should also look at the economy. Bush the elder was a perfectly competent statesman, who had won a war with few casualties and within a reasonable time, and he was sunk by a short recession.
Look at the recession now.
True, it was not Trump’s fault, but he presided over it, and he’s made it worse by 1) mismanaging the pandemic response, and 2) failing to pass another relief bill.
Sure, the voters are “shy.” yeah! that’s the ticket!
@James Joyner: You’re an optimist that a coalition that firmly believes that being called a racist is far more damaging than being systemically racist will approve of any pre-clearance criteria that is heavily focused on the former Confederacy and their political allies.
@Teve:
I like to play with that a little bit.
I’m not a racist, but the capybara is the world’s largest rodent…
I’m not a racist, but polling indicates that Trump is within the margin of error in several battleground states…
That moment when people clench right after “I’m not a racist, but…” is so fun to play with. I’m not a racist (well, I’m less of a racist that most white folks my age anyway), but I am an asshole.
@Kathy: The shy-Trumper theory is actually a variant on the Bradley Effect. I’ve even occasionally heard people use the term “Bradley Effect” when referring to the shy-Trump hypothesis, though the umbrella term used by social scientists is “social desirability bias.” It refers to any time a person lies to a pollster (or to any researcher) due to being embarrassed about their views. Whether this ever happens, leading to significantly inaccurate polls, is still a matter of debate. Count me as a skeptic.
In any case, people definitely talked about the Bradley Effect in 2008, though it didn’t mainly come from conservatives–maybe because they thought it was tantamount to admitting Republicans were racist. I think I remember Jonah Goldberg defining it as a Republican fear of being unfairly accused of being racist, and therefore that that would skew the polls toward Obama–but I don’t think that argument was particularly widespread. I frankly don’t remember Republicans being in quite as much denial over Obama’s impending victory as they would be in 2012. Some were (“John McCain will win the presidential election, Kellyanne Conway, one of the country’s most respected Republican pollsters, tells Newsmax.” — 9/22/2008), but it wasn’t ubiquitous. Remember Karl Rove’s meltdown on Fox in 2012? In 2008 he freely admitted Obama was probably going to win.
While conservative skepticism of polls goes back a long way, I think 2012 was somewhat of a turning point for them, where they truly began to jump down the rabbit hole. And instead of rethinking their position after Obama won reelection, they doubled down on it. That seems like a very odd thing to do, but it’s part of their entire mindset, like when Ben Garrison recycles his “red wave” cartoon he used back in 2018, seemingly unable to grasp the irony that it was proven demonstrably false.
@Kylopod: While I admit that shy trump voters may still be a significant number in this election, I agree with you that it’s super unlikely.
Something you didn’t address that I think bolsters your argument is that if there were a significant number of shy trump voters in this election, while some of them would say they were Biden voters but there would be a significant number claiming to be undecided.
The fact that there are so few undecided voters in this election seems to indicate that there aren’t nearly as many shy Trump voters in this election as 2016, if there even were shy trump voters 4 years ago.
(if you did indeed mention the few numbers of undecided voters, and I missed it, I apologize.)
@Mike in Arlington: That’s an excellent point, and I think it requires more elaboration. I actually have over the years defended a theory that might on the surface sound like a variant of the shy-Trumper hypothesis. For example, in 2016 when there was a poll showing Clinton and Trump neck-and-neck in TX, I noted the large number of undecideds whom I expected to break for Trump, and I openly wondered if that was going to be a problem for Clinton more broadly–that there were a lot of Republican-leaning voters who found Trump personally repugnant, enough to remain undecided in the polls, but whose partisan tendencies would ultimately lead them back to him.
I have noticed, going back to 2016, an attitude that I’ve called “Yes Trump is awful, but….” It consists of the people who engage in Trump apologia but feel compelled to pad it with formulaic criticisms of Trump’s demeanor and personality. It’s one of the reasons why I was skeptical that his low approval ratings throughout his presidency spelled doom for him, especially after seeing him win in 2016 despite low favorability ratings. (Many people don’t distinguish between approval ratings and favorability ratings. Approval ratings are only taken of sitting office-holders. They refer to people’s opinion of the job the person is doing while in office, as opposed to an overall opinion of the person. In practice, the two metrics are closely related and tend to go hand in hand.) You could call this a version of the shy-Trumper theory.
The difference is that I wasn’t saying anyone was necessarily lying to pollsters. To some extent I think there’s an element of self-deception here, people lying to themselves. A lot of people think they aren’t Trump supporters but really are in effect. In 2016 I believe there were a lot of people who thought of themselves as undecided who ultimately broke for Trump in the end, even though that was probably inevitable whether they consciously realized it or not. It’s kind of like the way I was in mid-2008 when I was so pissed at Hillary I told some people that if she was nominated I’d vote for McCain. I doubt I ever would have. I was just throwing a hissy fit. Partisan voters are highly prone to rationalization and self-deception for voting behavior that’s baked in.