It’s Not Just Nate Silver
Brendan Nyhan points out a number of political science forecasting models that see Obama winning this election:
The most well-known quantitative analyst of politics is Nate Silver, whoseFiveThirtyEight blog now appears on the New York Times website. Even though his black box statistical models have not been publicly disclosed or scientifically validated, Silver’s analyses have acquired a talismanic quality among political junkies, particularly with nervous liberals who are reassured by his forecast of a likely Obama victory (current estimate: 72.9%).
Unfortunately, Silver has become the target of a vitriolic backlash from innumerate pundits whose market dominance is under threat as well as ill-informed conservative commentators who think Silver is somehow skewing the polls for partisan reasons.
(…)
Ultimately, however, the debate over both Silver himself and the specifics of his model misses the point. The best available evidence from both statistical forecasting models and betting markets suggests that Obama remains the favorite in the election. These charts below, for instance, present the estimated probabilities of an Obama victory and current Electoral College forecasts (where available) from the political scientists Jay DeSart and Tom Holbrook, Stanford’s Simon Jackman, Emory’s Drew Linzer, Silver, Princeton’s Sam Wang, the British sports book Betfair, and the Intrade futures market (here and here):
Here’s the chart showing the probability estimates from these analysts:
As Nyhan points out, Silver is not an outlier and, if he turns out to be wrong, which is possible, then a lot of other people will have been wrong too.
Yes, but are any of them really, truly burly? Slender, effeminate statistics are not to be trusted.
Nate’s model has actually been fairly conservative compared to the others. Sam Wang, who was a hair more accurate in terms of the electoral college in 2008 has Obama winning 303 t o 235. Drew Linzer has Obama winning 332 EC.
The data simply doesn’t point to a horse race that pundits like Joe Scarborough and Chuck Todd are blabbering about on a daily basis.
Also RAND, who have used a completely different methodology. They’ve been checking with the same 3500 individuals since early July. They show popular vote for Obama 50 to 45. (They do have a large margin of error.)
https://mmicdata.rand.org/alp/index.php?page=election
First, it was those dastardly fact checkers. Now, it’s the poll pundit, and all because he’s “effeminate”.
Conservatives will pick on anyone.
Plus you have the poll aggregation websites. I think Nate Silver is the gold standard, but Pollster, Talking Points Memo, and Real Clear Politics also have very similar numbers across the board.
I also think Nate Cohn is providing a lot of interesting demographic commentary over at TNR Electionate’s blog.
The one thing I would say about Silver is that while he maybe hasn’t put out the exact recipe he uses for his numbers, he has been quite clear about the ingredients. And he has run the model almost every day since he debuted it in June once the nomination process was over. I have heard no evidence of him making after the fact adjustments to his model, and at least a couple of times during the last several months he has pointed out when his gut instinct of the state of the race has differed from the model’s output.
Speaking of innumeracy. Romney could win and Silver could still be very much right. His model says there is a 72.9% chance that Obama wins. That means that according to Silver’s own model, 27.1% of his simulations result in a Romney win.
Also Nate and I share a lawyer, so he must be right.
@MM:
That means that according to Silver’s own model, 27.1% of his simulations result in a Romney win.
Romney just went from zero chance to 27%! He’s surging!
Models that are right are right until they are wrong. Models that are wrong are wrong until they are right. Experience says to take more seriously the insight of knowledgeable, experienced and unbiased observers than the hocus pocus of T-Tests and R-Squared. At this point, such observers say its a toss-up, or if they are strong believers in momentum, they pick Romney.
@Let’s Be Free:
Experience says to take more seriously the insight of knowledgeable, experienced and unbiased observers than the hocus pocus of T-Tests and R-Squared.
Math = magic. Got it.
@michael reynolds: I prefer my statisticians to be at least 6’6″ and all man. Or, better, 5’10” and all woman. One way or the other, though, I want them to be exemplars of their gender. Because, after all, that’s what’s important in a statistician.
@MM: That’s mathematically true but unsaleable to the general public. “Well, I said there was a 21% chance of Romney winning” will not be interpreted by most as saying there’s a pretty good chance of Romney winning.
@michael reynolds: In light of the controversy about Silver’s manliless, you might want to rephrase that. Unless it’s a female lawyer, then it’s all good.
@Let’s Be Free: There’s also GIGO. At some point, the public polls will fail spectacularly again, as in Dewey vs. Truman, because they become self-selected samples of people who want to talk to pollsters.
@Let’s Be Free:
So math is unreliable, but “momemtum” actually exists.
Got it.
@James Joyner:
This was a punchline on SNL’s Weekend Update two weeks back. Responding to that Gallup poll that had Romney up by 7, Seth Meyer said “Yes, Romney leads by seven among those who still have a land line and are willing to answer a call from a blocked number.”
@James Joyner:
Hah!
@James Joyner: Why is this so hard for most people to grasp?
I mean lots of people bet on games, etc. Presumably they understand odds at least in a rudimentary sense.
If I roll a 4-sided die, the chance of rolling a 1 is 1/4. The chance of rolling anything else is 3/4. If you have to bet, the smart money is on 2-4, but if a 1 comes up it isn’t because the above analysis is wrong.
@Rick Almeida:
So math is unreliable, but “momemtum” actually exists.
Never met a statistician or model builder who thought statistics was the same as math. Any statistical model is one person’s way of manipulating the data. There is frequently highly relevant, non-quantifiable information which is every bit as important, if not more so, than the data which can be parameterized and massaged in a statistical model. Remember that approximately zero percent of statisticians and zero percent of econometricians (including Krugman who considers himself both) predicted the financial meltdown. Those who predicted the meltdown (which includes myself) looked at a series of circumstances and events that had conflated to create a unique situation, and who understood the consequences. The world frequently works in ways that are not predicted by then extant models, though the modelers will be sure to back into a new model after the fact that precisely predicts the unpredicted event.
@Let’s Be Free:
Never met a statistician or model builder who thought statistics was the same as math
You ever meet any who thought statistics was a type of mathematics?
There is frequently highly relevant, non-quantifiable information which is every bit as important, if not more so, than the data which can be parameterized and massaged in a statistical model.
Sure, Friedman’s cab drivers. When did you meet your first cab driver? What secrets did he impart?
@Let’s Be Free:
Please explain how, say, a basic linear model like OLS is “one person’s way of manipulating the data”?
Otherwise, your vapid posts are complete fabrications.
@James Joyner:
It’s an important teaching moment. An 80/20 split is not a win by 80. It is a roulette wheel with 80 black and 20 red. Place your bet and spin.
BTW, this means that the ONLY predictions that can be disproved by events are 100% and 0%.
If you say Obama has 99% chance of winning, and he loses, you might have been right. You can never know.
See also.
@john personna: Agreed, this whole ordeal about Nate SIlver really proves that people have no clue about probabilities.
Even conservative leaning meta-pollster’s formula’s continue to give Obama a commanding lead in the EC. For example take a site @Jan pointed to on a thread over the weekend: http://www.electionprojection.com. While the proprietor has been predicting a Romney will win (and that, at any moment, state level polls will start pointing to that), his own algorithm continues to show Obama with a commanding lead in the EC (290 votes as of earlier this morning).
@Let’s Be Free:
And you didn’t reap a fortune by playing the stock market? Or at least audition to be Jim Cramer’s replacement?
Then you are almost certainly spending too much time playing Dungeons and Dragons.
The hard way.
@Let’s Be Free:
Oh? http://economistsview.typepad.com/economistsview/2006/12/paul_krugman_ec.html
How serious a slump is the bond market predicting? Pretty serious. Right now, statistical models … give roughly even odds that we’re about to experience a formal recession. And since even a slowdown that doesn’t formally qualify as a recession can lead to a sharp rise in unemployment, the odds are very good — maybe 2 to 1 — that 2007 will be a very tough year.
Here’s an article written in 2007, dated 2008. Granted he was a little premature, the meltdown didn’t happen till September and he was predicting March, but it sure looks like he got something close to right: http://select.nytimes.com/2007/03/02/opinion/02krugman.html?hp
I think Krugman saw it coming. So did many other economists, for the record, from both conservative and liberal schools of economy. This was really only a surprise to TV stock pickers.
@James Joyner: “@Let’s Be Free: There’s also GIGO. At some point, the public polls will fail spectacularly again, as in Dewey vs. Truman, because they become self-selected samples of people who want to talk to pollsters. ”
To be more accurate, in LBF’s case, there’s also word salad, because that’s all that that post was.