Outside the Beltway

Polling Errors in the UK Elections

Steven L. Taylor · Friday, January 1, 2016 · 3 comments

Some may recall that the polling for the House of Commons elections in 2015 did a pretty poor job of predicting the final outcome.  A piece in the Guardian discusses some studies as to why that may have been the case:  Why opinion pollsters failed to predict overall majority for David Cameron.

Analysis undertaken by polling companies, including YouGov and ICM, of what went wrong in May has found that that a relative over-representation of politically engaged young voters produced a forecast that flattered Ed Miliband. Conversely, the over-70s – who broke heavily for the Tories – were under-represented in YouGov’s internet panels.

The findings come before the publication in January of the initial findings of an independent study for the polling industry, led by Prof Patrick Sturgis of Southampton University, to examine why so many failed to predict a majority win for the Conservative party in May.

YouGov research into its election errors – it underestimated Tory support by 3.7 points and overestimated Labour by 2.8 points – identified an excess of politically engaged young respondents. Because of their age, they were disproportionatelyLabour but – because of their interest in politics – they were also more likely than the rest of their age group to turn out and vote.

Getting the sample right is the trick.

Also, the means of contact matters (and is increasingly making the sampling difficult):

Sturgis’s study, in which he led a team of eight expert colleagues, principally fellow academics, looked at several scenarios to understand what went wrong. A late pro-Tory swing; error in the final technical adjustments that pollsters apply, through weighting and filtering schemes; and particular categories of missing voters, who no longer talk to the pollsters at all, were all looked at. There were also awkward questions for the pollsters, including whether the “bunching” of final surveys reflected a lack of nerve on each firm’s part to be out of line with the others.

Sturgis had previously indicated to the Guardian that online polling, which draws samples from large panels of volunteers, might be a flawed method, saying: “Not everyone puts themselves forward to take part, and those who do will not be representative.

“Many households are increasingly reluctant to pick up their landline, and a growing proportion of young people in particular rely exclusively on mobile phones.”

A firmer conclusion of systematic bias in the way people are polled has developed, particularly after the late publication of the one poll to have called the 2015 election right – the face-to-face British Election Study (BES) survey.

The piece is worth a read.