RSS
The electoral polls were wrong, as every election watcher in the UK knows. But sometimes it takes an American to tell us just how wrong. Roger Pielke Jr dissects the ‘science’ of polling.
The Guardian
May 9, 2015
by Roger Pielke, Jr.
In 1948, the American writer E.B. Write opined that “the so-called science of poll-taking is not a science at all but a mere necromancy.” While political scientists might disagree, they probably would keep their objections quite right about now. It is well known that expert predictions of the 2015 General Election were off target. But how bad were they and what might they mean for how we think about “data journalism”?
For almost 20 years I’ve studied and evaluated predictions, on subjects as varied as global sea level rise, hurricane damage, the English Premier League table and the quadrennial World Cup. The graph below shows the performance of the forecasts for the two biggest parties, the Tories and Labour, which were the only parties which had a realistic chance of forming a government. The x-axis shows the total number of seats that the forecast missed the outcome by, as compared to a simple baseline expectation based on the March, 2015 composition of Parliament. The RED bars indicate a performance worse than the naive baseline, and BLACK bars indicate an improvement, and no, there are no black bars on this graph. (You can see the details of the forecasts and the evaluation methodology at this post.)
It is difficult to describe this graph as anything other than mass carnage for the forecasters. The predictions were off, and not by a small amount. Nate Silver, known for his accurate predictions of the past two US presidential elections and his website FiveThirtyEight, toured the UK before the election. During his visit he opined to the British public, “What we know is that it’s highly likely you won’t have a majority.” Well, no.
We can expand the evaluation to include predictions for the Liberal Democrats, UKIP and Scottish National Party. (Note: Only 8 of the original 13 forecasts included LD, UKIP and SNP; a graph of these results, as well as one for just the Conservatives, Labour and the Liberal Democrats can be found here.) With the SNP revolution occurring in Scotland, we would expect that this would improve the forecasts, since the naive baseline had only 6 SNP members in Parliament. (UKIP turns out to be mathematically irrelevant in this exercise.) Even so, adding in the other three parties only raises four forecasters above the naive baseline. It is worth noting that the worst performing forecast method overall had the very best prediction for the number of SNP seats. Read more …
