How good was my US elections model?

2020 projection

The honest answer, I think – see the picture – is “not very”. But I offer various excuses below.

As a reminder the grays indicate “too close to call” – which means either candidate had a 40 – 60% chance of winning, the lightest blues or reds (pink) is 60 – 75% chance of winning, the mid-shade is 75 – 90% and the deepest shade is 90% and over.

To generate the above picture I used Biden 51.0% and Trump 47.3% – the count is still happening in a few places and as a result Biden’s vote share might climb by perhaps another 0.1% but that won’t make a significant difference.

There is also a bit of a cheat here because the model was designed to generate a projection from an opinion poll rating and not an actual result – so it injects uncertainty based on the idea of a standard error in even the best polls (as always, I recommend Statistics Without Tears for anyone who wants to get to grips with some of the core concepts here). Without that additional fuzziness I suspect I’d be even further out.

But despite all this I still think the basic idea works – the polls were generally pretty poor in the election and that meant a lot of projections I posted were out by a lot – but that isn’t the same as saying the model idea – that we can use a national poll to make estimates about the likely outcome in the election state-by-state is flawed.

One thing that was wrong was the (small – no more than 0.5%) adjustments in the model for the trend in state voting. I already knew it was wrong in Georgia – since 2016 the trend there has been strongly pro-Democratic and the model didn’t take that into account. But it also assumed that the pro-Trump trend seen in the mid-west (and a couple of other places) in 2016 would continue – i.e. those states would become even more “Trumpy”. If instead we assume a reversion to the mean and reverse a few signs on the trends, and nothing else, you get this for the projected outcome:

There is no movement in Arizona or North Carolina as I marked them as trending Democrat already, while Florida was trending Republican and also remains unchanged.

This looks a little more like what we actually saw (in fact the national projection is about a 51% chance of a Biden win) – though I admit it’s still not great.

In my defence, I knocked this up in not many lines of R in an evening (with a bit of extra time to smooth out some bumps later). If national polls – generally – were less awful than – generally – state polls proved to be, then this approach still might be of use.

Advertisement
%d bloggers like this: