Bots for #Brexit


First thing: I think the UK’s vote to leave the European Union is a calamitous mistake. The worst in foreign policy since Suez in 1956 and quite possibly second only to Munich in the last century.

What I want to write about here, though, is the way in which that Leave campaigners (in the broadest sense) leveraged the use of Twitter bots in the campaign. A report now available on Arxiv (here) suggests that bots generated over three times as many pro-Brexit tweets (97,431) than pro-Remain messages (28,075) in a one-week period in June.

(The report also suggests a slightly higher proportion – 15.1% – of pro-Remain tweets were bot-generated than for Leave – 14.7%)

Did it matter? The paper suggests bots have “a small but strategic” impact. In a referendum of huge importance that was lost by a narrow vote that could be very important.

My personal experience was that the online field was much more important in the Scottish referendum, where the “Yes” campaign (in favour of Scotland leaving the UK) were very effective in mobilising online resources for people seeking to “research” the question.

One thing where both referendum campaigns were similar was that the pro-change campaign accused the other side of being “Project Fear” and used online resources to repeatedly reassure people that they need not fear the consequences of a Yes/Leave vote.

Happily, in Scotland, disaster was averted and so the accusation of Project Fear merely lingers. Over the EU it has now become Project I Bloody Well Told You So.

Not fit to hold public office



Embed from Getty Images

John Mason, Scottish National Party MSP for Glasgow Shettleston, has (today) tabled the following motion in the Scottish Parliament:

That the Parliament notes that South Lanarkshire Council has issued guidance concerning the appointment and input of chaplains and religious organisations in schools; understands that some people believe that God created the world in six days, some people believe that God created the world over a longer period of time and some people believe that the world came about without anyone creating it; considers that none of these positions can be proved or disproved by science and all are valid beliefs for people to hold, and further considers that children in Scotland’s schools should be aware of all of these different belief systems.

Lessons from the Scottish referendum


It is just over a month since the people of Scotland made their decision to stay in the United Kingdom.

The final result was – in referendum terms – something close to a landslide, with “No” finishing 10.7% ahead. Perhaps that was not as big a victory as had been hoped for even two months earlier, but it was enough to encourage many to say they had known all along Scotland would indeed, to use David Bowie’s words, “stay with us”.

From the point of view of some commentators and businesses the outcome may have seemed obvious from the start. Splitting the world’s most successful single market, injecting massive currency, regulatory and political uncertainty and pegging your future to the value of the rapidly diminishing North Sea Oil reserves seemed so ridiculous that many thought it unnecessary to speak out.

Yet in the end many did join the debate and they did so not because they thought they were backing a winning proposition in opposing the UK’s break up, but because they were worried they were about to end up on the losing side without having said a word.

Understanding why that was, and why the “Yes” campaign propelled what many regarded as, at best, an eccentric idea to the point where their opponents seemed to be in the midst of a frenzied panic, is what I want to look at here – because I think the Scottish referendum campaign tells us a lot about the contours of public policy debate in the future: and much of it is far from good news.

I have four, related, propositions to make.

Firstly, in the future, all stories, even the made up ones, will be famous for 15 minutes. It is not that long ago that I remember celebrating the way that the Internet had broken down the people’s deference to the professional closed shops that denied them choice and autonomy. The classic case was the medical profession, where doctors were confronted with patients newly armed with self-diagnoses informed not by old wives’ tales, but credible websites.

In the referendum, though, this questioning of existing authority became something much uglier – a lack of respect for the very idea of any authority. In short, any rumour or conspiracy theory could be treated as fact and believed by many.

Since the referendum result was declared this has manifested itself in a belief that the whole poll was rigged – as though Alex Salmond was all along an agent of Unionism.

Before polling day the focus was on oil – with members of the Scottish parliament putting their names on leaflets claiming there were secret reserves of oil off the coast of Ayrshire or endorsing bizarre theories that the Prime Minister was actively engaged in a conspiracy to hush up details of a massive new oil field found to the West of the Shetlands.

These ideas did not just circulate on social media – the BBC felt compelled to treat the latter story as though it had merit and asked people about it.

The whacko world of 9/11 “truthers” and “birtherism” came to Britain and played a real part in deciding the biggest issue facing the country since the end of the Second World War.

Secondly, on the Internet, all ideas are created equally. Lots of voters, but especially “Yes” voters, prided themselves on the “research” they undertook before deciding how to vote. And, truly, many Scots did treat this momentous question with the seriousness it deserved.

But, when asked to research a question, what is the first thing you do today? It is not to go to the library or to ask an acknowledged expert, but to fire up your favourite browser and type a search term into Google.

What comes next is crucial. If you are a school student researching, say, Pythagoras’s Theorem, you’ll get a Wikipedia article and links to some quite good graphics illustrating the sum of squares rule. But type a term that relates to an ongoing political controversy and you are in the lap of the gods – or rather you are standing downwind of who can shout loudest. And in the Scottish case the loudest shouters were very much the Yes campaign.

As just one example, towards the very end of the campaign, Ipsos Mori published the results of a focus group into why people were switching from No to Yes – and sure enough one person was quoted as saying their “online research” had convinced them that “Westminster” had declared the waters off the coast of Fife (in central Scotland) to be English and they were going to vote against that.

Of course, there is no such declaration. What he plainly had seen was a claim by a Yes conspiracist about some secret plan. But, on the Internet, there is nothing much to tell you whether any particular idea is barmy or rational.

Thirdly, if you don’t like these facts, why not make your own? French postmodern theorists resisted the idea of objective facts, claiming that all ideas are merely “narratives” promoted by elites. And Yes campaigners in the Scottish referendum proved to be excellent pupils of Lyotard and Baudrillard by simply creating their own narratives through a small number of very high profile blogs.

These sites – particularly Wings over Scotland, Newsnet Scotland and Bella Caledonia – grew to be huge and well-funded, via crowd-sourcing, operations. Feeding and feeding-off the obvious anger of many Yes campaigners, they proved to be both the exo-skeleton of sprawling Yes campaign and lenses that focused all the issues that angered and motivated Yes voters. In short, you were quite likely to end up on one of these sites if and when you did that “research”.

Fourthly, you do not have to believe in anything as long as you believe in us. The Yes campaign was highly motivated and very big, in terms of its street presence at least. But what did they all believe in? Not very much beyond the idea that Scottish independence was the answer to more or less any given problem.

At the same time that Alex Salmond was arguing that an independent Scotland could surf a global oil boom, the leader of the Scottish Greens was saying an independent Scotland could end its dependence on fossil fuels. Far left parties claimed a Yes vote “guaranteed” thousands of additional civil service jobs, while the governing Nationalists committed to making the private sector, fired by tax cuts, the engine of jobs growth.

Perhaps the No campaign could have done a better job in exposing these obvious contradictions and so forced at least one group of Yes supporters to confront the reality that they were on a hiding to nothing. But, regardless, the ethereal nature of so much of the campaign – conducted through social media – meant there were a limited number of hard targets for No to hit.

The approach of newspapers and broadcasters, desperate to look relevant to younger audiences, to social media did not help. Despite actually very few Scots being regular users of Twitter, it took on a special significance and that only served to emphasise that who could make the most noise online mattered almost as much as the quality of their arguments. What is more, the Yes campaign skilfully tried to turn the whole issue into a horserace, discount rational argument and only rely on emotion and energy.

They failed, of course. Facts carried more weight than they had hoped and the Yes campaign’s inability to give credible answers to some core economic questions – especially about what currency an independent Scotland would use, cost them dear.

All of which leads to me to a final conclusion. Which is one should never elevate tactics over strategy. The No campaign stuck doggedly to its strategy of hitting hard on the core economic questions and that was undoubtedly the right thing to do – after all they won. A better campaign might have had a more flexible response to the four horsemen of the Yes apocalypse discussed above, but a No campaign which abandoned its strategic vision to just hit at the latest conspiracy stupidity, fact-free assertion or contradictory comment from its opponents would have ended up losing the voters’ attention and the public’s vote.

No’s focus on continuing to ask the small number of really difficult questions for Yes was about more that getting their voice heard amongst all the shouting: it was a reminder to the voters that facts and assertions are not the same.

It had been a long, and at times desperate, struggle, but No’s discipline saw them through. That is worth remembering in any future public campaign no matter how far from Scotland.

The real reason why you haven’t been polled in #indyref


English: Graph showing support for political p...
English: Graph showing support for political parties in New Zealand since the 2008 election, according to various political polls. Data is obtained from the Wikipedia page, Opinion polling for the New Zealand general election, 2011 (Photo credit: Wikipedia)

Some people – Yes supporters essentially – are claiming that it is plain that the opinion polls – none of which (so far, at least – I hope I am not tempting fate) have reported a Yes lead – in the Scottish independence referendum are rigged is because they have never been asked.

Well, there is a simple reason for that: polls are small and the electorate is very large.

There are about 4 million electors able to vote in the Scottish independence referendum.

If we assume every elector has an equally random chance of being asked (which is not true for many cases: if you are not on an online panel it just won’t happen), and that each poll asks 1200 electors then the chances of you being asked in any given poll are 1200/4000000 or about 1 in 3,333: a bit better than winning the lottery jackpot I’d admit, but who bets on a 3332/1 chance?

Of course, though, there are multiple polls but to have just a 1 in 100 chance of being asked then 33 polls would have to be taken. To make it more likely than unlikely that you had been polled then 1667 polls would have to be taken.

What Scotland Thinks, at the time of writing, records 80 polls on the referendum question – so the chances of any individual elector being asked are (given all my approximations) about 1 in 42, or in bookies’ odds terms, it’s a 41/1 shot.

If you think a race is fixed because your 41/1 wager never comes home, I’d suggest you weren’t to be trusted in a betting shop.

Update: Should make it clear this is a pretty crude approximation to make a point – opinion poll sample sizes vary and if they are closer to 1000 in sample size then the odds of you being asked go up to about 49/1 (ie., it’s a fair bit less likely).

A further update: My intention on writing this was to demonstrate, in the broad brush terms why an argument based “I have never been polled so therefore the polls are wrong” didn’t hold any water. It seems the article now being touted around as an exact prediction of how likely it was you’d been asked: it’s not. As I say above much (most probably) polling these days is via online panel – if you are not on the panel you cannot be asked to begin with.

(Scottish) opinion polls – a reminder


English: Northern Ireland opinion poll on Prot...
English: Northern Ireland opinion poll on Protestant identity in Northern Ireland. Data from the Northern Ireland Life and Times Survey http://www.ark.ac.uk/nilt/. (Photo credit: Wikipedia)

This is not about which way you should use your vote if you have one – that is here.

Instead it’s reminder of the maths of opinion polling, because I suspect we are going to see great numbers of polls in the next few weeks.

So here are some things to remember:

1. The best an opinion poll can do is tell you what a good opinion poll would show. In other words, opinion polls cannot be thought of as reliable predictors of results. Simply put – if people systematically fail to tell the truth to opinion pollsters then no opinion poll is going to perfectly correct for that (the pollsters try but their work here is merely based on informed guessing). So when pollsters talk about “margins of error” they don’t mean in comparison to a real election result, but to what another – well taken – poll would show.

2. One in twenty opinion polls – no matter how well conducted – will be very wrong. This is the so-called “rogue” poll and it’s incorrectness is not because it has been conducted improperly but because sampling a small subset of a large population is inherently statistically risky.

3. Doubling the sample size does not mean your poll more twice as accurate. In fact it only makes it \sqrt2 more accurate. The important point here is that when you look at small samples – such as Scottish regions – you are looking through the other end of this telescope – so a smaple that contained \frac{1}{5}th of the poll sample would actually have a margin of error that was \sqrt 5 (or about 2.2) times bigger (and that assumes the sampling in that region actually matches the population in that region as opposed to Scotland as a whole – if it doesn’t, and the chances are that it won’t, then you are better off just ignoring the subsample).

4. “Margin of error” is really a measure of how likely other polls will give the similar results. We have already covered this – but here’s a longer explanation. If we say that the margin of error on a poll is plus or minus three per cent, then typically what we mean is that 95% (i.e., 19 out of 20) polls will give results where the figures do not differ by more than three per cent. This also means if you describe a 1 per cent change in a rating as in some way significant then you are very wrong – because actually your poll does not give you enough information to make that claim. To go from a 3% margin of error to a 1% margin requires you to increase the sample size by a factor of 9. To go to a margin of plus or minus 0.5% would require an increase in sample size by a factor of 36.

5. The margin of error actually depends on the score polled. The highest margin of error is at 50% – where for a 1000 sample poll it is:

2 \times \sqrt \frac{0.5 \times 0.5}{1000} = \pm3.4\%

For 40% the margin becomes 2 \times \sqrt \frac{0.6 \times 0.4}{1000} = \pm 3.1\%

(And these figures are for the numbers before the don’t knows are discounted.)

The Scottish referendum: taking a stand


I am not Scottish. I lived there, once, a long time ago but I have no vote in the referendum next month.

I do have a pretty direct personal stake in the outcome though. My partner is Scottish and a yes vote would, to some extent (and I think a greater extent as the years went on) make her a foreigner in what is now her own country. My two children certainly have as much claim to be Scottish through her as they do to be Irish through me. And, of course, the eldest is actually resident in Glasgow, at least while the University is in term time.

But, actually, my personal stake is much bigger than any of that. I fear a yes vote on September 18 will lead to a nasty, and possibly permanent, disfigurement of politics both north and south of the Tweed. And I am drawn to that conclusion by both the character of the campaign for a yes vote and the inevitable changes in political calculus a yes victory would bring – both short- and long-term to the politics of the remainder of the UK. It worries me enough to break my self-denying ordinance about politics here and to, in a way, make my stand. I can do no other.

It is common for those campaigning for a “Yes” vote to say it’s not about Alex Salmond or his Scottish National Party (SNP). But, of course, it very much is. The SNP are the only party of real significance in Scotland campaigning for a “yes” (I really do wonder how many Green voters are pro-independence as opposed to just anti-politics), they have a majority in the Scottish Parliament, they will negotiate the terms of any independence settlement and they will form the first government of an independent Scotland if that happens according to their timetable. They control all the levers on the Scottish side of this nightmare equation – and it will be their Scotland we will get if the vote is yes.

They say that people like me (if I lived in Scotland) should not worry about that. After all, they say, they – like me – are social democrats. Indeed many of their supporters go further and say that people like me – bought and sold by English gold – have no longer any real claim to be on the left, content as I appear to me, to ask the Scots to continue to suffer under Tory rule.

But then, I do not believe them. I am sure there are people in the SNP who genuinely believe themselves to be social democrats or even socialists – but their actions convince me that they are above all nationalists and, in a way that is fundamentally alien to social democrats, are seeking to divide people.

Scotland is no colony, it is not a victim of imperial divide and rule or exploitation and so its nationalism cannot claim to be anything other than a desire to separate, to negate the claim that Die Arbeiter haben kein Vaterland.

And I want no part of that. More than that, my internationalism, my compunction to solidarity, makes me want to do all that I can to stop it happening and to urge my readers here to join me.

The rationalist wing of the SNP would no doubt respond that: no, Scotland is not a colony but my judgement is blinded by tribalism. The reason the Labour Party detests the SNP is not, they might claim, because they are so different, but because they are so alike. Not so. Not so at all.

On 10 March 1998 I had been the Labour Party’s Chief Press and Broadcasting Officer (for the first time) for about a week. That morning I was phoned by Jim Murphy, an MP for not yet a year but already clearly one of the brightest hopes in an exceptionally strong Scottish Parliamentary Labour Party.

Jim had been up all night but he was fizzing with energy. That night the House of Commons had heard the report stage of the National Minimum Wage Bill and the Tories had kept the House up all night debating and voting on amendments. And instead of taking part in the defence of the Bill, the SNP had gone to bed.

These were the early, and heroic, days of Labour government. Everything, or almost everything, the government did was as loaded with symbolism as content – but in the National Minimum Wage we were surely – alongside the Good Friday Agreement and devolution – talking of an epoch-making policy. This stuff really mattered and would do for decades to come. The livelihoods of millions of people were being debated.

The SNP said their vote didn’t matter because Labour had such a large majority. And, mathematically, they were right, but politically they revealed the huge gulf between them and us.

The hundreds of Labour MPs who went through the lobbies were proud and not angry at having stayed up all night to see off the Tories. A minimum wage was the antithesis of the Thatcherite vision for the economy: in the previous 18 years the Tories had actively removed what little protection for wages had existed when they came into office and revelled in the idea of growing low waged employment – boasting that the future of work for millions would be “no so much low-skill as no-skill”.

(The idea was also an example of the influence of the feminised New Left on New Labour – no previous Labour government had legislated for a minimum wage because it had been actively opposed by the big craft unions.)

The SNP just did not see any of that. Because, in the end, they just didn’t care about social progress in the same way as we did. Like Trotskyists considering the politics of social democracy they saw, and see, their principal task as being to destroy the credibility of the reformer, not to secure the reform.

That morning Jim and I agreed a text that later got us both into a bit of trouble:

“Thousands of low paid Scots were on the night shift working to support their families.

“Labour MP’s were at work too – beating off the Tories’ attempts to preserve low pay.

“Where were the Nats?

“Their absence was an insult to those Scots who have campaigned long and hard against low pay.

“I never want to hear another Nat say they stand up for Scottish values. Last night they were not standing up at all – they were down in the gutter with the sweat shop sewer rats.”

The SNP used this for the next year to say Jim had said they were sewer rats. Of course, he hadn’t – but, carried away, we had let emotion over-ride judgement.

The words were a mistake yet, looking back, I can still see why we did what we did. And I have seen nothing to make me think the SNP have changed their approach to social progress – indeed they have spent the last two years telling us, in effect, that social progress is impossible in the UK context, they have rubbished every piece of progress that has been made or issued promises of jam tomorrow under independence using phrasing that indicated they had neither thought about, nor cared about, the issue but were focused entirely on what they saw as its vote winning potential.

(A recent statement on the minimum wage was one example – they said they would consider a “Scottish minimum wage” that would always rise by at least inflation after independence. But today’s minimum wage is meant to be set on the basis that it should grow without increasing unemployment: are the SNP really pledging they would enshrine in law that a minimum wage would grow even if it increased unemployment? Or are they just trying to find a sound bite that makes them look “progressive” while actually pledging to do nothing at all?)

To cap it all, they effectively offer up daily prayers for a Tory victory at the next election: the worse, the better is their approach.

What evidence is there that an SNP-run independent Scotland would be any more progressive? Little or none from the SNP government in Edinburgh. Their flagship policies include a freeze on council tax (which is starving Social Work departments of money and leaving teachers on the dole queues) and paying for free university tuition for the middle class by cutting bursaries for working class entrants. Their flagship economic policy is to cut corporation tax in the hope that Scotland might emulate Ireland as a home for profit recycling (though these days they no longer mention Ireland even if they have kept the policy).

In response I will be told that the SNP need not govern an independent Scotland. As I have already set out that is a false claim (assuming that the SNP manage to keep to their timetable) and in any case Alex Salmond has already stated that independence will mean “Labour no more” – and I fear he is right.

https://twitter.com/MisssyM/status/499672963101106178

An independent Scotland will surely be dominated by populist nationalism while those who campaigned and voted to stay in the UK will be slammed as traitors and quislings and worse.

The online pro-independence campaign is deeply nasty and intolerant – and infected with the usual internet paranoia of the online far-right/far-left alliance (no campaigners are part of a “new world order” conspiracy, are in the pay of a secret higher power, the broadcasters and the pollsters are all knowingly telling lies and so on).

https://twitter.com/ratelmel/status/496676261381820416

 

Of course these days the Nationalists know that a direct attack on “the English” generates revulsion amongst most people in Scotland, so they have found a new way of blowing that dog whistle by talking of “Westminster” and “Westminster elites”: the whole thing reminds me of the way anti-Semites think that saying they are only against “Zionists” gets them off the hook (I am making a comparison of tactics here – not saying the Yes campaign is a haven of anti-Semites.)

Even the SNP’s argument about nuclear weapons is empty – voting Yes won’t lead to any nuclear disarmament – it will merely see the nuclear weapons move (eventually) from one base to another. Indeed voting yes is to consciously opt-out of any effort to secure nuclear disarmament by simply handing your nuclear weapons off to someone else. I cannot see how anyone serious about nuclear disarmament could see this as any sort of progress.

People will be voting yes for many reasons, and the vast majority of them will do so for what they see as progressive reasons. But I think they are wrong, I intend to keep saying so and I hope that more and more people on the left throughout the UK, Europe and wider yet will join me in making that argument.

A further point about Scottish referendum polls


Free Celtic Nationalist Jesus Surfers Sunshine...
Free Celtic Nationalist Jesus Surfers Sunshine Vote Yes Van (Photo credit: GerardFerry)

One (smallish) point I left out of my discussion of opinion polling a few days ago was that the “margin of error” for a 95% confidence interval varies according to the reported score.

This range of possible error is actually highest for parties or opinions that score 50% – which is more or less where the “no” vote in the Scottish referendum is polling now – for a 1000 sample poll with 95% confidence interval the range at 50% is:

\pm 2 \times \sqrt{\frac{0.5 \times 0.5}{1000}} = \pm 3.2\%

Good news for the yes camp? Not really, because apart from the obvious point that it is mathematically equally likely to be an over-estimate as an under-estimate, the corollary is that the error in smaller figures is less. For 30%, roughly where the yes campaign are, the error is:

\pm 2 \times \sqrt{\frac{0.3 \times 0.7}{1000}} = \pm 2.9\%

One thing the Scottish referendum campaign can do without: polling voodoo


English: Picturing 50 realisations of a 95%-co...
English: Picturing 50 realisations of a 95%-confidence interval (Photo credit: Wikipedia)

In the year ahead one of the biggest – probably the biggest – political stories in the UK will be the September referendum on whether Scotland should leave the UK.

I am not going to comment here on what I hope the outcome will be – other than to say I hope and believe there will be a strong ‘no’ vote.

But I am going to take issue with how the campaign is reported and, in particular, the dismal way in which opinion polls are covered.

My ire has been provoked by a claim by columnist in today’s Scotsman that a 1% change in one side’s support between two polls in September and December indicates the race is “tightening”.

My argument is that it indicates nothing of the sort. The two polls are essentially mathematically identical. I realise that “things just the same” does not, as a headline, sell many papers, but it does not make it acceptable to invent new mathematical facts where none exist. The fact that opinion polls today essentially show the same result as opinion polls of two months ago and – in this case – two years ago and twenty years ago – may be a journalistic disappointment, but it is also the reality.

So here is my brief guide to the mathematics of opinion polls. If you want to know more I strongly recommend the classic Statistics without Tears: An Introduction for Non-Mathematicians which, as the subtitle suggests, gives the reader a clear grounding with requiring a lot of maths knowledge.

I will begin with a few ground rules…

Firstly, remember what a poll is based on: not the truth about people’s opinions but what they say their opinions are. If some people systematically lie to pollsters (as, in certain cases, it is known they do because they might be afraid or ashamed to tell the truth) then your poll is flawed from the start. And the best you can say of any poll’s accuracy is that it is as good as the best poll can be.

Secondly, the best we can say about a poll is that, if conducted properly, it has a given degree of accuracy compared to any other poll. So when people talk of a “margin of error” in a poll, what they typically really mean is that 95% of all properly conducted polls will give an answer within that margin of error. (This is both an amplification of the first point but also completely independent of it – if people lie then they will likely lie to all pollsters and so no polls are immune.)

Thirdly, it is a mathematical fact that for even the best conducted polls, we should expect one in twenty to give results outside that “margin of error” – this isn’t because we can expect pollsters to mess it up one time in twenty, but because of the mathematical rules of the universe in which we live. It is an unavoidable feature of opinion polling. And because it is unavoidable we do not know which of the polls is the “rogue” and whether any seeming shift (or non-shift, remember) is because of this “rogue” effect or because of a real change in what people are likely to say to opinion pollsters.

And now a little bit of maths…

Claims about polling accuracy are based on the fact that opinion poll results (surveys of a small part of the population from which we hope to draw conclusions about the whole population) will be distributed about the “real” result (ie the answer we’d get if we asked every single person) in a bell-shaped “normal distribution“. The maths of this “normal distribution” are very well understood and so we can make some well-grounded claims about the potential accuracy of our polls.

These include the fact that, above a basic minimum sample size, the margin of error in our poll (i.e., the error compared to other polls) varies by the inverse of the square root of the sample size. Or to be blunt about it, a poll with 2000 respondents is not twice as precise (i.e., with half the margin of error) as one with 1000, but merely 1.4 times more accurate, while the gap between 2000 and 500 is not a shrinkage in the margin of error by a factor of 4 but of 2 (you can tell straight away that the economics of large scale polling is a bit perverse – if you go from a 1000 to 10000 sample poll, your costs increase by a factor of 10, but the margin or error only shrinks by a factor of 3).

The “one-in-twenty will be rogue” rule comes from the fact that when we talk about the margin of error in a poll what we really mean is that in 95% of all polls the result will be in a band twice the size of the margin of error, centred on the result we have published. This 95% figure is the “confidence interval” (more precisely this is the band of two “standard errors” in each direction about the sample mean).

You may interject now and say “but that doesn’t mean a 1% difference is not real” and you would be right – if you are willing to live with a lower confidence interval or pay for a very much bigger sample. So, to make a 1% figure “real” we might be prepared to live with a margin of error of 0.5% on either side of the reported poll result. We could get that in two ways – shelling out to increase the sample size to roughly 40,000 (compared to  the typical 1,000), which would keep our 95% confidence interval, or accepting that about 60% of polls would give a result that was not within +/- 0.5% of our figure – or, crudely, we were more likely to be wrong than to be right when we claimed the 1% was a “real” shift (we would have a 40% confidence interval).