Chances Are: Shy Tories

“After the inaccuracies of the General Election polls, are there better ways to find those shy Tories?”

The 2015 UK General Election spectacularly demonstrated the dangers of using people’s introspective self-report to predict voting behaviour. The collective failure of every pollster to forecast the election duly precipitated yet another ‘independent’ inquiry, this time by the British Polling Council (BPC). Presumably the BPC membership is now staying true to form by over-estimating the chances of the inquiry’s success.

Indeed, pollsters are already discussing solutions involving voter sampling and survey wording. We believe that a much more radical solution will be required. The main challenge facing the BPC is shown in the above graphic. People are simply embarrassed to admit that they are going to vote Tory. This ‘surprise finding’ has been true, to varying degrees, for the last 23 years and six elections.

Even the 2015 exit poll of 22,000 people under-estimated the Conservatives by 15 seats. Since exit polls can’t, by definition, suffer from sampling and wording problems, fiddling with these can’t be the solution. That the exit poll moved significantly in the correct direction merely shows that people find it easier to lie to a pollster about something they might do than something they just did.

To understand people’s preferences with minimal distortion requires techniques that can remedy the sources of that distortion. An example method for circumventing embarrassment is the ‘unmatched count technique’. Essentially this involves asking people a question in a way that anonymises their individual responses but still allows you to recover the agreement rate across the aggregate population (see here for details).

In order to illustrate this approach we’ve chosen another taboo blue behaviour–what proportion of the population has used online pornography in the past six months? First, when asked directly in a confidential survey, the incidence rate was 21%. Then using the unmatched count technique on a parallel sample we obtained a rate of 31%, which is corroborated by internet traffic data. This x1.32 ’embarrassment factor’ is larger than the x1.11 Shy Tory effect. In fact, voting Conservative turns out to be about as embarrassing as voting in Britain’s Got Talent (see here).