Understandably the popular media will be raising the issue of how accurate the pollsters were, and everyone that wants to cast aspersion on market research will do so. Especially after the Brexit referendum that similarly defied what the pollsters forecast.
The question that crossed my mind was: Is this the end of political polling? What went wrong? What can we learn from this?
It is not the end of polling. People will always want to know what is going to happen before it actually happens. The answer is obviously to ask a lot of people that are going to vote what they will vote for. So don’t despair. You will see political polls in future elections in the USA and UK. For the same reason market research will remain popular.
However, this does not answer the issue of why the pollsters got it so wrong.
After 40 years in market research I have learned that there are several reasons why survey results can be misleading. Things like sample size, honesty of fieldworkers, sample selection, analysis and reporting of the results, respondents giving misleading answers and the whole questionnaire design process.
In business one learns a lot about your competitors and I know that the honourable and respectable companies all work toward minimising dishonesty among fieldworkers. No-one will deny that there are dishonourable fieldworkers, but a lot of cost and time is devoted to identifying these and eliminating them.
The major reason for misleading results from research, in my experience, comes from bad questionnaire design. Not surprisingly this is the case for the USA election results. The pollsters asked the wrong question.
A few weeks before the election one of SKY’s talking heads mentioned that Hillary was the second most unpopular presidential candidate ever, only closely beaten by Trump as the most unpopular. Whilst most people talked about this, few realised that this should have been the pollsters’ real question.
Only one company asked the correct question. We know that they asked the correct question because they published their results well before the election, and they were right. They did not ask respondents “Who are you voting for?” they asked “Who are you voting against?”
I will be the first to admit that I would not have thought to ask the question this way. However, a good analysis of how to ask the question, at the time and given that everybody was talking about the unpopularity of both candidates should have guided one to the right track.
In my 40 years research experience I have come across this problem of the research design not asking the right question. Sometimes the client favours a particular outcome and is scared that the way a question is phrased might lead to an unwanted result. Sometimes the researcher has not thought through the question enough.
Very often the problem is only obvious once the project has been completed and the analyst does not understand the result until it becomes obvious that the question was interpreted by the respondents in a way different from the researcher. This is often the case when the questionnaire was designed for a culture group different from that of the researcher – often because the company uses a design by their overseas headquarters that does not understand the South African cultural groups.
This problem will only get worse as companies are trying to save money by doing cheap research using cheap ways of getting respondents and using cheap ways to compile questionnaires. It is especially dangerous for companies to rely on DIY research. This is especially true when the research is done by their advertising agency when it has a stake in the outcome.
