OPINION30 August 2012

Insult or injury?


Discussions about the wording of the referendum question on Scottish independence have been branded a waste of time and an insult to the public’s intelligence. No offence, says Ruth Stevenson, but the question is too important to get wrong.

I’d been all concerned with getting the question right because everything I’ve learned in my education and career tells me that how you word a question will affect the answer that people give. I’ve gone on ad nauseum before about what the Market Research Society expects us to do. We have to write questionnaires that are unambiguous and unbiased in order to uphold high ethical standards within the industry. It makes sense to me to extend this to any situation where the answer to a question is going to be important – and that includes the Scottish referendum on independence. But as it turns out, not everyone agrees with me.

Soon after the blog appeared I was invited to contribute to the Scottish Affairs Select Committee inquiry about the Scottish referendum question. On 8 May, the Select Committee’s final report was published. “Based on the evidence,” they said, “we have no choice but to conclude that the question… is biased. Experts told us that this was a leading question, biased towards a ‘yes’ answer.” So the question is to be re-written.

Yet on the day of the report, the BBC Scotland radio programme Call Kaye debated the outcomes with listeners, several of whom phoned in with the opinion that discussing the referendum question wording was not only a waste of time but was also patronising and insulting.

Here’s a selection of what was said:

“Frankly I think it is insulting to the intelligence of the people of Scotland that they can’t actually go into a polling booth and say yes I agree with independence or no I don’t.”

“Do you think we are so thick that we can’t work our way round what this is all about?”

“This committee must think the people of Scotland are stupid and unable to make up their own minds.”

It hurts me, it really does, because I try my best to do right by respondents and conduct research that is ethical. It would certainly be at odds with the MRS Code of Conduct to undertake research that is patronising and insulting.

And I can see the point they are making. From the voter perspective, many people have already decided whether they want independence or not and the wording of the question isn’t going to make much difference to them.

Where it does make a difference is in the grey area of people who are still making up their minds. Most people probably don’t realise the effort that goes into writing an unbiased questionnaire, or the years of training and layers of quality checks that happen before these things fall through their letterboxes or appear in their inboxes. I hate to shatter the illusion, but writing a questionnaire is harder than it looks and there is a tonne of academic literature out there to back up my assertion that question wording can lead people towards one answer or another. Saying that, it only takes an expert five minutes to write a good question, and if someone had written a good referendum question in the first place there wouldn’t need to be all of this ‘time wasting’… but I digress.

I can see that bickering about the question feels irrelevant to decisive Scottish residents who know how they are going to vote, but the importance of the wording of the question extends beyond the act of voting and that will make a difference to everyone. Becoming independent is a massive decision for Scotland with far-reaching social and economic consequences, and the decision will be made based on the data collected during the referendum. At that point all of our votes will be reduced to a single figure – the proportion of people who voted in favour of independence. And that figure will be subject to extensive scrutiny from all directions, which means people will look back at the question and pull apart what the answer really means.

I’m all about making evidence-based decisions, and therefore I’m certain that any decision that is made needs to be based on the most definitive and conclusive data possible. This decision could change our lives. I truly believe it is best for both sides to try to smooth out as many issues as possible at this point so that voters and policy-makers alike can believe the process has been fair, that Scotland has spoken, and that we should all get on board with whatever decision is made.

And if that is patronising and insulting, so be it.

1 Comment

12 years ago

"Respondents in opinionated, overstating own comprehension shocker!" ;c) A very interesting subject, this, and a great way to show the importance of good researchers in drawing up what should be just a single question! But you're not alone; it's clear that the politicians do have a grasp on how important the wording will be, with the (Westminster) government keen to have a relatively black and white case of "independent or not", and Alex Salmond seemingly more in favour of something that perhaps blurs the distinction between 'in' and 'out'. Both have a lot to lose; we must hope that they settle on something that is sensible and that makes sense to voters.

Like Report