THE BIG CONversation

Blair’s ‘The Big Conversation’ is underway. A boost for democracy, or a cynical exercise that could prove damaging to research? Marc Brenner reports.

We are in the midst of the biggest consultation exercise ever conducted between a government and its electorate. Launched with much fanfare last November, The Big Conversation (TBC) is designed to allow the population a voice on matters of ‘social justice.’

In an attempt to offer topics for discussion, the Prime Minister published a 77-page document setting out the challenges faced by Britain. The nation has been cordially invited to offer its opinion on matters as diverse as economic stability, health, immigration, education, policing, poverty and provision for old age.

So, Tony Blair, who confessed that he had ‘no reverse gear’ seems to have found the gearstick, and is now happy to engage, learn and listen. Every means of communication has been opened for you to participate. You can post your views directly on the party’s website, you can email a response, send a letter or text message, or simply pick up the phone and call in. There’s even a chance that you will be ‘visited’ by a senior member of the Government who will be happy to sit, listen and nod. In fact, you may only need to switch on your radio; in January Blair hosted a phone-in on LBC as part of the initiative. People of Britain, raise your voice. We’re all policy advisors now.

1 ) A defined base of respondents has been identified. (The population)

2 ) A questionnaire has been drawn up. It is designed to predominantly elicit open-ended responses.

3 ) Response mechanisms have been put in place.

4 ) Responses will be ordered and collated.

5 ) Responses will be analysed.

6 ) The results will offer fuel for change.

These six points will sound pretty familiar to researchers. They represent the traditional template for any research project. Labour may not want TBC to be thought of as research, but they have been quick to ‘borrow’ a number of techniques from the industry they want to remain at such a distance from.

Research has discovered that the research community has profound concerns about TBC, and those concerns are unclouded by political bias. Researchers believe that there are major flaws at the heart of the initiative, and that if the public perceives TBC to be ‘research’, the implications for the industry could be highly damaging.

Is this research?
Why should the research industry worry about TBC? No research agency will be engaged at any point in the process and the government, often criticised for its perceived reliance on pollsters and focus groups, has tried to make clear that TBC bears no connection or similarity to traditional research. However, if we look closely at the structure of the initiative, we will discover an entire raft of connections and similarities.

Fatal flaw 1 No reliable sample
Any research project stands or falls on the reliability of its sample. Bad sample equals bad research. Within moments of the initiative’s launch, MORI’s Bob Worcester was heard on the BBC’s Today programme voicing grave concerns about TBC’s safeguards against abuse. “There’s a real danger that pressure groups will hijack this exercise to promote their causes rather than the concerns of individual citizens.” The Liberal Democrat’s chairman Matthew Taylor agreed, “This exercise is no way representative. It is a lobby group’s dream.” The reliability of the sample was already under heavy attack.

However, it is not only the single issue-based groups that are attempting to exert pressure. Conservative and Liberal Democrat party workers have been corralled to ensure that fierce criticism appears on the website. However, one is hard pushed to find such criticism live on the site. As we went to press, TBC’s website offered either fulsome praise for the Government, or seemingly sanitised criticism. TBC may allow for open conversation, but it does not seem to allow for open publication.

ICM’s Nick Sparrow believes that pressure groups are not the only factor that could skew the sample. “The problem with this entire project is that you have self-selecting respondents. You cannot gauge whether this sample will be truly representative of opinion. The sort of people you are going to find responding to this are, frankly, people with time on their hands. It’s a massively unstructured focus group.” OLR’s Vikki Cooke is also unimpressed with what she believes will be a roll-call of ‘the usual suspects’ who always raise their voice when exercises like this are undertaken.

Bob Worcester flags up another fascinating barrier to balanced response. The Labour Party has revealed that the internet is the easiest and most popular response tool for those wishing to join the debate. Worcester observes, “Fifteen percent of the over-55 age group do not use the internet. The over-55 age group represents over forty percent of the electorate. That’s a mighty proportion of people who are being shut out of this debate. A proportion that, one might imagine, has an awful lot to say.”

It’s not only the internet that seems to be shutting out potential respondents. YouGov has set up a shadow site that allows the public space to comment on TBC. The public seem to have cottoned on to how the easily the sample can be skewed. As one respondent claims, “Those who use the phone lines are greeted with an invitation to join the Labour Party; those who decline are kept on hold until they give up.”

If the sample itself is flawed, how can the Government claim that Britain has spoken? The Government may have been criticised for its past use of focus groups, but these were founded on sound sampling methods. As Gould and Greenberg state in the opening quotes to this article, there really is only one way of doing these things properly. It seems that TBC is a free-for-all, open to abuse from both respondents and the government.

Fatal flaw 2 No methodology
Vikki Cooke cites an example of how a ‘consultation’ process can really work. The only problem with the example is that it was tried and scrapped. “MORI’s People’s Panel was a fantastic idea. It was managed with real rigour and transparency. Bob Worcester had the right idea.”

The People’s Panel experiment was an early initiative that was independently run and subject to the usual research code of practice. Worcester has examined TBC’s questions and methodology closely and he told Research, “I am all for consultation. But to be honest, a first year trainee of mine would not be able to get away with that line of questioning.” Another researcher who did not wish to be identified told us, “There is no structure to the questionnaire. It’s obvious that the open-ended answers it will produce will be twisted to suit whatever the Government wishes to say. There is spin-science at work here.”

Let the research industry, for a moment, give the Government the benefit of the doubt. Let us assume that the sample turns out to be sound, that everyone has had a chance to speak and that the questions provide fair answers. What then? Peter Kellner paints a hellish picture. “How is the Government going to process this information? It’s all free-form and there’s a ton of it. Let’s say that there are about 100 questions that one could comment on. Let’s suppose that there are about 100 responses to each question. Now let’s assume that each response generates 100 words of comment. If that response is coming in on a weekly basis, you are dealing with one million words that need to be read, considered and analysed. Staggering.” Nick Sparrow is also bemused at how the responses will be

dealt with, and believes the project

to be ‘un-analysable.’

In the rush to launch this idea, Labour appears to have forgotten to provide hard details on how the TBC will be audited. There certainly doesn’t seem to be much agreement at Labour HQ. A spokesman originally told us that “policy officials at No.10 would sift through the findings.” An image comes to mind of Tony, Cherie and the little ‘uns being squeezed out the door due to the number of print-outs littering the carpet. Another call to Labour HQ and another explanation. This missive now seems to be the official line, “Every contribution to The Big Conversation will be heard or read, and all contributions will be forwarded to the relevant Labour policy commission to be considered at Labour’s spring conference and our spring National Policy Forum. Ideas generated by The Big Conversation and respondents priorities will be used to improve local areas and help shape Labour’s programme for the next general election.” Hmm, well that clears that up then. We ask, how will the response be analysed and distilled into a digestible form? This is the ‘bread and butter’ work of the research industry. It’s depressing to see such a vital process being done on the hoof.

The Big Confusion
If TBC has a flawed sample and a flawed methodology, isn’t the entire exercise and its results a complete waste of time? Worcester said, “Despite everything, the results will not be worthless. In a democratic society, any input and opinion should be cherished.” Peter Kellner agrees, “A listening party can only be a good thing. The motives are right, the methodology is wrong.” The lack of methodology pains Vikki Cooke too, but she identifies an issue that should give real cause for concern for those in the research industry. “The problem is, if the findings receive a mauling and the word ‘research’ is mentioned by the government or the press, the research industry is bound to be tarred by this exercise.” Worcester, Kellner and Sparrow are also wincing at the thought that the industry might be perceived to have been involved.

If the press or Government use words such as ‘focus groups’, ‘samples’ or ‘poll’ it is likely that, rather tiresomely, research will once again be associated with the art of spin. We have already spotted numerous instances of the word ‘survey’ on TBC’s website. For something that does not purport to be research, The Big Conversation could be a PR time-bomb for the research industry.

February | 2004