For its annual summer event yesterday, the Social Research Association chose the theme of the Big Society. But the challenge of researching and evaluating the Big Society requires us to first understand what it actually means – and that’s a matter of some debate.

Prime Minister David Cameron has said it’s about a shift “from state power to people power”. His strategy director Steve Hilton has been more candid, calling it “an audacious attempt to fashion a notion of social solidarity from the bricks of centre-right ideas” and an attempt to “wean this country off its apparently unbreakable dependency on the state”.

But not everyone is convinced. Yesterday John Mohan of the Third Sector Research Centre compared it to the snark – the unseen mythical creature of Lewis Carroll’s poem, which always evades its hunter.

Most people seem to understand it to be about encouraging people to take responsibility for their communities rather than relying on the state. It has been a controversial initiative (if initiative is even the word) – one that the government has tried to articulate a number of times without great success, and which has fallen into the trap of being seen cynically as a front for huge public spending cuts.

So it was hard not to wonder yesterday why many of the social researchers in the audience were abbreviating Big Society to ‘BS’ in their notes. Indeed, the question of how wary researchers should be of the government’s Big Society agenda became the key debate of the day.

Anna Coote, head of social policy at the New Economics Foundation warned that agencies need to approach the idea with a healthy scepticism.

Judging by its aims, the so-called Big Society seemed to be “a force for good”, Coote said, but she hastened to point out that “it doesn’t exist”. “It’s simply a story that we are told about, perhaps, the way things should be, or perhaps more to the point, how we’re supposed to imagine things are unfolding.”

After hearing speakers describe efforts to track, evaluate and help implement the Big Society, Camilla Child of the Tavistock Institute said she was troubled by “the extent to which, as social research organisations, we take up that narrative of the Big Society and accept it almost unchallenged.”

Ceridwen Roberts of the University of Oxford said that agencies working for government clients should ensure they are also “talking to other people who may be critical of this ideological onslaught on the welfare state”.

Coote said that any researcher being paid by government “is bound to be influenced” by that relationship, and that agencies that work predominantly for government clients need to be especially careful. “We should always be sceptical about everything we’re asked to do,” she said. “But it’s not always easy – especially if you’re desperate for money.”

NatCen’s Gareth Morrell accepted that social research is “a little behind” on understanding the difference between the policy rhetoric and what’s really going on. “Maybe there’s some more deliberative, qualitative work that we can do to try and get to the bottom of whether people understand what’s happening,” he said. “I think there are a number of things we can probably do better there.”

But the big question – as Ipsos Mori’s Anna Pierce pointed out – is who is going to pay for that.

Pierce said Ipsos Mori has sought to challenge the rhetoric in its work, and that “there is an interesting debate about the gaps between the theory and practice of the Big Society – and whether Big Society is the right title for it”.

She also described how the role of the researcher was broadening to include “building this Big Society rather than just researching it”, through workshop events that served to encourage participation as well as getting a feel for people’s views. Whether or not this is a positive development is another matter.

The NEF’s Anna Coote urged researchers to be aware of how the government has used the narrative of the Big Society as a distraction from the wider impact of the cuts agenda.

“I have a real sense in this room of fiddling while Rome burns,” she said, arguing that researchers must understand the bigger picture of how a transfer of responsibilities away from the state will affect those most in need of support. “There’s going to be a widening of inequalities and huge geographic disparities and very little to fall back on,” said Coote.

And the research that is being conducted on these questions isn’t necessarily being heard. John Mohan, whose work has highlighted how Big Society policies risk reinforcing inequalities, said it was frustrating to see soundbites from think tanks get more attention than finely-tuned survey data, “even when some of the findings are contradictory or patently rubbish”.

Gerry Stoker of the University of Southampton, who chaired the debate, reminded the audience that social research did not need to fit the agenda of the government of the day in order to be relevant to current issues.

He urged researchers to think about what they will say if, in a few year’s time, they are asked what they did during the “extraordinary social experiment” that is currently taking place in the UK.

It will be embarrassing, Stoker warned, if they don’t have a good answer.