Making qual count

It was billed as an opportunity for practitioners and commissioners of social research to discuss the perceived value of qualitative research in a policy setting. It was also the occasion for NatCen to present the updated second edition of its handbook, published by SAGE: Qualitative Research Practice: A guide for social science students and researchers.

The panel of experts responding to the issues raised were:

• Professor Martyn Hammersley(MH), Open University
• Carol McNaughton Nicholls (CMN), NatCen Social Research
• Teresa Williams (TW), Nuffield Foundation
• Laura Hood (LH), The Conversation

Here is a selection of the key points discussed.

Do we still encounter issues with what counts as ‘research evidence’ among users and commissioners of social research?

TW: Yes, but it’s important to remember that it’s not homogenous.The best research users are clear on what they want to achieve and are not wedded to one approach. They’re curious about the context of the research and confident in using non-research data to understand where the research fits in. It’s easier to do that if you’re a trained researcher. But most important is for them to be clear on the questions that are to be answered, and to understand the wide range of methods that can address them.

Are consultations/other methods of gathering stakeholder feedback authentic?

MH: There’s a lot of variability. Some of that is to do with what the findings are for. Underneath it all is the problem of trust in numbers – it’s easy to slip into the thinking that numbers say what you think they say.

CMN: Different approaches result in different outcomes. It’s important to understand the limitations and benefits of the approaches you adopt. There’s a responsibility to educate people on how to interpret the data: not just the users, but those creating the consultations.

Is qual research a softer science? Or is it equal in status to other approaches?

MH: What are the dimensions you’re focusing on if that’s how you describe it? Quantitative data is seen as ‘hard’, but as soon as you dig a little deeper into how the numbers were produced, you find some squidgy bits. Many people see fundamental differences but I think they have quite a lot in common. There’s measurement in qual research – trying to see how things vary in degrees, as well as seeing who and what falls into different categories. I’m not sure we’ll ever get over the divide, but we can have productive discussions around what is shared between the two approaches.

TW: The analytic method is common to both. The reason the question keeps being asked is that it’s supposedly harder to be transparent with qual. But it’s not as easy as you think to be transparent with quant.

With quant research, the client can ask to look at the numbers themselves, but there’s nothing to show with qual research. Does that mean it’s more dependent on the researcher?

MH: Rather than worrying about qual in that context, I’d worry about quant. The work that’s gone into producing it is very complex, so you have to make a judgement about how reliable the data is. Figures are no more reliable than quotes.

CMN: A transcript is like a completed questionnaire – you can analyse it and build up a picture of that respondent, but not of the whole research. One transcript is meaningless just like one completed questionnaire is meaningless.