A match made in the future?

We live in an increasingly automated digital world. More and more often company agents are being cut out of the purchasing processes to provide seamless services across many consumer sectors, often increasing profitability as a happy by-product.

This same trend is reflected in market research with digital data collection and self-service fieldwork touted as a key movement in the evolution of the trade. These methods are certainly cost-effective and in many cases cut out the middle-man for the client, reducing costs further. This means that more traditional data collection methods, such as CATI, might be seen as less appealing.

I’m not the first person to come out to say there is still a place for CATI at the table, but there are major advantages to CATI that online surveys just can’t offer, and there is a huge opportunity for automation in this area that would elevate the value of CATI surveys significantly.

Quantitative surveys, by their very nature, are structured in a way that allow answers to specific questions to be recorded in a very black and white way. But at Network we record all our CATI interviews in their entirety, which allows us to listen to the whole conversation between the interviewer and participant. In doing so we have discovered that we do, in fact, capture a huge volume of valuable, tangential information that is not often made available for immediate analysis.

Normally in the CATI process the answers are coded in a very straightforward manner, based on options provided by the interviewer. For example a question like: "Did a member of staff suggest the visit or did you request it?" Might get this response: "Right, here’s an important point. I spent nearly four days trying to make an appointment. Yes I requested it. Have you got any room to make a point on how, with your central call service and the changes that are going on at the moment, it was nearly impossible to get hold of anybody?"

But when it comes to recording the answer in the usual way, the research will code that as “2”. It’s clear to see that there is significant depth and value in that full response that is not often made available in the summarised data. But, of course, for an agency to run through each full CATI interview is incredibly time and cost prohibitive – a common barrier for clients.

Can automation help us make sense of this peripheral data? In a word, “no”. Or at least not yet. Bolt-on speech analytics solutions are available, but the quality of these tools varies considerably and they are often only as good/reliable as their algorithms that have been programmed by humans with pre-defined key phrases or expressions.

End users are forced to wade through massive volumes of recordings or transcripts to find the key phrases that matter, resulting in a continuously evolving set of phrases over the lifetime of the project.

One might use a transcription tool to create a text version of the interview, and then mine this with a text analytics tool to categorise non-specific responses. But we have found that, despite bold claims, most transcription tools struggle to capture the diversity of colloquial English accurately; and that the text analytics tools are only really accurate when used with short phrases (e.g. brand or product lists) rather than contextualised conversational responses.

As such, we find ourselves reverting to the only reliable alternative – the human element. We, much like a number of other CATI agencies, train our interviewers to put a virtual ‘marker’ in their data if they think something the respondent said was particularly interesting or different. But, the obvious challenge with this is that our CATI interviewers, while great at their jobs, are not trained researchers and don’t often appreciate the full scope of the client’s business.

As such, we’re currently exploring ways in which this ‘big data’ can be classified and interpreted. Every day the research industry consigns hours of insight-rich phone interviews to the dusty shelves of the server room. Sadly, the investment involved in developing a technological solution, both in terms of hardware/software and in man-hours needed to crank the handle, is still likely to be beyond most research agencies’ (and clients’) pockets.

In spite of this, it’s important that the industry continues to explore the possibilities and endeavour to push the limits of the technology available, because we owe it to our clients to exploit the richness of response that only CATI can give provide.

Virginia Monk is managing director at Network Research