FEATURE22 October 2009

Stan Sthanunathan on why quality doesn't matter


Stan Sthanunathan, insight boss at Coca-Cola and co-chair of the Advertising Research Foundation’s Online Research Quality Council, told conference delegates yesterday that ‘quality doesn’t matter’. We talked to him afterwards to find out more.


Research: You told a session at The Market Research Event in Las Vegas yesterday that ‘quality doesn’t matter’ and that survey research is ‘terminally ill’. What did you mean?

I’m exaggerating a little bit to make a point. Quality is an urgent issue but there’s a more important issue to do with what needs to be done going forward, because the industry isn’t keeping pace with the change going on around us. People are too focused on quality, people are too focused on probability and non-probability samples, people are too focused on respondent engagement – this is all about making minor changes to what we are doing right now. Those are all necessary, but they’re not sufficient conditions for the success of the function.

“I’m a little bit concerned about the focus on the nuts and bolts of research”

We are focusing our time on technicalities of research, whereas the expectation from clients is actually in a totally different space. The clients are saying, ‘Inspire me, help me to take some transformational action,’ and we’re busy creating better mousetraps. So that’s why I’m a little bit concerned about the focus on the nuts and bolts of research. We constantly complain that we don’t get a seat at the table. But think about it, if you’re creating a better mousetrap, is the CMO of the company interested in that mousetrap? He would never give you a seat at the table for creating a better mousetrap.

Research: So if these aren’t the big questions, what are?

The most important thing is that in all the work we do, we’re basically going after the tip of the iceberg. There’s a huge amount of insight that’s below the water and we’re not necessarily best-in-class in terms of understanding insight about those things that are below the water line. It’s about understanding the human condition. We’re too focused on understanding consumption behaviour and shopping behaviour. We need to understand the human condition, which you’ll only know by observing, listening, synthesising and deducing. We don’t really do a good enough job.

Research: Do you see discussions about data quality and other research issues as a distraction?

No, it’s not a distraction – those are issues that we need to resolve. But think back to twenty years ago when we moved from face-to-face to telephone sampling – we had the same debates. This is the same movie, different characters, slightly different script. And if we can have that same conversation for twenty years, then we’ll find another excuse to have the same conversation in another ten years from now. It’s an important conversation to have, don’t get me wrong, but we cannot have eternal debates.

If somebody wants to question the representativeness of online panel, I can’t really argue with them. But don’t raise a problem if you don’t have a solution or at least alternatives. The thing I would focus on is, how do I make sure the result I get from online panel is replicable and consistent? Those are the two key factors in my opinion.

Research: If survey research is ‘terminally ill’, what does the future hold?

Again, I’m exaggerating a bit. As it stands now, survey research is going to become less and less important over a period of time, because if you ask consumers a question they’ll give you an answer, and I’m increasingly beginning to wonder if I’m getting truthful answers or not.

“Every panel company says, ‘I have one million, I have five million, I have 55,000 panellists.’ It’s almost like a men’s room conversation”

I can see two directions in which the research industry will go. For example, what will happen if tomorrow Facebook becomes an insight provider to the world? They have 300 million active users every day. Their users share some really personal and intimate details on Facebook, and if they create a technology whereby they do good, solid, meaningful content analysis of what goes on in Facebook by different demographic groups, they can give you much, much better understanding of the human condition than anybody else, ever. You might say there are privacy issues around that, and I agree with you that there are issues, but the other day I got an invitation for a survey from LinkedIn. They wrote the note in such a way that I did not see that as an invasion of my privacy, and I very happily took part in a survey. And LinkedIn has one of the best databases of professionals around the world. So if LinkedIn can get in on it, why not Facebook?

The second thing is, every panel company says, ‘I have one million, I have five million, I have 55,000 panellists.’ It’s almost like a men’s room conversation. Tomorrow, Google wakes up and says, ‘My earnings per share is not going as rapidly as I’d like it to, but I’m sitting on a billion users. Why don’t I get into providing panel service to all the research agencies?’ Then this one million and five million becomes a completely meaningless conversation.


15 years ago

Great article. I had the privilege of hearing Stan in person at the IIR. It was a terrific presentation -- tops in my mind. Thanks for your leadership Stan!

Like Report

15 years ago

Ah, another round of agency bashing...do we not get enough. Plenty also needs to be said of the quality of the Client Side Manager Often it is even they who are unable to get a seat at the table simply failing to identify contributions that insight could make to any transformational work that is being discussed. But tend to get very talkative when criticizing agencies. BTW there is a large soft drinks company whose MR contacts are always very keen to discuss panel size and panel quality - maybe they who need to change the discussion rather than finding their comfort zones in tactical evaluative conversations.

Like Report

15 years ago

This is such an realistic view to how the MR Industry can be impacted by just three entities i.e. Facebook, Google & Linkedin! If these big three decide to get into supplying panel, blog insights, focus groups - it can throw many companies out of business overnight & no one can also compete with the prices that they could offer!! I think 3-4 years from now, this would definitely be the scenerio as Stan has rightly put it...

Like Report

15 years ago

It is not about agency bashing... if you had heard my presentation I certainly did say that clients have equal responsibility. It takes two to tango!!!

Like Report

15 years ago

If you feel threatened by challenges like Stan's, I encourage you to work through the discomfort to find the truth beneath. My angle on Stan's observations is that we are all busy fitting mag wheels to the horse and cart and polishing them like mad. In doing so, we're missing the opportunity to build a Ferrari... or maybe even a Time Machine!

Like Report