This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here

OPINION10 November 2016

Let me ask you a question

Behavioural economics North America Opinion Trends

Polling, and market research in general, needs to stop playing around the fringes and make some real changes, says Nadim Sadek, and this could mean being more like Trump and less like Clinton. 

Do you enjoy sex?

Find that question a bit intrusive? Recoil a bit? That’s because you were asked a direct question, and that alerted your System 2 cognitive system to take control of your response, which often involves shielding your true feelings, until you know you’re safe.

Will you vote for Trump?

I’m not saying it’s engendering quite the same thoughts as the first question…but it’s still uncomfortable and blunt. Who am I to ask? What does it expose you to if you say yes? It’s obvious to you that the media largely think he’s a buffoon at best so what benefit is there to you in the transaction where you reveal to me that, yes, he’s your private favourite?

This is the problem with polls. They expose people. They rely on System 2 cognitive processing. And there’s no gratification nor benefit for participating, so motivations are incredibly low to reveal the truth.

Pollsters have invented a Frankenstein sophistication to counteract that. They up-weight responses and account for tendencies – underdogs are under-represented, so lift their scores a bit; favourites self-perpetuate, so depress them a bit; Hispanics this, Blacks that, working class another; the artistic, the privileged, those in the sun, those voting in the evening – everything gets an algorithm.

And all of this messing about with numbers and prejudices and subjectivity, masks the central truth of polling: they are measuring the wrong things, and measuring them the wrong way.

Before I describe a better way, let me say why they keep doing it over and over. When you create norms, you begin to be fascinated by them. They’re lines in the sand, and under pressure, let’s say sand becomes granite, un-erasable and forever the baseline.

So you have to do the same things over and over to make stuff ‘comparable’, ‘reliable’ and ‘significant’. A bank of data becomes an incontrovertible compass to the true heading.

Humans are brilliant. We can walk in straight lines, and we can go round in circles. We do so better than any other known creature because at this point in our evolution, our brains can be rational, linear and logical (System 2 ), or we can be reflexive, instinctual, and simply ‘feeling’ (System 1 ). We can read the lines. And we can read between the lines.

Hillary Clinton spoke the lines. Donald Trump tweet-spoke between them. She promised to fine-tune and fettle an engine she claimed was already purring. He said we need to change the engine or we’re going nowhere. Their styles were their messages. Clinton was the assumptive, privileged type. Trump was the gauche insurgent, promising to reset the game.

The USA just had its first System 1 election. The words didn’t matter. And that’s why President-elect Trump’s absence of manifesto detail – other than the promise to be Great Again – was perfectly fine. More people were sick of the status quo than wanting to preserve it. And really, that was the only debate that mattered.

None of the polls in which huge investments are repeatedly made measure System 1. They all rely on rational, linear collection of responses. The answers they get are cognitive, numeric or otherwise rational. It suits the polling industry not to change this. Why? Because if you change it, you lose the norms. And that opens the playing field to innovative measurement, which, commercially, means opening an artery and bleeding your revenues into the great plains, to be lost without trace, except for memories of an ugly expiry.

It does not have to be like this. It is perfectly possible to have better measures, measured better. Instead of asking people direct questions that trigger their System 2 cautious, circumspect and, dare I say, sometimes dissimulative frameworks, we can actually ask them about things that underpin all relationships – with politicians or brands or even each other. We can get to their System 1 heartfelt responses.

At TX, we have invested years in taking the science, from the Theory of Social Exchange of the 1950s to increasingly sophisticated psychology and neuroscience, to identify 16 universal drivers of relationships. They apply in all categories, to all things. They are better measures. Then we embraced the mass availability of brilliant computer science and mathematics that allows subtle, meaningful scoring to be clearly analysed. And twinned it with a measuring system that bypasses System 2 and gets straight to what we really feel in System 1.

Technically, we measure the 16 drivers through implicit response timing. We get people to reveal, without embarrassment, censure or insecurity, what they really feel and plan to do. About anything and everything. Including candidates for the American Presidency. Does it work? Well, I wrote an article in July, predicting Donald Trump’s victory. We scored him as winning on 15 of the 16 drivers.

Polling, and market research in general, is a largely moribund industry, fiddling around the edges, while the world it seeks to measure and explain moves fast all around it. It’s very Hillary, if I may say so.

How many more elections, with their attendant poll-failures and repeated post-mortems, do we have to go through, before we say enough is enough? Uncomfortably, perhaps, we have to be more Donald.

Nadim Sadek is CEO of TransgressiveX

9 Comments

3 years ago

Completely agree with almost all of the above and respect to you for highlighting how misguided and unscientific most market research is. However, I have one question, given that you have to think (that is cognitively process) to be able to any question you are posed, isn't it inevitable that whatever or however you are asked, System 2 will always have be engaged to answer it?

Like Report

3 years ago  |  1 like

Hi Tim. There's a fine line between System 1 and 2. The way to stay on the System 1 side of things, as you probably know, is to have such brief and intuitively accessible 'questions' (one-worders, by preference) that the response is reflexive, not reflective. That's how we do it, as well as gamifying our surveys, so people remain alert and interactive. Thanks for asking! Nadim

Like Report

3 years ago  |  1 like

Great article, definitely some home truths in here for the industry in general. I think the challenge for the industry is that most folks don't want to engage in the detail and are happy to just stop at, "Well, polling is clearly broken. RIP!". A charm offensive is desperately needed to avoid the moribund industry situation that you mention. There are some crunchy questions - how to implement behavioral economics approaches is definitely one, but there are also others - how do you get cost efficient random probability samples, fix low response rates etc. However, the really big one is how do you wrap this up into something that excites folks (statistics lesson anyone?). Well done to you for taking the lead in this area!

Like Report

3 years ago  |  1 like

I think the most effective charm offensive is to show much greater success in marketing investments, for our clients - which is what we are working hard to demonstrate. Actions not words! :) I agree that there are infrastructural challenges to deal with. We're building a better place, but it'll take us more than a day...

Like Report

3 years ago

Hi Doug - I replied previously, but it seemed not to post...The greatest charm offensive we could run is to prove the worth of our guidance by achieving better investment success for their brands. There are still infrastructural challenges, yes, but the place to start is with a clear plan that brings worthwhile change, and that's where we are. We're building a better place, and it might take longer than a day! :)

Like Report

3 years ago

Hey Doug. I think the greatest charm offensive is proving better success with investment - of resources, time, creativity and all the rest we put into the marketing industry. There are also the infrastructural knots you describe, to deal with. We've started with the foundations: better measures, measured better. We've now built better analyses too. And these are allowing us to give forensic guidance as to what to do next with clients' resources. I agree it's an endless task of constant improvement. We're trying to build Rome as quick as we can! Thanks for your encouragement. N

Like Report

3 years ago

Hi Nadim! Yes, I can see where you are coming from, and I haven't seen examples of your 'questions', but I think that asking ANY question is by definition going to activate System 2 to some degree, and therefore make the response subject to any or all of the kind of conscious biases you mention in your article. However, it sounds like by gamifying your surveys you're probably trying to pick up System 1 responses as a scientist might do by designing a smart experiment. Wouldn't it be great if you could access System 1 directly perhaps by monitoring neurological and physiological reactions to actual experiences in real time?

Like Report

3 years ago

There's always a balance to be struck between integrity and pragmatism. Asking people to respond to, say, a new Coke can with wires attached to their heads during an encephalogram would no doubt give us interesting 'scientific' readings, but my view is that they'd be overly pre-occupied by the weirdness of experience, and that would create an unjustifiable bias. Subtly-developed and administered IRT acts as a pretty good proxy for pure System 1, and our data are showing that we really are 'getting under the skin' of brand-people relationships.

Like Report

3 years ago  |  1 like

The trick is to try to make the 'question' as close to being a stimulus that elicits an immediate response as possible. Only encephalograms would allow us to get beyond a semantic system, and then one has to balance practicality and cost against an ideal measurement. People also feel pretty weird when you wire them up and ask them about something as 'trivial' as a slightly changed can of Coke!

Like Report