Mediawatch
The cheapest way to measure ad effectiveness
In a letter to the Daily Telegraph, former Tory MP Edwina Currie suggests that the most cost-efficient way of gauging the effectiveness of public service advertising might be to stop doing any.
“Exhortations to stop smoking, seek help for cocaine sniffing, enter further education, avoid drink-driving, join the Armed Forces, and take a chlamydia test are doubtless worthy,” she writes. “But seeing them all at once, any intelligent viewer is bound to wonder whether the Government is trying to keep the advertising industry in business all by itself.”
Her suggestion is to scrap the COI’s £540m budget and see what difference it makes. And with post-election spending cuts looming, that prospect seems less unrealistic than it might once have done.
The difference between 'gay' and 'homosexual'
Today, a lesson in the importance of questionnaire wording. According to a CBS/New York Times poll, 70% of Americans support ‘gay men and lesbians’ serving in the military. But when it comes to ‘homosexuals’, it’s a different matter – only 59% are in favour of them serving.
The Obama team will no doubt be scratching their heads to see if they can get an 11-point poll lead on any other issues just by re-arranging the words.
Kate Harding at Salon.com says the poll provides hard evidence in support of activists who insist that word choice is important, and who are “widely dismissed as free speech-hating PC whiners who need to get a life”.
An important point
The Tories are in trouble today over an errant decimal point.
In a document released yesterday with the title Labour’s Two Nations, they claimed (three times) that in the most deprived areas, 54% of girls aged 15-17 get pregnant, compared to 19% in the least deprived areas. Labour were quick to point out that in reality it’s 5.4% and 1.9%. The Tories have acknowledged the error, and said a decimal point got lost somewhere.
The Tories were quick to correct the mistake, but Labour has leapt upon it as evidence of how out of touch they are. Everyone knows how easy it is to let a decimal point slip, but this was a major policy document. To believe that pregnancy rates for under-18s range from one in five at best, to more than half, does suggest a rather loose grip on reality.
This time they’ve corrected the mistake quickly (and quietly – the document now available on their website states the correct figures with no mention of the initial mistake), but it’s less than two weeks since shadow home secretary Chris Grayling got his wrists slapped for making comparisons between crime figures in the late 1990s and 2008/09, which the UK Statistics Authority said were “likely to mislead the public” because of changes in the way violent crimes are defined and recorded.
Rational behaviour
In this Guardian article from last week, shadow chancellor George Osborne shares a byline with Nudge author Richard Thaler.
The Tories are big on behavioural science and Thaler is clearly perfectly happy to align himself with them. “We are working together to formulate a series of public policies,” the pair write, and the ideas they have in mind present a hell of an opportunity for research.
The article reveals that under a Tory government, public bodies carrying out marketing campaigns would be required to state how they intend to change behaviour, and that a portion of the ad agency’s fee will be contingent on achieving the desired outcome.
That’s a pretty ringing endorsement of behavioural economics and social psychology, the academic disciplines that have provided the models for how behaviour can be influenced. The Tories are pinning their hopes on this science to provide a framework for ensuring and measuring effectiveness in their communications. The COI is already in the process of developing an organisation-wide approach to embedding behavioural theory in everything it does.
It’s an approach that could transform how research is used and how much it is valued.
Oh, behave
In an opinion article in yesterday’s Times, Daniel Finkelstein writes about how “group identity is just as important as economic incentive in the way we behave”.
Ideas like this about human behaviour and how it affects our understanding of society and economics are working their way into the mainstream. There seems to be a growing realisation among those who have to try to understand and change people’s behaviour that it’s quite a lot weirder and less rational than we all thought. Finkelstein explored similar issues in a BBC radio report about ‘the science of persuasion’ last year.
We’ll be looking at new thinking on human behaviour, and what it means for market research, in the February issue of Research.
Talking to real people
In this age of digital communications and social networks, we’re forever hearing about how technology is allowing companies to keep in touch with their customers. What we don’t tend to hear so much talk about is how technology has helped them avoid their customers.
After all, a big part of the success of many internet companies lies in the fact that if you operate online, you don’t have to have staff in shops or call centres. This works fine a lot of the time, but as soon as a customer has a query that’s outside of the usual realm of experience and which requires intervention by a sentient being with a brain and mouth and hands, they find themselves in a terrible maze of FAQs, links marked ‘Help’ which go miles out of their way to not provide help, and links marked ‘Contact us’ which lead to pages containing not a single option that approximates to ‘contacting’ anybody. This is the reality of a lot of ‘customer relations’, if it can be called that, and it ain’t pretty.
Clive James was ranting about this on BBC radio the other day. He spoke about being asked by a machine in the BBC accounts department for his VAT registration certificate every year for the past seven years. There is no reasoning with the machine. Then he has a go at automated phone systems, saying: “Getting in touch with any large organisation by phone has got harder and harder as the system of getting in touch has purportedly been made more efficient, by the provision of ‘options’.”
So let’s not get carried away with how technology and the internet have revolutionised the way companies relate to people. It works both ways.
Loser-generated content
The Guardian’s Charlie Brooker has written a rather enjoyable rant about ads featuring what he terms ‘loser-generated content’.
The ads he talks about are really less about crowdsourcing and co-creation than the appearance of crowdsourcing and co-creation, which can be just as artificial as doing things the old-fashioned way. Which goes to show that there’s still plenty of maturing to do in this area. Presumably once the idea of ordinary people getting involved in this sort of thing is less shiny and new, we won’t see it presented quite so ostentatiously.
A question of trust
Over at the FreshMinds Research blog, Dave Bevan has posted this amusing video from the Onion about Google and online privacy. Bevan argues that attitudes to privacy include two questions: how you feel about consumerism and how much you trust those in power.
“It only makes sense to jealously guard your data if you believe either that it will be used to dupe you into consumption that is bad for you or others, or that it is likely to fall into the hands of nefarious governments,” he writes. Otherwise, you risk just “sticking tape over your own mouth”.
It has to be said, the privacy campaigners who have argued vehemently against online tracking activities have not done a good job of articulating the threat. It’s more that they believe that information will inevitably be abused because either they don’t like consumerism or they don’t trust those in power, or both.
There’s no question that the systems used by public and private organisations to manage our data are open to abuse. The UK’s information commissioner says 434 organisations have reported data losses in the past year – a figure it calls “unacceptable”. That includes hospitals and various government departments, as well as 200 private companies. There’s nothing to suggest that the behaviour of any of these organisations was “nefarious” – just irresponsible or incompetent to some degree – but the potential is there.
Companies engaged in ISP-level behavioural tracking say they want to provide better advertising, but sceptics might argue that’s just another way of saying they want to watch everything you do in order to sell you more stuff.
As for cloud computing, there are obvious concerns associated with putting all your personal data in the hands of a big corporation like Google, but then a lot of us are already doing this with our web-based email, photo albums and social networks (which, of course, we all access for free).
Whether these threats are real or perceived is not really the point because this is ultimately a question of trust, which, as we all know, is something that has to be earned.
The rise of social media has revealed that how we thought we felt about privacy isn’t how we really feel. It’s not just about privacy, it’s about control. In the age of Facebook and Twitter, we seem willing to share pretty much anything online – but it has to be on our own terms. So we might choose to put information online but it remains our information, and woe betide anyone who breaches our trust by trying to do anything with it that we didn’t know about or explicitly allow.
As Adam Phillips, who chairs Esomar’s professional standards committee, said at the IJMR Forum earlier this month, researchers should avoid observational techniques that are “not yet technically illegal”. Going as far as the law allows might seem OK in the short term, but as trust erodes beneath your feet you’ll soon find yourself in difficulties.
The reason that behavioural targeting companies like NebuAd (now defunct) and Phorm have had trouble is that they have not won people’s trust. Privacy is turning out to be a very touchy subject, and it’s all too easy to cause suspicion or alarm if you don’t tread very carefully. Convincing yourself that you’re doing something harmless, or even virtuous, is one thing, but winning the trust of the people whose information you want to use is quite another.
The mind's eye
BBC One’s Inside Out London featured a nice segment on neuromarketing and eye-tracking research yesterday, with some interesting footage and interviews with Iain Janes of Eyetracker, Gemma Calvert of Neurosense, Richard Malton of JCDecaux and Robin Wight of ad agency WCRS.
“The future for market research is open to any kind of technology that will help advertisers understand what really motivates us at the most basic level,” said presenter Jo Good. Can’t argue with that.
But seriously, folks...
In an interview in today’s Frankfurter Allgemeine Zeitung, the German satirist Martin Sonneborn tells of his experience of the world of MR.
Sonneborn did some door-to-door survey work while he was a student, before joining the satirical magazine Titanic, of which he later became editor.
He says his main priority in the job was to put in the minimum possible amount of effort, which meant that if the most positive answers were listed first on the questionnaire, then an overwhelming majority of respondents would end up saying (for example) that having a fridge in their living room was a brilliant idea.
But it wasn’t a complete waste of time – Sonneborn, who now writes for Spiegel Online and can be seen on ZDF’s Heute Show, says the experience taught him some useful lessons. “I know now how surveys can be manipulated and how to evaluate market research,” he says.
Speaking of satirists, Armando Iannucci (the man wholly or partly responsible for On The Hour, The Day Today, Alan Partridge, In The Loop and The Thick Of It) will be one of the keynote speakers at Research 2010 in March. Keep an eye on the Events section of Research-live.com for more details.
Ask a stupid question
We’re a little late in highlighting this nugget of survey abuse, but it’s definitely one worth covering. The culprit this time is one who we think really should know better: the service veterans’ charity Erskine.
Erskine put out a press release in time for Remembrance Day which begins: “School children believe Adolf Hitler was coach to Germany’s national football team and that the symbol of Remembrance Day is the McDonald’s Golden Arches, according to new research released today.”
Perhaps you saw it reported in The Times, the Mirror, the Express, the Mail, the Telegraph, the Metro, the Star, the Herald, the Daily Record or the Dundee Courier & Advertiser.
Words fail us on this one. First, as you may have guessed, these “shocking” answers were not provided spontaneously but came in response to multiple choice questions. So less a ‘survey’, more a quiz. We wouldn’t be surprised if many of the kids ticked the ‘German football coach’ box just because it was funnier than saying Hitler was leader of the Nazi party.
Choices offered on other questions included that Joseph Goebbels was Terry Wogan’s replacement on Radio 2 (1.27%), Winston Churchill was a 1950s pin-up (1.47%), and the First World War was triggered by the murder of John Lennon (15.02%). As you can see, Erskine has decided that these important findings merit two decimal places.
In the case of the Hitler question, 6.88% was enough for them to claim that “school children believe” he was the German football coach. On the McDonald’s question, it was 12.73%. Both those questions were answered correctly by more than three-quarters of the kids.
Erskine calls the results “astonishing”. No. It’s the questions that are astonishing.
An invitation to comment on one of the press reports of the survey asked: “Do you despair of education in this country?” No. We despair of PR surveys in this country.
Watch out for the December issue of Research, in which we’ll be taking an in-depth look at the public understanding of research and stats, and what the MR industry can do about it.
Coffee house culture
Last night on BBC Radio 4, Adam Hart-Davis’ The Eureka Years looked at 1650, when an exciting new drink called ‘coffee’ was transforming London’s intellectual culture. Coffee houses became the respectable alternative to taverns, serving a drink that sharpened rather than dulled the senses and fuelled conversation about arts, science, politics and business. Lloyds’ insurance market, the Stock Exchange and Newton’s theory of gravitation all have their origins in the coffee house.
Tom Standage, business editor of The Economist by day and an expert in the history of coffee by night, draws parallels between coffee house culture and the internet: “Coffee houses tended to have subject-specific alignments, so if you were the clergyman you would go to this one, and if you were an actor you went to that one and if you were a sailor you went to that one, and so forth. They were a bit like websites, and you’d sort of go to the ones that matched your interests…
“The scientists all went to a bunch of coffee houses where they would discuss scientific matters, so the Royal Society would meet and then they’d retire to a coffee house. There were scientific lectures held in coffee houses - they were sometimes called penny universities because you could go in, buy your coffee for a penny and then listen to interesting people talk about interesting things and join in and pick up so much information in these places…
“Coffeehouses had publications on the table, the newspapers, pamphlets, that sort of thing, so you could absorb information on lots of different subjects very easily. You walk in, you buy your coffee, you go over and see what people are talking about, read the news and so on. So it’s very much like browsing the web today.”
The power of coffee house conversation was understood by Charles II, who tried to ban them in the 1670s, fearing that they were a hotbed for radical political thought. But by that point they were just too ingrained in the culture, and he had to live with them. In Paris, coffee houses were teeming with spies working for the crown, and archives describe the anti-royalist buzz they picked up. Companies concerned about what people might be saying about them do similar things via social media monitoring today (although generally with a more constructive approach when things don’t go their way).
These days, coffee shops don’t really play the role of a venue for networking. We’re not short of places to buy a latte, and different coffee shops attract different crowds, but I don’t know of any in London that cater particularly to people who want to talk about naval history or microbiology or Japanese animation.
Nor do they typically host lectures, and in most cases, striking up a conversation with a complete stranger about science or politics or some other weighty subject would be considered odd at best, and rude at worst. Unless, of course, you’re doing it on the internet, via your laptop, using the free wi-fi.
350 years ago, niche audiences and communities thrived with nothing more high-tech than some caffeine, a pamphlet and a roaring fire. It’s taken a while for digital technology to bring us to the same point.
NB. because of the vagaries of the BBC iPlayer the audio link above only works if you’re in the UK, and the actual programme begins about 2 minutes into the recording
Social media is...
Social media has a habit of inspiring people to purple prose. Here are some of Mediawatch’s favourite examples of flowery descriptions and ambitious analogies:
“Social media is like teen sex. Everyone wants to do it. No one actually knows how. When finally done, there is surprise it’s not better.”
Avinash Kaushik of Google says what we were all thinking, March 2009
“Social media measurement is the meteorology of marketing, and a lot of social media ‘work’ might as well be rain dances.”
Tom Ewing of Kantar warns that some people don’t know as much as they claim to, September 2009
“The expanse of available content ranges from the list servers and bulletin boards of yesteryear to the more ephemeral staccato of Twitter.”
Scott Evans of Harris Interactive makes Twitter sound cleverer than it is, November 2009
“[Researchers have become] ticks on the back of the information hippo”
Tom Ewing again, in philosophical mood, October 2009
“In real life we can sit through a boring presentation and then have a whispered chat with our neighbour about how dull it was, before putting on a bright smile and congratulating the speaker as they breeze up to us and ask what we thought. If we thought that the entire conversation with the neighbour would be retrievable later by the speaker, we might act rather differently.”
Alison Macleod says social media isn’t always as unfiltered as we might think, May 2009
“As more human behaviours emit trails of digital residue, more opportunities reside for algorithms to harness those human-induced data.”
Max Kalehoff, ex-BuzzMetrics, makes it all sound rather messy, March 2007
“[The potential of tracking vast amounts of information] raises the question of why anyone would want to track all the information in the world. It’s a bit like saying, ‘Let’s drop in on every conversation in the UK now to see if they’re talking about our product.’”
James Cherkoff of Collaborate Marketing suggests that some of what’s said on the internet might not actually be that interesting, January 2008
“Adding ‘listening’ techniques to ‘asking’ is like going from an X-ray to a CAT scan.”
David Wiesenfeld of Nielsen on tracking your brand’s health, September 2009
Crowdsourcing... or just crowd pleasing?
Levi Roots hasn’t done badly for himself. Since his appearance on the BBC’s Dragons’ Den in 2007, his Reggae Reggae Sauce is available in all the big supermarkets as well as Wetherspoons pubs, and he’s got his own TV show on Caribbean cooking.
But the dragons cheated a bit on this one. The point of Dragons’ Den is supposed to be to pick out people with great business ideas and potential. All they did with Levi Roots was cash in on the primetime TV exposure. However tasty it is, would Reggae Reggae Sauce have made it on to the supermarket shelves without Levi’s TV appearance? Mediawatch doubts it.
It’s a similar story when you look at high profile examples of crowdsourcing. Millward Brown’s Nigel Hollis (from whom we admit to having half-stolen the title of this post) has been blogging about the technique entering the mainstream. But how often is it just an attention-grabbing gimmick? Crowdsourcing exercises so often involve high-profile calls for action that it can be difficult to tease apart the effect of the publicity you get when you loudly ask for the public’s views, and the value of the ideas that they actually come up with (compared to what ad agency creatives might have managed).
Unilever has ditched its ad agency of many years and turned to crowdsourcing site Ideabounty to find the idea for its next Peperami ad. More than 1000 entries have been recieved, and here we are writing about it before the idea has even been decided on. Of course, engaging the public is all part and parcel of how crowdsourcing works, but Mediawatch wonders whether, when the brands involved have enough clout to rely on getting some coverage and attention, the search for ideas is really an afterthought.
Most suspicious of all is the tale of Kraft’s new Vegemite variant, Cheesybite, which started out with the silly name of iSnack2.0 – supposedly crowdsourced from entries to a competition. The name was then ditched with much fanfare and replaced by Cheesybite, which came out of an online vote.
Since nobody in their right minds would call a product iSnack2.0 unless they were joking, we can take this to be a very successful publicity exercise. In a conversation between some Australians (the main – or perhaps only – target market for Vegemite) overheard by Mediawatch this weekend, there was much talk of Cheesybite, focusing mostly on the naming controversy rather than the merits of the product itself. Could Vegemite’s marketing team have come up with as good a name by themselves without all this hoo hah? Of course they could have. But (as the Google Trends results for Vegemite testify) they wouldn’t have got anything like as much publicity.
Learn to count with Nick Griffin
Among all the outrageous things that British National Party leader Nick Griffin said on Question Time last week, one detail caught Mediawatch’s eye:
Nick Griffin: “Our immigration policy is, I think, supported by 84% of the British population at present, who, according to a very recent opinion poll said they were very worried about immigration, it should stop…”
Bonnie Greer: “Which opinion poll, Nick?”
Nick Griffin: “I honestly don’t remember.”
Bonnie Greer: “Well there you go!”
Nick Griffin: “It was in the papers only the other day.”
Don’t worry Nick, Channel 4 News’ FactCheck has stepped in to look up the source for you. It turns out that the stat in question was cited by Andrew Green of anti-immigration group Migration Watch UK in a Daily Mail article published on the day of Griffin’s BBC appearance.
But wait a second, it looks like Griffin may have got his numbers a little mixed up. He seems to have decided that everyone who said they were “concerned” about the prospect of the UK’s population rising to 70 million in a poll conducted in July must be in support of his party’s policy (which, let’s not forget, involves an end to immigration and voluntary repatriation for people “of foreign descent”). And there’s a question mark over the figure cited by Green anyway.
The figure for the percentage of people who actually said they’d support an end to immigration in YouGov’s July survey was just 22%.
OK, so it’s a long way from being the most shocking thing that Griffin said on Thursday night, but it tells us something about his attitude to poll data.
Is someone tracking my mobile?
After stumbling across an article in The Times the other day entitled ‘Are foreign governments tracking your mobile?’, Mediawatch has been pondering what the future might hold for mobile as a research medium.
Our first reaction to the headline was to recall Andrew Marr’s advice on how to read newspapers: in his book on journalism, Marr suggests that whenever a headline is a question, you should try answering ‘no’, because if the answer were ‘yes’, the writer would probably not have phrased it as a question.
But The Times’ Mark Frary claims that some companies are worried enough that foreign governments (for ‘foreign’, read ‘Chinese’) are tracking their employees’ mobiles that they’ve issued them with special signal blocking pouches to put their phones in when they visit China.
It works both ways, of course – your own employer might equally want to use the phone they’ve issued you with to keep tabs on where you are.
If you’re carrying a mobile, then your incoming and outgoing calls can be tapped, your approximate location can be plotted by looking at which transmission masts you’re in range of (the same way some phones find your location on a map) and if you’ve got a smartphone with GPS, your location can be pinned down with a great deal more accuracy.
Clearly mobile offers massive potential for “in the moment” research, location and event-based research, personalisation and so on. But The Times’ story shows how these things can also seem a bit creepy, and raise unexpected questions, which could make it much tougher to get people to take part in mobile phone-based research.
It’s worth treading carefully because, as anyone working in behavioural targeting can probably tell you, it doesn’t take that many headlines with question marks at the end to make people suspicious.
Seeing Red
The Telegraph offers another example of the credulous reporting of surveys today.
The headline reads: “Women without children should be allowed to take maternity leave, survey says”. The story goes on to say that “women who do not have children should be allowed to take maternity leave… according to a study”.
A study, you say? By an influential think tank perhaps? Or some boffins at a top university? Actually no – it was a survey of 2,000 women by Red Magazine.
Clearly the Telegraph was relying on snaring readers with an exciting quirky headline. If it were true, this would indeed be big news, as the idea of offering maternity leave to non-mothers is as mad as the idea of offering free trained labradors to people with perfect vision, or free lifts around town in ambulances to people who aren’t ill.
But let’s be clear about things for a second. Red’s press release states that 74% of women who answered its survey were “in favour of introducing the option of leave to reassess their career or take time out from the stress of work”. So it might have been more accurate for the Telegraph to say that a lot of women (and probably men too, if they’d been asked) “quite fancy” a break from work, rather than that they “should be allowed to” – a phrase which, combined with the rather loosely applied term “study”, gives the false impression that somebody other than the respondents themselves has considered this.
It’s a bit like asking 2,000 children if they want some chocolate, then running an article saying: “Children should be given free chocolate on demand, according to a study”.
The comparison with maternity leave only adds another layer of confuddlement.
But we should have been expecting this. This is, after all, the same paper that recently brought us “Crumbs: half of Britons injured by their biscuits”.
Picking on research
If you want to convince people you’re a clued up, no-nonsense businessperson, try picking on research.
We see examples of this all the time, and Research Rockstar has just pointed us to a new one – a Forbes article about the shortcomings of research.
Marc Babej of marketing strategy consultancy Reason writes that “conventional research tools are ill-suited to assess the impact of marketing investment on behaviour”, and offers five sound (if obvious) tips for making sure you don’t waste your investment in research.
“Just because someone thinks a BMW is the best car out there doesn’t mean she’s going to buy one,” he writes, which puts me in mind of a comment that Millward Brown’s Graham Page made to me in an interview last year: “The argument that you can’t ask people why they do stuff if probably right. But that’s why most market research doesn’t.”
Babej’s points are valid, as far as they go, but really all he does is to take a very narrow definition of ‘research’, and describe how it can be misapplied.
The real issue here is how research is perceived. If you believe market research = surveys and charts and focus groups and stuff, and if you believe the value of research = the thickness of the report or the amount of money you spent on it, then you’re going to run into problems.
Fortunately, most people realise that the implementation of research is part and parcel of the process – and that the responsibility for making sure that it has an impact lies with both clients and agencies.
Shifting focus
David Cameron appeared on the Today programme this morning, discussing a proposal to raise the state pension age to 66 as early as 2016. One brief exchange made our ears prick up:
Evan Davis: “Have you focus group tested this, out of interest? Have you tried it out on people? Is it going to go down well or badly?”
David Cameron: “I’ve never been to a focus group.”
He didn’t say categorically that he never will go to one, or that he’s never used one, but still.
New Labour made heavy use of focus groups and other qual research techniques in an effort to be seen to be listening to people. But the practice got a bad name for being used to justify bad policy, and deliberative research exercises were accused of being nothing more than thinly disguised PR campaigns.
Now, Cameron – a man frequently accused of being all style and no substance – is distancing himself from the idea that the best way to make policy is to sit down with the general public over a cup of tea and see which of your proposals least annoys them. “When it comes to reducing public spending and getting our public finances under control, there are not any popular options,” he said.
Labour’s recent ducking of questions on spending cuts has given Cameron the opportunity to come across as refreshingly blunt on this issue. But it’s also yet another example of “focus group” as shorthand for “research done badly”. Let’s hope whatever government we choose for ourselves next year can find a way to make research strengthen public trust rather than undermining it. It can’t be that hard.
Getting the right results
The University of Westminster’s Steven Barnett criticises the government’s “blatantly one-sided questionnaire” (commissioned from BMRB, where Barnett himself once worked) and the use of the results in a speech by culture secretary Ben Bradshaw.
The way questions were presented meant that the 65% of respondents who Bradshaw says agreed with the government’s proposal “had virtually no choice”, Barnett said.
This sort of analysis of survey data is all too often lacking in the press. Often the best they can manage is fluff like this. And The Guardian gets extra points for illustrating the online version of the story with a clip from Yes Minister.
View results 10 per page | 20 per page | 50 per page
