Mediawatch
Consultation or nonsultation?
In the Daily Telegraph, Andrew Gilligan has been having a go at ‘nonsultations’ – cynical attempts by government to lend legitimacy to pre-determined decisions, masquerading as consultation exercises.
The purpose of nonsultation, Gilligan writes, “is almost never to act on the public’s views. It is to manage, manipulate, or suppress them”.
He is particularly scathing of the government’s current Spending Challenge, which seeks to gather suggestions for saving public money. Gilligan’s suggestion, predictably enough, is to scrap consultations.
“Nonsultation’s glossy brochures are no substitute for informed decision-making and real democratic engagement,” he argues.
Cynicism about public consultation exercises is, of course, nothing new, but Gilligan has given us a rather nice new term to describe it.
The difficulty of proving your point
In the preface to their 2009 book The Spirit Level, Richard Wilkinson and Kate Pickett use data from the World Bank, WHO, the UN and the OECD to argue that more equal societies do better in almost every respect than more unequal ones. The prevalence of problems as diverse as infant mortality, imprisonment rates, teenage pregnancy, murder and obesity is, they say, statistically linked to income inequality, and the book is filled with graphs illustrating these correlations.
But their conclusions have come under attack from right-wing think tank the Policy Exchange, as well as the Taxpayer’s Alliance. In an article for the Guardian the Policy Exchange’s Natalie Evans says the claims they make are not supported by the statistics. Some of the conclusions, she says, rely on one or two extreme cases to make a case, while others result from “clusters” of countries that tend to show similarities, such as the Scandinavian or English-speaking nations.
Of course, a heated debate has ensued, not least on the comments thread of Evans’ article. What’s interesting is how little the discussion focuses on the facts. The problem with topics like this is that some people are simply inclined to believe one thing or the other, regardless of evidence. Many people (particularly Guardian readers, we suspect) hold the book’s central assertion that inequality is bad for us all to be a self-evident truth. They will continue to believe it with or without statistical analysis to back it up, and they are happy to say so. Similarly, one suspects that the likes of the Policy Exchange and the Taxpayer’s Alliance are inclined to believe that a society where people are free to enrich themselves is better because, well, that’s what they believe.
If research is about making an argument or telling a story based on evidence, it’s worth considering just how much people’s view of (or interest in) the evidence can be influenced by what they already think.
Is anybody listening?
The UK’s new government has been keen to position itself as one that listens, using crowdsourcing in high-profile attempts to garner comments and suggestions on its legislative programme, on planned public sector spending cuts and on unnecessary laws that should be repealed.
But government consultation exercises have a chequered reputation, and questions are already being asked about whether the new administration really wants to hear ideas.
The Guardian reports this morning that the 9,500 online comments on the government’s programme have resulted in zero changes to policy. In responses published “without publicity” various govenrment departments have, for the most part, either interpreted comments as an endorsement of existing policy or dismissed them.
Were the public’s suggestions too daft to be of any use? Were the government’s policies so perfectly honed that there was nothing we could add? Or was government not as interested in hearing constructive criticism as it thought it was?
Simon Burall, director of Involve, a not-for-profit body promoting public engagement, told the Guardian: “You have to give the government some credit for trying to do this, but badly designed consultations like this are worse than no consultations at all.”
It doesn’t bode well for the ongoing ‘Spending Challenge’ and ‘Your Freedom’ consultations.
In Burall’s words, trying to take credit for listening to people without actually doing it will only “diminish trust and reduce the prospect that people will engage again”.
In iPad we trust?
We couldn’t help but be a little bit concerned at reports earlier this week about how the iPad had given a boost to market research interviewers, by suddenly making the public more interested in them.
If researchers are having to rely on shiny new devices to persuade people to talk to us, things are worse than we thought.
What happens when the fuss about the iPad dies down? Will interviewers need to be issued with jetpacks? Lightsabers? Hoverboards?
There is certainly potential in the iPad to make the survey experience cooler, but let’s not pin our hopes purely on its novelty value.
No time like the present
Tim Harford had a nice article this weekend about how working out what might happen in the future is only slightly more difficult than working out what’s happening now.
He highlights a prediction made in the Financial Times on New Year’s Eve 2007, that the US would “avoid tipping over the brink” into recession. Not only did it turn out to be wrong, but we learned later that at the time of publication, the US was already in recession.
Real-time data from the web, despite all its limitations, provides a way around this problem. And while Harford is talking mostly about economic forecasts, the same goes for market research.
Zogby turns his guns on Nate Silver
The relationship between pollsters and the media is becoming increasingly fuzzy. It used to be that pollsters polled the public and papers published the results. Politicians questioned the findings and everyone moved on to the next survey. Times were simpler when the polls were the story. Now its the pollsters themselves in the spotlight.
One can credit this turn of events to the increasing numbers of poll analysts – the Nate Silvers and the Mark Blumenthals of this world. The former runs FiveThirtyEight.com, which just recently signed a content-sharing deal with the New York Times, while the latter runs Pollster.com, acquired last week by online news site The Huffington Post.
Pollsters are fast becoming as much of a talking point as the subjects they investigate, but its not clear that all welcome the increasing scrutiny they find themselves under. Del Ali is a case in point. His firm, Research 2000 has been accused of fabricating polls based on a statistical analysis of results – charges he dismisses as “pure lies”. Whether they are or not, it can’t be good for business.
Then we have veteran pollster John Zogby, who this week attempted to turn the tables on FiveThirtyEight’s Silver with his ‘Note to Nate’, an open letter published in the Huffington Post that was a cocktail of fatherly advice that often crossed the line into condescension and frustration with things that Silver has written about Zogby in the past (He once called Zogby ‘the worst pollster in the world’).
“You take other people’s polls,” Zogby said to Silver, “compare records for predictions, add in some purely arbitrary (and not transparent) weights, then make your own predictions and rankings.
“To date you have many fans. But the real scrutiny is just beginning and some fans are ephemeral.”
Silver, though, was clearly in no mood for a drawn out debate, and you can read his response here.
Most intriguing in all this is Zogby’s reasons for writing the letter. Perhaps it stems from a piece written by Silver last year when the blogger called Zogby International’s online polls “error prone” before deciding that: “These internet polls, simply put, are not scientific and should not be published by any legitimate news organisation.”
Of course, the NY Times deal puts Silver in the ascendent, so is Zogby simply looking to keep the analyst’s feet on the ground?
Zogby rounds up his letter by reminding Silver that the blogger is “a statistician – a very good one – but you are not a pollster” before advising him to carry out some polls of his own to learn how the rest of the industry does it. The underlying tone is that of an old lion stamping its authority on the young cubs.
But a pollster should know better than anyone that you don’t need to be an expert at something to have an opinion on it. There would, after all, be fewer qualified respondents and fewer opinion polls to analyse if that was the case.
Sponsors recover from World Cup ambush
While the questions about formations are asked and managers draft their resignation letters, another type of enquiry is going on in the boardrooms of the companies that have ploughed large sums of money into the 2010 World Cup to become official partners or sponsors. “Has it worked?” they’ll ask.
Big brands arrived in South Africa having spent top dollar on securing their place as sponsors and received backing in the form of new legislation that restricted the sale of ‘non-sponsor’ products at grounds and within a certain distance of the stadiums.
With this sort of support in place the likes of Budweiser, Adidas and Coca-Cola couldn’t lose – or could they? Any football fan will tell you that there is always a shock result in the early stages of the World Cup, whether it’s Cameroon beating Argentina in Italia 90, an England goalkeeper spilling a weak American shot into his own net or any number of tiny principalities or republics beating Scotland.
This year this tradition played out in the marketing game too, as non-sponsor companies Nike and Bavaria stole the limelight with their epic adverts and orange dresses.
Research from NM Incite, a Nielsen and McKinsey joint venture showed that in the build-up to the World Cup Nike stole a march on the official sponsors in the online buzz stakes, claiming 30.2% of World Cup-related messages, twice as many as its rival Adidas.
However, the research shows that after the first two weeks of the tournament Adidas had increased its share of buzz from 14.4% to 25.1% while Nike has seen its share fall to 19.2%.
On the beer front Budweiser saw off early ambushes from Carlsberg’s rather spooky advert and Bavaria’s girls in orange dresses to become the most talked about brand, claiming 4.9% of online buzz compared to Carlsberg’s 2.4%.
NM Incite compiled a list of the top ten most talked about brands after the first two weeks of the tournament and, to continue the footballing analogy, the big boys have done well and are progressing nicely, despite the earlier blips.
Adidas, in part because of the conversations about its controversial Jubilani ball, tops the list with 25.1% of World Cup-related buzz, followed by Nike (19.4), Coca-Cola (11%) and Sony (9.8). In fact, of the top ten most talked about brands, only three – Nike, Pepsi and Carlsberg – are not official World Cup sponsors or partners.
So it would seem that the investment has paid off for the official partners, but the ambush companies can feel satisfied with their performances too. They arrived in South Africa with no expectations and shook some of the bigger players with their early displays and can definitely leave with their heads held high.
There is one interesting footnote to this story that might affect Nike’s performance should they ever try a similar campaign to this year’s ‘Write the Future’. The players featured in the campaign all had a World Cup to forget, if they even made it there in the first place.
Brazil’s Ronaldinho and England’s Theo Walcott feature heavily and were not named in their country’s squads while the stars of the ad - Rooney (England), Riberry (France), Cannavaro (Italy), Drogba (Ivory Coast) and Ronaldo (Portugal) - all performed well below par and were on early flights home after they were knocked out.
The voice of the public
Deborah Mattinson, who founded Opinion Leader and until earlier this year was joint chair of Chime’s research division, appeared on Radio 4’s Today programme on Monday to talk about her new book, which is based on her experience as a pollster during the New Labour years.
The book’s title ‘Talking to a brick wall’, hints at a rather jaded view of how government has used public engagement. She spoke of her frustration at the many examples of “the public being in one place and the Westminster village being in another”.
Mattinson speculated about how the new coalition government’s latest engagement initiative, on the nasty subject of spending cuts, might go down with voters.
“It’s quite interesting now to see how that will unfold,” she said. “Will the voter really be taken along? Will the voter really be properly consulted? Will the voter really be made aware of those trade-offs as those cuts are considered, or not? If not, then it will, I’m afraid, be more of the same.”
The (marketing) goal of the tournament
The 2010 World Cup is less than a week old and the biggest story – off the field at least – has highlighted the complex and cutthroat world of official sponsorships and marketing.
Let’s start at the beginning. During the match between the Netherlands and Denmark earlier this week the camera paused for a second on a group of Dutch supporters in the crowd, all wearing identical orange mini dresses. This searching out of attractive female fans for a quick crowd shot has become a World Cup tradition over the years so the majority of people didn’t pay much notice.
But some people most certainly did. Before the end of the game all 36 women were ejected. As it turns out they had been kitted out in their dresses by Dutch beer company Bavaria, who are not an official sponsor.
In a further twist the women were in seats allegedly allocated to ITV pundit and former international footballer Robbie Earle, who has since been sacked by the broadcaster.
As the story developed the general public were introduced to the phrase ‘ambush marketing’ – or, for the uneducated, the act of promoting a company that is not an official sponsor. It’s a practice that has been going on for years but has never grabbed the headlines as much as this incident.
The situation is snowballing quickly and it has now emerged that two of the women have had their passports confiscated and face charges under the Merchandise Marks Acts and the Special Measures Regulations. Police spokesman Colonel Naidoo said the women were “suspect to be involved in organised acts to conduct commercial activities” and warned that the police view ambush marketing “in a very serious light”. The duo are on bail and will appear in court next week.
Dutch foreign minister Maxime Verhagen has weighed in to the situation, saying it it is “outrageous” that the women have been arrested.
While all this is going on there will be some very satisfied faces in the marketing department at Bavaria. For the cost of 36 little orange dresses they have got more publicity than the likes of Budweiser – who have poured millions into their World Cup campaign.
When the surveys start to ask which brands consumers remember from South Africa 2010 you can bet that Bavaria will be high on the list and the fate of the gaggle of Dutch women and the sacking of Robbie Earle, who is protesting his innocence, will ensure that their name remains in the headlines – and everyone’s minds – throughout the tournament and beyond.
The value of social research
Polly Toynbee spoke out in defence of Britain’s birth cohort studies in the Guardian on Monday.
A new study planning to track the lives of 93,000 children born in 2012 has yet to be given the green light by the new government, and in the face of massive public spending cuts its fate hangs in the balance.
The wealth of information from past studies is remarkable, says Toynbee: “It was the 1958 study that revealed mothers who smoked have smaller, sicker babies. Comparison between children of 1946 and 1958 saw them grow longer legs, with better nutrition ironing out class differences. The millennium study gave the wake-up call on finding a quarter of children obese by four years old. Only cohort studies could have revealed the sudden slowdown in social mobility between those born in 1958 and those in 1970. Answers are here to all the perplexing questions: what makes some children resilient to dreadful early beginnings while others are damaged for life? How do you protect the vulnerable before it’s too late?”
But these longitudinal studies have not been used to their full potential because of changing governments with changing priorities. The question now, she says, is “whether this government wants to keep a good record of what happens on its watch. That, in itself, will reveal a great deal about their true intentions”.
Daily Mail 'exposed' as serial online data-collector
A PR blogger has exposed the array of web analytics and online tracking tools used on the Daily Mail’s website – on the same day that the paper “revealed” how some companies use social media sites to track down unhappy customers.
In an article explaining how firms “spy” on customers’ Facebook accounts and “trawl the internet looking for disgruntled customers”, the Mail listed telecoms provider BT, EasyJet, Carphone Warehouse and Lloyds TSB as companies that were “monitoring social networking sites to see what is being said about them”.
It goes on to tell the story of an unnamed consumer who called the telecoms firm a “bunch of unaccountable, business-shafting bastards” on his Facebook account. This unhappy customer was contacted “within hours” by a BT representative through the site, offering to help him with the problems he had been encountering.
BT’s efforts were not well received, however, and the unhappy customer told the Mail: “What happened was quite Big Brother-ish and sinister.”
But blogger Andrew Bruce Smith, who runs the PR and analytics consultancy Escherman, sought to take the paper to task with a report on his own site that detailed the range of web analytics and tracking software used by the Daily Mail – including tools from Omniture, Sophus3, Google Analytics and ComScore – to gather information about its readers.
With tongue firmly in cheek, Smith said: “The amount of information that the Mail is gathering about its online readers is immense – everything from the kind of browser they are using down to their IP address. There can be no doubt that they are openly using this information to try and personalise their readers’ experience or – worse – coerce them into buying third-party products and services.
“We can only hope that their own journalists will apply the same rigorous approach as they’ve used with other organisations to write a follow-up story to expose their own colleagues’ questionable behaviour and flagrant disregard for privacy.”
The Daily Mail had not returned calls seeking comment at the time of publication.
Do we live in an age of measurement?
The headmaster of Eton College has said that an obsession with data is harming boarding schools.
Speaking at the Boarding Schools’ Association’s annual conference earlier this month, Tony Little said: “It is a sad thing, it seems to me, that where once men were able to speak of sweeps of history such as the Age of Enlightenment or the Age of Reason, we now seem to inhabit the Age of Measurement… Our day-to-day lives are circumscribed by a variation of the McNamara fallacy: only that which can be measured has worth, if it cannot be measured it can have no worth. This kind of thinking cuts to the heart of everything I believe in as the head of a boarding school.”
Inspections have helped to bring improvements over the years, he said, but measurement “comes at a cost”, distorting the way staff and students work and putting short-term goals ahead of long-term success. Teachers have become more risk averse and pupils have worked out how to achieve the stated targets with the minimum effort.
In a commercial context, market researchers are the ones who have to justify the cost of measurement – both the money spent on it and the unintended consequences of how you choose to do it on how your business performs.
In recent years there has been growing pressure to demonstrate returns on investment in research – to measure your measurement. Little’s comments provide a warning that this sort of thinking can only go so far.
Market research is about making better decisions. But it’s not always easy to know what decisions you might have made, or what the consequences would have been, had it not been for the research. So for all its boardroom appeal, attaching a simple, numerical value to research can be misguided.
Tony Little says this sort of overemphasis on meeting targets is at odds with everything he believes in. Before relying too much on measurement for reassurance, buyers of research need to consider what they believe in too. Putting your customer at the heart of your business. Basing your decisions on evidence. Staying alert to change. There’s value there that everyone can understand.
Research cuts hit home for Canadian government
It’s to the Canadian market research industry’s eternal credit that they’ve managed to resist the urge to scream “told you so” amid criticism of massive government cuts in spending on public opinion research which have left policy-makers and civil servants with little evidence to work with.
A recent article in the Ottawa Citizen reports on the concerns of Parliamentary Budget Officer Kevin Page and others about a lack of data needed to assess the performance of government policies and programmes.
The paper pins the blame for this situation squarely on the huge decline in research spend which has fallen from a high of C$31m a year in 2007 to just $8m last year – a far cry from the earlier, comparatively modest (yet still substantial) government pledge to reduce the public opinion research bill by C$10m.
According to Ekos Research’s Frank Graves:
“Research is a dirty word in Ottawa right now. It’s not being done when the country is going through a period of turbulent change and enormous challenges.”
Ekos was once the top supplier of public opinion research to the Canadian government. In 06/07 it handled over C$6m in contracts, which was cut to C$3.28m 07/08 and again in 08/09 to C$1m.
Graves continues:
“We have a policy community operating without evidence and empirical guidance. Every major debate from national unity, free trade, health care, bank reform should have extensive public opinion research to see what the public thinks.”
Judging by the Ottawa Citizen’s report, the situation in Canada is playing out exactly as predicted by the country’s Marketing Research and Intelligence Association, which warned at the time the cuts were announced that the reduction in spend could easily damage policy-making and programme delivery capabilities.
“The real issue is not one of ensuring that this government spends less on public opinion research… but rather of ensuring that Canadians receive the best value and best output from government programmes and policies,” said the MRIA.
The industry must be hoping now that finally their argument will be heard, however belatedly.
The cheapest way to measure ad effectiveness
In a letter to the Daily Telegraph, former Tory MP Edwina Currie suggests that the most cost-efficient way of gauging the effectiveness of public service advertising might be to stop doing any.
“Exhortations to stop smoking, seek help for cocaine sniffing, enter further education, avoid drink-driving, join the Armed Forces, and take a chlamydia test are doubtless worthy,” she writes. “But seeing them all at once, any intelligent viewer is bound to wonder whether the Government is trying to keep the advertising industry in business all by itself.”
Her suggestion is to scrap the COI’s £540m budget and see what difference it makes. And with post-election spending cuts looming, that prospect seems less unrealistic than it might once have done.
The difference between 'gay' and 'homosexual'
Today, a lesson in the importance of questionnaire wording. According to a CBS/New York Times poll, 70% of Americans support ‘gay men and lesbians’ serving in the military. But when it comes to ‘homosexuals’, it’s a different matter – only 59% are in favour of them serving.
The Obama team will no doubt be scratching their heads to see if they can get an 11-point poll lead on any other issues just by re-arranging the words.
Kate Harding at Salon.com says the poll provides hard evidence in support of activists who insist that word choice is important, and who are “widely dismissed as free speech-hating PC whiners who need to get a life”.
An important point
The Tories are in trouble today over an errant decimal point.
In a document released yesterday with the title Labour’s Two Nations, they claimed (three times) that in the most deprived areas, 54% of girls aged 15-17 get pregnant, compared to 19% in the least deprived areas. Labour were quick to point out that in reality it’s 5.4% and 1.9%. The Tories have acknowledged the error, and said a decimal point got lost somewhere.
The Tories were quick to correct the mistake, but Labour has leapt upon it as evidence of how out of touch they are. Everyone knows how easy it is to let a decimal point slip, but this was a major policy document. To believe that pregnancy rates for under-18s range from one in five at best, to more than half, does suggest a rather loose grip on reality.
This time they’ve corrected the mistake quickly (and quietly – the document now available on their website states the correct figures with no mention of the initial mistake), but it’s less than two weeks since shadow home secretary Chris Grayling got his wrists slapped for making comparisons between crime figures in the late 1990s and 2008/09, which the UK Statistics Authority said were “likely to mislead the public” because of changes in the way violent crimes are defined and recorded.
Rational behaviour
In this Guardian article from last week, shadow chancellor George Osborne shares a byline with Nudge author Richard Thaler.
The Tories are big on behavioural science and Thaler is clearly perfectly happy to align himself with them. “We are working together to formulate a series of public policies,” the pair write, and the ideas they have in mind present a hell of an opportunity for research.
The article reveals that under a Tory government, public bodies carrying out marketing campaigns would be required to state how they intend to change behaviour, and that a portion of the ad agency’s fee will be contingent on achieving the desired outcome.
That’s a pretty ringing endorsement of behavioural economics and social psychology, the academic disciplines that have provided the models for how behaviour can be influenced. The Tories are pinning their hopes on this science to provide a framework for ensuring and measuring effectiveness in their communications. The COI is already in the process of developing an organisation-wide approach to embedding behavioural theory in everything it does.
It’s an approach that could transform how research is used and how much it is valued.
Oh, behave
In an opinion article in yesterday’s Times, Daniel Finkelstein writes about how “group identity is just as important as economic incentive in the way we behave”.
Ideas like this about human behaviour and how it affects our understanding of society and economics are working their way into the mainstream. There seems to be a growing realisation among those who have to try to understand and change people’s behaviour that it’s quite a lot weirder and less rational than we all thought. Finkelstein explored similar issues in a BBC radio report about ‘the science of persuasion’ last year.
We’ll be looking at new thinking on human behaviour, and what it means for market research, in the February issue of Research.
Talking to real people
In this age of digital communications and social networks, we’re forever hearing about how technology is allowing companies to keep in touch with their customers. What we don’t tend to hear so much talk about is how technology has helped them avoid their customers.
After all, a big part of the success of many internet companies lies in the fact that if you operate online, you don’t have to have staff in shops or call centres. This works fine a lot of the time, but as soon as a customer has a query that’s outside of the usual realm of experience and which requires intervention by a sentient being with a brain and mouth and hands, they find themselves in a terrible maze of FAQs, links marked ‘Help’ which go miles out of their way to not provide help, and links marked ‘Contact us’ which lead to pages containing not a single option that approximates to ‘contacting’ anybody. This is the reality of a lot of ‘customer relations’, if it can be called that, and it ain’t pretty.
Clive James was ranting about this on BBC radio the other day. He spoke about being asked by a machine in the BBC accounts department for his VAT registration certificate every year for the past seven years. There is no reasoning with the machine. Then he has a go at automated phone systems, saying: “Getting in touch with any large organisation by phone has got harder and harder as the system of getting in touch has purportedly been made more efficient, by the provision of ‘options’.”
So let’s not get carried away with how technology and the internet have revolutionised the way companies relate to people. It works both ways.
Loser-generated content
The Guardian’s Charlie Brooker has written a rather enjoyable rant about ads featuring what he terms ‘loser-generated content’.
The ads he talks about are really less about crowdsourcing and co-creation than the appearance of crowdsourcing and co-creation, which can be just as artificial as doing things the old-fashioned way. Which goes to show that there’s still plenty of maturing to do in this area. Presumably once the idea of ordinary people getting involved in this sort of thing is less shiny and new, we won’t see it presented quite so ostentatiously.
View results 10 per page | 20 per page | 50 per page
