All posts from: May 2010
The headmaster of Eton College has said that an obsession with data is harming boarding schools.
Speaking at the Boarding Schools’ Association’s annual conference earlier this month, Tony Little said: “It is a sad thing, it seems to me, that where once men were able to speak of sweeps of history such as the Age of Enlightenment or the Age of Reason, we now seem to inhabit the Age of Measurement… Our day-to-day lives are circumscribed by a variation of the McNamara fallacy: only that which can be measured has worth, if it cannot be measured it can have no worth. This kind of thinking cuts to the heart of everything I believe in as the head of a boarding school.”
Inspections have helped to bring improvements over the years, he said, but measurement “comes at a cost”, distorting the way staff and students work and putting short-term goals ahead of long-term success. Teachers have become more risk averse and pupils have worked out how to achieve the stated targets with the minimum effort.
In a commercial context, market researchers are the ones who have to justify the cost of measurement – both the money spent on it and the unintended consequences of how you choose to do it on how your business performs.
In recent years there has been growing pressure to demonstrate returns on investment in research – to measure your measurement. Little’s comments provide a warning that this sort of thinking can only go so far.
Market research is about making better decisions. But it’s not always easy to know what decisions you might have made, or what the consequences would have been, had it not been for the research. So for all its boardroom appeal, attaching a simple, numerical value to research can be misguided.
Tony Little says this sort of overemphasis on meeting targets is at odds with everything he believes in. Before relying too much on measurement for reassurance, buyers of research need to consider what they believe in too. Putting your customer at the heart of your business. Basing your decisions on evidence. Staying alert to change. There’s value there that everyone can understand.
It’s to the Canadian market research industry’s eternal credit that they’ve managed to resist the urge to scream “told you so” amid criticism of massive government cuts in spending on public opinion research which have left policy-makers and civil servants with little evidence to work with.
A recent article in the Ottawa Citizen reports on the concerns of Parliamentary Budget Officer Kevin Page and others about a lack of data needed to assess the performance of government policies and programmes.
The paper pins the blame for this situation squarely on the huge decline in research spend which has fallen from a high of C$31m a year in 2007 to just $8m last year – a far cry from the earlier, comparatively modest (yet still substantial) government pledge to reduce the public opinion research bill by C$10m.
According to Ekos Research’s Frank Graves:
“Research is a dirty word in Ottawa right now. It’s not being done when the country is going through a period of turbulent change and enormous challenges.”
Ekos was once the top supplier of public opinion research to the Canadian government. In 06/07 it handled over C$6m in contracts, which was cut to C$3.28m 07/08 and again in 08/09 to C$1m.
“We have a policy community operating without evidence and empirical guidance. Every major debate from national unity, free trade, health care, bank reform should have extensive public opinion research to see what the public thinks.”
Judging by the Ottawa Citizen’s report, the situation in Canada is playing out exactly as predicted by the country’s Marketing Research and Intelligence Association, which warned at the time the cuts were announced that the reduction in spend could easily damage policy-making and programme delivery capabilities.
“The real issue is not one of ensuring that this government spends less on public opinion research… but rather of ensuring that Canadians receive the best value and best output from government programmes and policies,” said the MRIA.
The industry must be hoping now that finally their argument will be heard, however belatedly.