OPINION26 March 2014

Data dunces?


The White House has become the latest political target of criticism from the market research industry, after posting a flawed online poll on big data and privacy. Does this raise questions about politicians’ ability to legislate on data?

The White House poll, which had the aim – ironically – of gathering public views on data collection, has come under heavy criticism from the American Marketing Research Association (MRA), which branded it: “a good idea, poorly executed.” This criticism comes just days after the Market Research Society (MRS) accused the UK Conservative Party of breaching data protection laws with a survey posted on its website.

With big data and associated protection laws becoming an increasingly hot political topic, the lack of understanding of basic research techniques and standard data protection practice is a growing concern.

Little to no value

While the MRA has praised the White House for grasping the value of survey, opinion and marketing research, it lists a number of ways in which its short survey falls short of industry standards. According to the MRA, it fails to include sufficient context in questions about feelings towards data collection, failing to differentiate between uses of data by different government agencies, and failing to include any screening questions – meaning any individual in any country could complete the survey.

“Any agency that tried to run data collection this way would be unlikely to ever receive approval from the Office of Management and Budget (OMB) because of flawed design,” the MRA statement reads, “but OMB answers to the White House and not vice versa.

“Serious research requires industry-standard design and controls to ensure quality. Without these in place, the results are guaranteed to be inaccurate and of little to no value.”

Different country, same story

Perhaps more worryingly, a recent survey posted on the Conservative party website was judged to be a “vast contravention of personal privacy”. Chief executive of MRS, Jane Frost, had apparently warned Conservative party chairman Grant Shapps three weeks ago that a survey ostensibly intended to shape policy was in breach of a basic legal requirement, as it failed to specify the purpose for which some of the data was to be used. The survey asked respondents to rate the importance of a number of issues facing the country, but also required them to provide their email addresses and postcodes – with a view to building a contact database, the MRS alleges – without informing them of this secondary aim.

MRS has threatened to report the party to the Information Commissioner’s Office (ICO) and could face a fine if the ICO agrees that it has built a contact database without properly informing those who completed the questionnaire. A Conservative spokesperson, quoted in the Independent, said in response to the allegations: “We are extremely surprised that this letter was sent without any prior personal contact with the party by Ms Frost or MRS Standards Team and now finds its way into the media. Especially when it makes unfounded and baseless accusations regarding ethics.”

Both examples are a worrying sign of a lack of understanding of basic research techniques. The White House either doesn’t understand the value of context, or isn’t interested in it; the Conservative Party appear to be using research as a thinly-veiled way of gathering supporters’ contact details. Both examples are at odds with the value that each group claims to place on the opinions and privacy of its people.

1 Comment

10 years ago

I'm torn on this; in Canada the MRIA has recently done the same with 2 opinion polls that were similarly challenged in their design. The issue is that standing in the square shouting "this research is WRONG, you're WRONG, stop doing it WRONG" isn't really the best answer. Frankly, it can make the body (MRA, MRIA etc) look incredibly out of touch and irrelevant, focused on this idea that the "right" way to do research is an absolute, completely independent of context, the needs of the output and so on. (NB, data protection is IMO a different, very serious, issue) Ultimately research is being done badly in many different places. The way to be consulted, brought into the fold, and become more influential is to take a realistic view of this - be part of the solution, not just the guy shouting "you're all WRONG, so there".

Like Report