NEWS28 February 2011

MRIA disputes ‘crisis of confidence’ in polling industry

North America

CANADA— The Marketing Research & Intelligence Association (MRIA) has taken out a full-page ad in the political weekly The Hill Times to tackle recent criticism of opinions polls as unreliable, inaccurate and far too prolific.

Headlined ‘There’s no margin of error on the truth’, the ad hits out at media articles that suggest a “crisis of confidence” in the industry and its abilities to accurately measure the opinions of Canadians.

At the root of the issue is a Canadian Press article, dated 13 February, in which pollsters including Harris-Decima’s Allan Gregg, Jaideep Mukerji of Angus Reid Public Opinions and Frank Graves of Ekos are quoted lamenting a proliferation of polls, a decline in standards and excessive competition that’s forcing prices lower and lower.

Other contributors to the piece accuse the media of being addicted to polls and criticise pollsters for feeding the habit.

The MRIA challenges the assumption that too many polls are a bad thing, arguing that “the more public polls are published in a democracy, the more its citizens become aware of the issues of the day”. Bad polls, the association says, are soon discredited by those conducted by reputable firms.

On the question of accuracy, the MRIA points to the industry’s success in calling the previous two elections. A PDF of the ad is available here.

Darrell Bricker and John Wright of Ipsos Reid also sought to answer the criticisms levelled at the industry by their colleagues in an open letter to the media.

“One misstatement,” they write, “is that political polls in Canada aren’t based on ‘true’ random samples so they are prone to error (supposedly, this is our ‘dirty little secret’). This statement is completely ridiculous. Nobody in the world does true random samples for political research. And nobody has ever conducted a random sample for a political survey in Canada, ever. True random samples take too long, are too expensive, and are overkill for the task at hand.”

Brian Tarran

North America

@RESEARCH LIVE

1 Comment

Anon

13 years ago

The main objective of survey sampling is to be substitute for expensive censuses and therefore there is always a need for clearly defined finite population. If members of general population do not have a chance of being selected in the sample i.e. they have zero probability of being selected or selection probabilities are not known or it is not possible to calculate these probabilities then how is it possible to draw inferences from the sample to the population of interest? How any pollster who use convenience sample can establish how many members of general population each selected person in the sample represents? Can any non-probability pollster who claim that sample is “broadly representative” of general population describe what scientific theory they use to justify calculation of margins of errors from such samples? My guess is that they apply theory that is linked to probability sampling even though they know this theory cannot be applied to convenience samples. If central Government commission the survey they do not have an alternative but to design and select probability sample. They have to make sure that every member of the population has a chance of being selected in the sample so that findings from such surveys can be generalised. This is the most democratic way of selecting the members of general population. Even if the response rate for this type of surveys is around 30% or 40% it does not mean that any convenience sample can be used instead. Still inferences can be made to 40% of the population who was willing to participate in the survey and this will allow inferences to 40% of the general population. Convenience sample of 1000 only allow inferences to itself. It is a common sense to think that convenience sample or any other type of non-probability samples do not allow generalisation of any findings to a larger population. A word “sample” is misused by non-probablity pollsters and by media since many of them think that if sample is selected and study conducted by a research organisation that finding represents the truth and general statements about the larger population can be made. If research organisation have a self-selected panel of, for example, 400,000 members, even if you would interview all of them it still would not allow research organisation or media to make a general statements about larger population. Probability samples are expensive and require more time to conduct but they have their advantages (allow inferences to the larger population). Advantages of convenience samples: cheap and fast (inferential population is always equal to the sample size of convenience sample).

Like Report