NEWS24 November 2015

The Sun’s pollsters criticised for methodology used in controversial survey story

News UK

UK — Polling company Survation has been criticised for the sampling approach of filtering its database against ‘1,500 Muslim surnames’ for the survey informing yesterday’s controversial front-page jihadi sympathy story.

Res_4014205_the_sun

The story, headlined “1 in 5 Brit Muslims’ sympathy for jihadis” was based on the poll’s finding that 19.5% of those surveyed said they had either ‘a lot’ or ‘some’ sympathy with young Muslims who leave the UK to join fighters in Syria.

However, according to a story on The Guardian news site, the company that carried out the survey — Survation — used a method claimed by rivals to not necessarily result in a representative sample of the British Muslim population. The method involved picking out potential respondents with the help of an academic expert on naming – it filtered its database of 42 million profiles against a list of 1,500 Muslim surnames, and then asked whether the people that this method had identified were Muslim or not before proceeding with the survey.

According to the story, The Sun’s regular pollsters YouGov refused to do the poll as it couldn’t be confident of accurately representing the British Muslim population within the timeframe and budget set by the paper. The representativeness of the sample apparently cannot be determined due to a lack of socioeconomic and demographic details on the respondents.

There has also been criticism of the wording of the statement used in the survey, with some claiming that the word “sympathy” was open to misinterpretation, and also that who is meant by “fighters in Syria” was not made clear.

@RESEARCH LIVE

5 Comments

8 years ago

The question is highly ambiguous on both fronts: "sympathy" and "fighters in Syria". Which means the ensuing data are open to interpretation and the Sun duly obliged. The lack of understanding of the civil war in Syria that "fighters in Syria" demonstrates would be laughable if the potential consequences of this inflammatory piece weren't so serious.

Like Report

8 years ago | 1 like

Basing a sample of British Muslims on a list of "Muslim surnames" (whatever that means) is like sampling the Scottish population by interviewing only those surnamed Mac something. Quite ridiculous, and Survation should be investigated by the Market Research Society for such appallingly poor quality work.

Like Report

8 years ago

Survation is rightly being criticised for dreadful errors in sampling and question wording, and this will once again bring all polling and hence research into disrepute with the general public. I think they need to issue a further statement on how this happened. Personally I also think that they were very naive in not predicting how this was going to be used by the ,,newspaper'' in question. The affiliated Fox News is famous for this kind of ...well I would call it lying.

Like Report

8 years ago

There is a statement on their website covering this, and to be fair they categorically disagree with the papers headline and were not consulted on the headline or interpretation of the findings before publication. That said, their question wording, despite their defense of it, is very poor in that it leaves so much open to interpretation - it doesn't actually answer any single question. They were hugely naive to enter into this piece of work for what is known to be a media publication that revels in sensationalist headlines which are regularly inaccurate. I think the MR industry really needs to review how it works with tabloids.

Like Report

8 years ago

Survation's statement says: 'neither the headline nor the body text of articles were discussed with or approved by Survation prior to publication'. Of course they weren't. No newspaper worth its salt will allow anyone, not even a Government minister, to approve or amend editorial content. Quite right too. It's the very essence of a free and independent Press. It's naive of Survation to imply that allowing them to approve the piece might have been an option and it's therefore no excuse that they weren't.

Like Report