OPINION23 April 2010

Why the thirst to be first can leave you high and dry

Features

YouGov appeared as a trending topic on Twitter last night as the Twitterati took up arms against the online pollster after its instant reaction poll for the second leaders’ debate – the first to be published – reported a win for David Cameron. This, it seems, was the wrong result.

YouGov appeared as a trending topic on Twitter last night as the Twitterati took up arms against the online pollster after its instant reaction poll for the second leaders’ debate – the first to be published – reported a win for David Cameron.

This, it seems, was the wrong result. For starters, other pollsters had Nick Clegg as the winner – ICM, Angus Reid and ComRes. Nevermind that Populus also had Cameron as victor.

But what about YouGov’s ties to the Tories? Twitter posts seized on the fact that agency co-founder Nadhim Zahawi is standing as a Conservative candidate in Stratford-on-Avon. Many said he is still the company’s CEO, though this is not true (he stood down when the election was called).

Interestingly those Conservative links didn’t stop Conservative politicians from accusing the firm earlier this month (via the Daily Telegraph) of being biased towards Labour in its polls. But there you go. The internet still resounded to cries of ‘fix’ and ‘scandal’.

I struggle to see why anyone would think YouGov – or any other reputable pollster for that matter – would deliberately fix results. These agencies secure plenty of coverage, but very little in the way of revenue, for their media polls. It makes spectacularly unsound business sense to risk millions in sales for the sake of a few thousand pounds from News International, Associated Newspapers or any other media owner.

Polling results can easily be wrong when trying to be right – why add the risk of faking the figures to appease a newspaper’s political agenda?

The quality of a pollster’s output should be judged on their methodology. So one point that deserves following up among the Twitter outcry is the claim that YouGov’s survey got under way before the debate had actually finished.

We put this to YouGov’s Peter Kellner, who admitted that yes, the poll did open up while the party leaders were delivering their closing remarks. This, he said, was to ensure that the weblink would be working correctly as soon as the debate ended.

Kellner said a “handful” of people completed the survey before the debate went off air – though he wouldn’t say exactly how many. “There will be some people who had nothing better to do than click the link until the survey opened up,” he said.

The question is, what effect did these people have on the overall result of YouGov’s poll? Kellner is adamant they didn’t have any effect. He said the agency compared results from the first 1,000 people who took part in the poll within the first few minutes of it going live with results from a second batch of 1,000 people who took part in the minutes that followed (but weren’t part of the published poll) and that no discrepancies were found.

Be that as it may, as one senior polling figure put it to me, “It seems daft that YouGov would risk undermining their results for the sake of being first. In order to be beyond reproach you start fieldwork once the last politician has summed up and sat down, and you finish the fieldwork before there is this storm of media analysis.”

In this election, more than any in recent memory, it is incumbent on all polling organisations to make sure their behaviour and their data is beyond reproach. The political landscape is volatile with the prospect of a hung parliament. Producing accurate polls in this environment will be tricky.

Speaking on Wednesday evening at an Esomar debate, GfK NOP chairman Phyllis Macfarlane warned: “I am worried about the polls. I feel a 1992 coming on” – referring to the infamous failure of the polls to forecast a Tory victory, the last before Labour came to power.

“Those of you who have polls in your organisation, please do not disgrace us,” she said. “We need a good result.”

4 Comments

14 years ago

The comments from Phyllis Macfarlane are very interesting, but a "feeling" is not good enough, what is it that makes Ms Macfarlane think that the polls are wrong?

Like Report

14 years ago

I have the original Yougov press release in front of me from the night. It says quite clearly "Fieldwork was conducted between 21.27 and 21.31 on 22nd April; sample size: 1,110 viewers of the debate." According to Michael Crick's blog, the last speaker, Clegg, began his upsum at 9:29:18 and finished at 9:30:47??. So I infer that most people voted while Clegg was still talking.

Like Report

14 years ago

I took part in the YouGov poll after the 2nd leaders debate. The instructions were that the poll would be open for only 10 minutes and that it would open at 9.25pm. I voted in the final 5 minutes after the debate ended.

Like Report

14 years ago

@Anonymous I have asked Peter Kellner about the stated timings of the poll. He said they were noted inaccurately on the press release you refer to and and were some two minutes out. The correct times should have been 21.29 and 21.33, he said. This would mean Clegg was talking when the poll opened, but again he said its was only a handful of people that completed the survey before the closing remarks had ended - though I still haven't been able to get any numbers on exactly how many. @Caitlin Would be very interested to see a copy of those instructions. Do you have them in email form you could send me? Thanks in advance.

Like Report