PPM panellists admit to ‘gaming the system’

Broadcast Architecture, which provides research and programming services to radio stations, conducted face-to-face interviews with 13 PPM panellists from seven households.

The firm said many of the panellists it interviewed saw the points system as a “game or competition”, and most were focused on getting more points to earn more money – either by trying to move around as much as possible in an attempt to get credit for carrying the device around, or by carrying other family members’ PPMs as well as their own when going out. Not surprisingly, most of the panellists said they were motivated primarily by money to take part.

Arbitron responded today saying the study was flawed and not a valid representation of PPM panellist behaviour.

The PPM – a small device that panellists carry with them to automatically track the radio broadcasts they hear during their day – is being introduced by Arbitron across the US to replace the diary-based system of counting audiences. It has proved controversial because of difficulties recruiting and managing panellists, and in particular because of its effects on the ratings of stations targeting ethnic minorities.

Some of the panellists interviewed by Broadcast Architecture complained that the PPM device is too big and “looks like a pager”.

The firm said the research was only the initial phase of what would be an ongoing study into the behaviour and motivations of PPM panel members.

Broadcast Architecture’s president Allen Kepler said: “These folks hold so much of our fate in their hands, literally, and we felt it was time to seek them out and see what’s actually going on at ground level… Some of you may find the results surprising. Others may be shocked at some of the statements and behaviours.”

Responding to the news, Arbitron’s vice president of research methods and quality Beth Webb told Research: “This so called ‘study’ of seven households relies solely on panellists who are willing to breach their confidentiality agreement with Arbitron. It suffers from all the limitations that makes opt-in research studies invalid and cannot be relied on to be a valid profile of the hundreds of thousands of persons who have been PPM respondents.

“There is the opportunity for inauthentic responses in all surveys. We also have methods to detect inauthentic behaviour and either exclude the suspicious data or remove the suspicious respondent… For obvious reasons, we don’t discuss these procedures in detail but we do exclude days and we do deinstall households when such behaviours are detected.”

Webb also pointed out that the panellists who were interviewed did not seem to fully understand how the PPM recorded listening information, so their estimates about its accuracy were misleading.