Past lessons for today’s researchers

Although the research industry is very different from the way it was in the past, the basic ground rules for the execution of research projects remain much the same. The easy and cheap accumulation of data creates a temptation to overlook such matters, but they should not be brushed aside and we can learn from the mishaps occurring in the past.

Although some of them may seem obvious, the lessons to be derived from the recollections of colleagues with experience across the last 50 years are instructive in helping current practitioners to avoid the pitfalls in this business.

In the working environment

90% of what you will ever need to know technically can be learned in your first two years, especially if you are guided by a senior colleague able to show you how things are done and how they can go horribly wrong.

It can be a good idea to check that you are suited to your current job: according to an HR lecturer at international business school Insead, to be happy you need to…

• Be good at what you do, otherwise you are a competence misfit
• Enjoy what it involves, otherwise you are an enjoyment misfit
• Believe in what you are doing, otherwise you are a moral misfit.

In matters technical

Weighting to correct and compensate for very low response rates rarely provides for full representative reliability. In the 1970s most quant surveys among all adults achieved 75% response rates, and the cheapness of current data collection methods with response rates well below 10% does not easily compensate for that.

Questionnaire design is a key skill and too many surveys nowadays are not preceded by pilot testing. By doing that, the research executive is taken out of his/her office-bound mindset and confronted by the realities of respondent non-comprehension.

Data analysis: Named after the director of a leading research company from earlier years, all researchers should still pay heed to Twyman’s Law, which says that “The more surprising a finding from any survey is, the more likely that it is the result of some error in sampling, questioning, analysis or interpretation.”

Presenting results: Researchers across the industry are so much better at this than they used to be. But even so, how many of us can remember occasions when the presenter turned up late, could not operate the presentation equipment, talked for far too long, showed charts which were illegible or too complicated, or the senior client present fell asleep, picked a fight or walked out before the end?

Some other problems to avoid

• If you are doing a taste test on Christmas chocolates in a busy marketplace in midsummer, be sure you have a way of keeping them cool if you don’t want everyone seeing them melt before you begin
• If you are conducting taste-testing research about beer, make sure you do not dispense samples of your client’s product too freely: on one occasion the word got around locally, a queue formed of people wanting to take part, and a raucous party got going with the moderator unable to restore order
• As a group of young men arrived in a hotel room for a focus group session, they signed in and were given a drink and the money they had been promised for attending. As the discussion got going, one after another asked to go to the loo. But they never returned, and the convenor ended up with a near-empty room. Lesson learned: give them their money after completion of the session.

International perspectives

There are many stories of British, and even more so American, researchers commissioning or conducting studies in faraway countries without sufficient local knowledge:

• One British researcher conducting a focus group to test food products with a group of softly-spoken men in Africa, suddenly found they all got up and left the room. She wondered what she had done wrong, but ten minutes later they all reappeared, apologising and saying they had to go and pray. So she learned for the future to choose her time of day carefully.
• One American client, unused to research practices, took delivery of a set of survey results delivered by a British research company. At the foot of each page, he found a line marked ‘DK/CR’ and said: “It’s on every damned page! What does that mean?” “Don’t know, can’t remember” came the reply, to which he retorted: “Well, if you don’t know, who the hell does?”

These anecdotes and more are to be found in The Life in Research, collected and edited by Peter Bartram. It can be bought from leading book retailers, with proceeds donated to the Archive of Market and Social Research. The e-book Post War Developments in Market Research is also available via the Archive.