It’s the taking part that counts: Improving the participant experience

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

There is increasing anecdotal evidence that many people who participate in market research find the process tiresome, and they can feel confused or misled when it comes to knowing what is expected of them. Engagement levels among those asked to take part has, arguably, hit an all-time low, according to some in the industry.

This trend should concern everyone; participants are the foundation of the research sector, but they can be overlooked and poorly paid. Issues such as poor survey design and time-intensive screeners are leading to an unsatisfactory overall experience.

There is a now a strong feeling within the industry that, unless the problem is addressed, the pool of willing respondents will shrink, and the quality of the research clients are paying for could decline significantly. The bottom line, say those concerned, is that the industry needs to pay more attention to the needs of participants.

While face-to-face research has work to do in this area, the biggest problem seems to be with online quantitative surveys. When it comes to digital surveys, the brief is often too loose – or non-existent – and less effort is made to find the best people to take part.

Compare this with in-person research, where people are invited to focus groups and have already bought into the idea of being involved before they arrive. They know, for example, that they need to be at a venue at a certain time.

One company determined to improve participant experience is Schlesinger Group. Internally, it is now using the term PX – in line with other marketing terms, such as UX for user experience and CX for customer experience – to ensure participant experience is front of mind.

Isaac Rogers, chief innovation officer at Schlesinger, wants everyone to stop taking participants for granted. He points out that, on the qualitative side of research, the incentives required to get people to take part have risen faster than the rate of inflation in recent years. This has been necessary to retain sample sizes and audiences, and to keep up with client demand. He fears things could become even worse for online research over the next few years.

“There are clients who think ‘well, we have 50 million customers, so it cannot be that difficult to find people to research’, but that’s not true. We need to worry more about the quality of the participants and the experience they receive,” Rogers says. “This is certainly the case as we move more to digital, where we really need people to be engaged in the research ahead of the survey taking place.”

He urges the industry to appreciate the effort participants are putting in. Research can involve online communities that last a few days or even a few months, and the research can be very intrusive and invasive into how someone lives their life.

“It can be easy to assume that people will be willing to participate, but the problem today is that we get respondents who are not well briefed on what we’re asking them to do,” Rogers says.

This is not only about assuming the pool of respondents will always be there, but also because of poor recruitment.

“In so many ways, we shuffle respondents to our surveys or screeners without really telling them anything about ‘why’,” Rogers adds. “They are being blindly led into the research, unsure of what they’re really signing up for, and this causes a lot of friction in the process and leads to higher attrition rates. Taking the time to explain the research process, why their voice is so important, and what we need them to do to complete the research, is vital.

“I look at it as similar to oil, or the rainforests. They have been there for so long that we think the supply is limitless. You can always get oil – and you can always get people to take part in research, right? Then you start to do the math and you see how things are going to run out soon, and you need to do something.”

Ultimately, if the experience is poor, many more respondents will push back and leave the process. “They will say, ‘look, you told me you needed me to do this online research, but when I get into it, it will take more time than I expected, or is more complicated, and you didn’t prepare me’,” Rogers says.

“They may be being asked to answer questions, take photos and shoot videos as part of a five-day community, and we are paying them just £50 for maybe 12 hours’ work.”

Simply offering higher rates of incentivisation alone may not be the answer, however. “We’ve actually found you can pay respondents less if you do a better job of communicating and telling them the reason why research is so important.”

Rogers cites a study Schlesinger conducted with respondents a couple of years ago, to explore their motivations to participate and what they find appealing about the research process. It found that some audiences were willing to accept lower remuneration – sometimes 20% lower or more – if they were simply told more about the research needs and how their input would impact products and services they use in their daily lives.

“I think, in a lot of ways, we bump up incentives to cover up the fact that we don’t do as good a job communicating this impact to respondents,” he adds.

Rogers recalls one occasion when the amount of effort needed on behalf of the participants was undersold, and changes had to be made to improve the experience for all parties. It was for a piece of research around smoking cessation and smoking habits, for which people were asked to keep a diary, participate in a group discussion the following week, and prepare for possible follow-up video interviews the week after.

“Looking back, we didn’t do a good enough job laying out to the respondents what all the steps involved. As people got further into the research, they became less engaged. In the end, we were begging people to show up for the video interviews. When we look back, we can see how we did a poor job setting up these people to be successful participants.”

In response, the company changed its participant experience strategy for the second phase of the research, leading to a big improvement in engagement. All the changes were around communication: the company told respondents exactly what was important, with clear instructions. These were repeated throughout the survey so respondents understood the value of the data they were providing.

“Rather than chasing people down, we were getting people contacting us, asking if they had been selected for the video interviews, because they needed to plan their week,” explains Rogers. “They were being treated better and were, therefore, more involved in the process.”

He believes this shows that, by investing more time in participant experience when designing research – and improving communication – engagement levels can be improved without, necessarily, having to increase the financial compensation.

Long-standing challenge

Jon Puleston, vice-president of innovation at Kantar, said the trend of people being reluctant to take part in or complete surveys has been growing for some time. He has seen improvements around mobile surveys in recent years, but puts some of the blame for the current problems on the commoditisation of market research.

“People tell us they find surveys boring and the questions dull,” he says. “As we have started to understand the data around people who take part, we see how respondents are becoming less and less attentive. It means the data clients are getting back can be less reliable.”

Puleston agrees there is a problem with the way surveys are designed, with repetition a big issue. He also points out how the relationship between the length of a survey and people’s attention span is not as straightforward as some might think. Someone’s level of concentration will often come down to how relevant and interesting the questions are.

“It is like going to the cinema, where you will quite happily watch a three-hour film if it is engaging – but, if it is a boring movie, you might walk out after half an hour. We have to make research more consumer-friendly. This includes removing jargon and being less judgemental in how we ask questions.”

The say-do gap is an ever-present issue for researchers, but Puleston says the industry must work harder to make people feel more comfortable about telling the truth, and not feel obliged to answer questions in certain ways because they are embarrassed by their own behaviour. This means being more empathetic when designing surveys and improving communication with those taking part. He cites the example of a project his team did with Mintel to evaluate consumer attitudes and brand awareness of new products.

Mintel identifies and catalogues around 18,000 new food and drink products in the United States each year for its global new products database. The team decided to add consumer evaluation research, and wanted to measure reactions to each of these new products as they were released, and give their global client base live access to the insight.

Mintel initially ran the research in the US before expanding it into Australia. The company is currently using it to evaluate more than 2,000 products a month across the two markets.

Consumer testing 18,000 products a year was an ambitious idea; it would mean fielding 400 surveys a week. If 100 people were to evaluate each product, it would require two million completes a year – an impossibly large number. The Kantar QuestionArts team had to find a way to conduct the evaluations more efficiently, and design a survey that participants, once they had completed it, would want to do again.

Respondents were struggling to concentrate because the work involved evaluating thousands of products, taking photographs, and completing a lengthy survey.

The first task was to slash the time taken to complete a typical new product evaluation survey from more than 10 minutes to just two.

All the remaining detailed questions normally found in a new product development survey were replaced with one simple, open-ended question, asking people to explain what they like or dislike about each product. A text analytics protocol was developed to convert this feedback into closed metrics, by analysing more than 150,000 of the comments, isolating the recurring words and phrases, and classifying them into categories.

Kantar also came up with a gamification element, with participants winning points by guessing if they felt a product would be successful or not, and if their view matched that of the market. The survey was also tightened up and made less repetitive, with the number of questions relating to how someone evaluated a brand cut significantly to reduce boredom levels.

In head-to-head tests, Kantar discovered that people completing the redesigned, ‘gamified’ version of the survey evaluated 25% more products voluntarily. The rating they gave to the survey increased from 7.8 to 8.9 (out of 10 ), putting it in the top 2% of surveys it has ever fielded. It also encouraged people to write 20% more open-ended feedback per product.

There were also noticeably higher concentration levels from participants, and feedback quality increased by 40%. Mintel now requires fewer people to deliver reliable, stable responses.

Puleston cites another example, where a bank client had a survey that was only about 12 minutes long, but people were not engaged. The researcher’s answer was to break up the different elements of the survey and interject some narrative into the process, so people understood why they were being asked the questions and the value of them. The questioning turned into more of a discussion around their experience of going to the bank, in terms of customer service and queuing, for example.

Puleston says a game mechanic is a way to re-engineer simple questions so they are more fun. Rather than simply asking someone to name their favourite meal, for example, a game is created by asking: ‘Imagine you are on death row and you have to choose your last meal?’

Adding a game element encourages participants to answer in more detail, rather than just a few words – such as ‘steak and chips’ – generating richer data.

Another example is giving people an exact word count when answering a question. Rather than asking someone to describe themselves, the question is made more fun: “In exactly seven words, how would your friends describe you?”

Keeping the numbers up

When it comes to online quantitative research, there is evidence that the supply of participants is not keeping up with demand. According to Puleston, it’s becoming harder and harder to reach people under 25 years old, for example, particularly young men.

This is not just because of poor participant experience. During the pandemic, many market research studies were put on hold and there are plenty of instances where investment in people, as well as technology, has stalled.

Consumers have also spent more time online during the various lockdowns, and the competition to reach them has intensified. The pandemic has also affected potential survey respondents in different ways on a personal level. People have had time to reflect, reassess, and reprioritise; some now feel less inclined to make long-term commitments to research projects.

Talking to Mathilde Lelièvre, chief operations officer at Toluna Corporate, and the company’s senior vice-president of global panels and supply, Paul Hambly, it is clear that there is a need, across the industry, to increase investment in proprietary panels.

“I don’t think the industry, at the client or supplier end, is intentionally neglecting the experience of participants,” says Lelièvre. “But we do know the experience affects engagement levels and the quality of responses.

“The bottom line is that, even if everyone is conscious of the importance of providing a good experience, respondents often end up with a discouraging one.”

She wants the industry to challenge the continuous push for lower cost per interview, because this has a direct impact on the rewards suppliers can offer their members. This will require a change in mindset, where respondents are viewed as more than data-generating commodities.

Any brand that wishes to thrive must deliver personalised and relevant participant experience when it comes to research, says Hambly.

“The importance of profiling comes into play,” he adds. “This enables us to contact respondents at the right time, with the right message, and through the right channel. This ensures a yield of higher-quality insights.”

However, participant experience isn’t merely a problem for online surveys. When it comes to qualitative research, it is also important to ensure that those taking part are fully engaged. After all, researchers like to feel that they are forging relationships with participants, and there are ethical obligations to take the experience of participants into account when planning and budgeting.

With in-person research, there are a number of reasons why it is important to give people a good experience. Not only are the outcomes often linked to how engaged someone feels with the research process, but it helps researchers to understand a participant’s motivations for taking part.

Younger people, for instance, can put more value on their time and be less willing to trade hours for an incentive.

Another factor when it comes to good experience is ensuring the findings are shared with those who took part. If people feel they have some ownership, that will foster trust with the researchers. This could also reduce recruitment time and costs in future when hiring participants who are a good fit. It would also reduce the risk of a study being delayed because of no-shows.

Data privacy

According to Toluna’s Hambly, many respondents – whether online or face to face – have increasing concerns about privacy, especially when they are asked for personally identifying information (PII).

This is also having an impact on the participant experience. “Respondents have a keen understanding of data privacy these days and the value of the insights they share. Accordingly, they are less willing to share it with third parties than in the past,” he says.

“For surveys that do ask for PII, it is important to ensure the information is relevant to the research. It needs to be communicated clearly to respondents why they are being asked, and a guarantee given that it won’t be used for other objectives.”

The concern around data privacy and transparency can certainly be a barrier, especially if people feel the compensation they are receiving for sharing their data is not a reasonable value exchange.

Armen Adjemian, chief executive of consumer insights platform Disqo, is a firm believer that people should be rewarded fairly for sharing details about themselves with brands.

“Consumers are catching on to this idea of being the owners of their own data,” he says. “This will fuel a zero-party data revolution, where consumers opt in to share information about themselves instead of being tracked persistently with somewhat questionable consent.

“As the insights industry continues to evolve, the winning players will enable a fair and sustainable exchange of value between brands and consumers, in a way that keeps people coming back for more.”

Companies must agree that creating value for people who share data is equally as important as ensuring value to enterprise clients, adds Adjemian. “It is critical to create an experience that is enjoyable and engaging, and that offers the least amount of friction as possible.”

Ethical stance

There is also a view that the industry needs a stronger ethical stance in how it treats participants.

Sabrina Trinquetel, UK sales director at Measure Protocol, wants researchers and clients to remember that respondents are real people and not just numbers. In fact, they are often the same people that brands are targeting and whose data they are using in the first place.

“It is not all about increasing the financial reward, but more about people feeling valued, because they want to help,” she says. “The research might be for a brand they believe in or a cause they are passionate about. It’s also about not wasting people’s time.”

Trinquetel agrees that people want greater control over their data, which some fear could be misused. A study by Measure Protocol discovered that almost half of respondents are concerned about privacy and trust when completing an online survey.

The long-term impact of not improving participant experience could be serious. It is why the industry needs to show more respect to respondents for the time they are giving up, and ensure research is interesting, inclusive and engaging. If it doesn’t do this, clients might start to view the insight they receive as being of limited use and not representative of what is actually happening in the world.