Seeing different realities: Making research accessible

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

Since 2023, PA Consulting has worked with the virtual reality (VR) team at Meta to research the experiences, needs and behaviours of people with disability, primarily to inform Meta’s VR offer, as well as future immersive technologies.

PA Consulting and Meta shared their approach at the MRS Equality Summit in May 2024.

Speaking at the conference, Lauren Adrover, senior UX researcher, Reality Labs and accessibility and equity lead, Meta, said: “As we build this immersive future of computing, we are acutely sensitive to wanting to do it with everyone.”

Elise McNamara, senior consultant at PA Consulting, said: “Working towards best practice is a continuous journey. We made a lot of mistakes along the way, but that helped us unearth some of our most valuable learnings. Progress requires compromise. When it comes to accessibility research, of course we all want to aim high. But there are quite a lot of practical barriers that come with running inclusive research.”

The first part of the study was at-home ethnographic research with people living with disability, to explore the barriers faced while using VR. A competitor review examined the benchmark for accessibility features within the market, to understand where Meta’s technology was at risk of falling behind. Lastly, the researchers ran a quant study to understand people’s behaviours around accessibility features – both those living with disability and the wider population.

“The risk of excluding people with disability isn’t a hypothetical concern – we’re already seeing this happen with other fast-moving technologies such as AI,” said Phoebe Canning, associate partner at PA Consulting.

Adrover said: “The bar for our research was especially high because we appreciate the enormous value that immersive tech brings to people with disabilities.” For example, people with disabilities could use VR to feel less isolated or gain a greater sense of agency by participating in a virtual world.

Addressing assumptions

The researchers first addressed what they called their ‘accessibility research orthodoxies’ – common, ingrained assumptions that risked hampering an inclusive approach. These were:

1. People living with disabilities would be extremely hard to find and recruit
   
2. Only specialist moderators could run accessibility research
   
3. People living with significant disability wouldn’t be able to fully complete a quant survey. 

The team then designed its research with the aim of subverting these assumptions.

1. Recruitment

The assumption that it would be hard to recruit for this research was “not totally unfounded”, said McNamara. The target audience for the qualitative stage was people who are frequently using VR, living with a range of disabilities, who were also customising their devices to cater for their needs.

The researchers partnered with specialist disability panels and charities, but one of the most successful avenues for recruitment proved to be online: the team worked with disability influencers who use their platform to talk about accessible gaming, and found the majority of respondents in this way.

2. Moderation

The second ‘orthodoxy’ was the belief that a specialist moderator would be needed. To address this, the researchers educated themselves in accessible language and understanding the power dynamics in a shared space with participants. “We also took our cues from participants themselves,” said Canning, “mirroring the language that they use.”

For one participant, this meant not using the word ‘disability’ as that was not how the individual identified.

In a post-fieldwork questionnaire, some participants said they would have found it helpful to have more of an upfront discussion to prevent anxiety ahead of the session – an insight PA Consulting took into the quant stage.

3. Survey design

The researchers co-created the survey with people with lived experience and experts in the space – consulting on issues such as format, flow, usability and language. Changes included replacing sliding scales with radio buttons to help with cognitive processing, and AI-generated voice-overs to avoid participants reading lots of lengthy questions.

To ensure anyone who wanted to participate in the research could do so, the researchers used dual participation. They also worked with a CATI agency to help field the research for those unable to use the online survey.

One area that proved particularly challenging was making the survey screen-reader accessible. “Although we had 100 people with total sight loss complete the survey, we needed to think about the user experience,” said McNamara. “There is a massive difference between making something screen-reader compatible, versus accessible.”

Findings

Prioritisation of themes wasn’t simple, as everyone’s experience of disability – and therefore the barriers experienced – is unique. One of PA’s approaches to distilling recommendations was assessing the prevalence of the need, and the size of the need, for a certain assistive solution.

Meta’s Adrover said: “We were thinking about how many people we can impact. We were taken aback by the intersectionality of disability; how many different groups need the same solution in their technological behaviours. Live captions are clearly of use for people with hearing loss, but we also found that people with cognitive difference really appreciated live captions for lowering cognitive load.”

Meta also held a co-design session with people who are blind or have low vision to inform creative, multi-sensory haptic solutions for immersive VR products. Another important factor was finding ‘curb cuts’ – solutions that also benefit the wider population. “Being able to identify where this overlap lies helps the team to build a commercial case for change internally,” said Canning.

Researchers have a key role in amplifying the voices of those living with disability, McNamara said, adding: “We need to continue to share our best practice, share our learnings and challenge our assumptions to build inclusivity one step at a time.”

This article was first published in the July 2024 issue of Impact.