Data interaction in 3D

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

Virtual reality (VR) went mainstream in 2016: Oculus Rift – the Facebook-owned system – arrived on the UK high street in September (it had been available in the US and online earlier in the year), as did rivals HTC Vive and Sony’s PlayStation VR. 

As VR’s reach extends far beyond its gaming roots, Matt Ratcliffe, co-founder of Masters of Pie – a creative software development studio that works with virtual and augmented reality (AR) technologies – believes it could help extract more value from data.

Last year, Ratcliffe and his team won the EPIC Games and Wellcome Trust Big Data and VR Challenge. This was designed to find games companies – or people who worked with game technology – that could help solve the problem of navigating and finding connections, trends and solutions among the huge data sets being gathered every minute by industries around the world.

Masters of Pie, alongside 3D software development consultancy Lumacode, worked with researchers at the Avon Longitudinal Study of Parents and Children (Alspac) on data from its ‘Children of the 90s’ project. The study is following 14,000 children born in the 90s, as well as their parents and their own children, until they reach the age of 70. Recorded data includes information on diet, lifestyle, socio-economic status, attitudes and parent-child contact, as well as samples of urine, blood and DNA.

The sheer amount of data, and the number of dimensions within the study meant the team at Alspac had struggled to find value in it, Ratcliffe explains. So Masters of Pie created a fully functional, scalable, VR visualisation of the study data: an immersive environment built from the data itself. Within that virtual space, users have full control and can intuitively interact with, and manipulate, data to aid research. 

The filtered information – putting raw data into the system with no modelling would be too overwhelming, explains Ratcliffe – is arranged like DNA strands or rings, to avoid issues with perspective interfering with data comparison. With data nodes arranged in a ring formation, a user can stand in the middle and compare, like-for-like. 

Work is still ongoing with the Alspac data and, in the meantime, Masters of Pie is working with a number of other clients on similar projects.

“Over the past three years, we have slowly been figuring out that VR and AR isn’t just some fanciful new piece of tech, like 3D TV,” says Ratcliffe. “It isn’t just something that’s going to expire and be replaced by something else.”

Masters of Pie’s focus now, says Ratcliffe, is to use this approach to “reduce the abstraction between human and computer or digital content”. 

“The whole point of VR is that you’re putting data into a 3D space; it’s no longer compressed and abstracted onto a 2D screen,” he says. 

“For me, that’s the paradigm shift – it’s what’s changing in terms of that interaction. It’s a lot bigger than just games and film – it’s a total shift in the way that people will interact with digital content in the next – probably – 50 years.”

The team is now deep diving into data analysis. Some already work with machine learning and data analytics, and are looking into different ways of visualising data, and whether new meaning can be gleaned from a data set when it’s put into a VR space.

An immediate ‘win’ with using VR in this way is that you can fit a lot more data in one space than is possible with a screen, Ratcliffe explains. 

Other features are that you can use your head to look around, rather than scrolling with a mouse. It’s also possible to grab data and spin it, rotate it, and use 3D axes as a way of comparing data from different views, as well as filter and cut the data. 

“And it allows users to grab data and pass it to someone else – even if they are not in the same room, or even the same country.

“There’s a whole area around collaboration – pairing people from different disciplines or with different skill levels,” says Ratcliffe. 

“The idea is to be able to pick up the data, hand it to someone else, maybe cut a part of it away because it looks interesting, and tell them to check it out.”

Ratcliffe believes that this way of interacting with information could make data and analysis more accessible to the layperson – which, in turn, has the potential to make research more accessible as a discipline. 

“There’s certainly potential there for allowing people who don’t have data science degrees to be able to access and look at data to get some kind of idea of trends,” he says. 

“It is a much more gestural and physical way of interacting; you don’t have to learn complex software systems and interfaces if you do it right. 

“A researcher using VR can say: ‘This is why this is valuable and why you should take notice of this trend I found in your data’. Trying to communicate that through current printouts can be incredibly confusing, and boring, if you’re not into that kind of thing. 

“You could absolutely walk a CEO or a stakeholder through and point things out. They don’t even have to be in the same country as you.”