Testing teams

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

I’ve written previously about the vast scale of experimentation that happens across companies in the tech sector but it’s important to note that those experiments extend beyond A/B testing product features or optimising technical infrastructure. Experiments can involve teams, structure, processes, culture and any aspect of the business. Indeed, the team that I now head was itself an experiment that launched at the end of 2018.

Last month, three other ‘experiment leads’ and I were invited to present to execs on why our experiments had succeeded compared with others. We were strictly limited to talking about why our team was successful and we weren’t allowed to dwell on the actual work or impact of what we do. Thinking abstractly about my team like this was an interesting – and surprisingly difficult – exercise so I wanted to share what we learned.

Above everything else, the experiments that had succeeded had teams that felt a strong sense of ownership in what they were doing. Here are the four lessons I believe most contributed to that:

1. A long-term vision is more inspiring than solving a problem. Quite often, I hear of research teams or agencies experimenting with hack weeks or side projects to try to solve problems. There’s nothing wrong with that per se, but all the experiments that were successful in this round had something in common – a clear vision that inspired people about what might be possible if we succeeded. Spending time at the start of the project to paint a picture of what you can make possible makes it much easier to find others who want to collaborate with you.

2. Make it more than a pet project. Projects like this need a lead who has some skin in the game; someone must be dedicated to its success. All too often, experimental projects start with high energy and get some quick wins, only to lose steam under the weight of the team’s day jobs. It is far harder than many people may think to ignore day-to-day responsibilities and focus.

3. Measure everything, regularly. My previous column talked about applying more rigorous measurement to market research, and projects like this are no exception. However, researchers are often reluctant to adopt any measurement approach that isn’t perfect. This is, of course, perfectly understandable given our day jobs but it means we can often be overlooked as we wait to report back a number we’re 100% confident in. Other functions in a business will be more than happy getting to 80% confidence and reporting that back. That’s exactly what we must be more comfortable doing. Collect as much data as you can and build in monthly check-ins to force you to assess what you’re doing, as you do it. Don’t wait six months for the perfect answer.

4. Hire people comfortable with ambiguity. The reality of working on experimental projects involves dealing with unexpected roadblocks, working out ways to measure things that you didn’t consider before, and fluidity in almost every element. This isn’t for everyone and we shouldn’t penalise people for not wanting to work in difficult, often ambiguous environments like this. Actively hiring people who are comfortable (or thrive) in such a working culture is essential. Look for people who find lessons in failure rather than taking setbacks personally and who want to lean into complexity. Without those people, any difficult experiment will be 10 times harder to make a success.

This article was first published in the October 2019 issue of Impact.