For the hack of it

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

How do you solve an age-old problem like predicting new product sales? “It’s something we’ve been dealing with for 10 years,” says Chris Brooks, head of analytical systems at Dunnhumby, the company best known for its work on Tesco’s Clubcard. Models are refined, algorithms are tweaked, new solutions are proffered, but still “the vast majority of new product launches fail,” says Dave Balter, CEO of BzzAgent, Dunnhumby’s word-of-mouth marketing company.

Time for something different then – and that something was a hackathon. Hosted in Boston in partnership with the “big data hacker space” Hack/Reduce (whose mantra is “code big or go home”), the event brought together 111 teams to compete (both online and in-person) to find the most accurate model for forecasting sales for a new product 26 weeks after launch.

“We’re well-versed in the traditional approaches for predicting new product sales,” says Piers Stobbs, Dunnhumby’s head of data science research and development. “What we were interested in was attacking the problem from more of a machine learning, computer science perspective, to see whether it could do as well or shed additional insight.”

The two approaches are very different, Stobbs explains. Where a traditionalist might start by trying to understand the problem and building a solution from the bottom up, he says: “The machine learning approach – people sometimes call it ‘the black box’ approach – doesn’t really care so much about fundamentally understanding the problem. You just try and generate as many potential attributes and features as you can that might be relevant, and then throw them all into the proverbial black box – into various algorithms that then use different analytical techniques to build what are often more complicated models, but which can lead to more accurate predictions.”

Each team had just 13 weeks of real-life sales data on which to base their algorithms and predictions (though they were also given 26 weeks of sales data for different product launches as a “training set”). They didn’t know what the products were – only that they were generic items like books, DVDs or yoghurts. And with that, so began 12 hours of furious hacking.

Outside in

The competition was run through Kaggle, a platform that brings together 100,000 data scientists globally to have them take part in such events. A Kaggle-powered leaderboard kept track of each team’s scores throughout the day. By 9pm, Jaysen Gillespie, a director of financial and marketing analytics at Criteo North America was declared the online winner, with a score of 0.19028. The in-person winner was a team of MIT students – Alex Levin, William Li and George Tucker – who scored 0.20031.

But what exactly do those numbers mean? “In some ways the score is arbitrary,” explains Brooks, “but to put it in a business context, 0.2 means that when we ran the predictions, three-quarters were within 15% of the actual value.”
“It’s a very good prediction,” says Brooks. Better than where Dunnhumby is now internally? “If we could implement that model, if we could successfully implement that level of accuracy in our internal production systems then, yeah, that would be positive,” says Stobbs. “But I think there are a lot of additional idiosyncrasies in the real world that might not have been fully captured in the dataset that we supplied. So whether or not that’s viable, we’ll have to see with more research.”

That’s the next step for Dunnhumby: to take the hacked models inside the business to see whether they can become part of the product suite. Regardless of what results from these rounds of tests and validations, hackathons are set to become part of the company’s innovation efforts. “We’ve practised a form of this for quite some time, running ‘Innovation Fridays’ with staff,” says Balter. “But what I think is most interesting here is that we’re able to tap into this external community that has maybe solved problems in vastly different ways than Dunnhumby ever has before.”

For Brooks: “It’s a very rapid way for us to understand the broader range of techniques that are out there”, while Stobbs says it’s helped expand his field of consideration when it comes to hiring talent for his R&D team.

Hackathons are starting to deliver results that have businesses paying attention. Adrienne Cochrane, of Hack/Reduce, says: “Hackathons are a way to generate ideas for new approaches and to find people that bring those ideas to the table.”

• Image courtesy of Skyscope Creative

• Our thanks to the Impact technology section sponsor Ixaris