What are you looking at?

x Sponsored content on Research Live and in Impact magazine is editorially independent.
Find out more about advertising and sponsorship.

On Saturday mornings between 1953 and 1957, a cartoon character called Winky Dink and his dog Woofer would entertain kids who tuned into the CBS network using interactive second-screen apps. Their ‘Magic Drawing Screen’ was a piece of clear plastic that kids could stick to the TV screen. Winky Dink crayons cost 50 cents, and during the broadcasts kids were encouraged to use their other ‘screen’ to complete a picture, solve a code or join the dots on screen.

Parents complained to CBS: their kids were so enthusiastic that the unlucky ones who were denied a Magic Screen and special crayons had taken directly to drawing on their family’s television.

After this exciting start, interactive television languished for half a century. Expensive innovations such as Time Warner’s Full Service Network – an interactive cable TV system that was provided to 4,000 homes around Orlando between 1994 and 1997 – offered the opportunity to order Pizza Hut pizza from advertisements, print pictures, or view curtains on screen from a catalogue in all the different colours they were available, to what turned out to be a largely indifferent audience. More recently, UK broadcasters have fitfully used the red button on the remote control, but as an exception.

The old Winky Dink magic proved impossible to recreate for adults, until – instead of waiting for a broadcaster to build an interactive television, we started to reach for our laptops to Google the Internet Movie Database or the websites of advertisers.

We became comfortable sending tweets or SMS messages while updating our status during the dull bits of a televised football game. No one told us to do this, but now it’s the way that most people watch TV.

The 43-hour day

This type of multi-screening was first reported around 2007. The innocent excitement of Winky Dink’s Magic Bit of Plastic was long gone, and there wasn’t a big company to organise interactivity any more, and so it’s fair to say that early reports could be grouped under the heading: “What Has Happened To Our Kids?”.

The proliferation of screens was creating multitasking behaviour that was clearly – in the minds of the reporters – unhealthy, abetted by a sinister new trend called social media: “Sarah Knight, 20, a sophomore at North Dakota State University in Fargo, said her grades were hurt last year by her indulgence,” reported a censorious Minneapolis-St Paul Star Tribune in January 2007, in the first of literally hundreds of concerned editorials that year. “You can easily spend an hour and a half answering email, going to FaceBook.com,” the shamed student admitted.

The report also mentions that media services company OMD had added up screen-based multitasking activity and found American youth were living a “43-hour day”. The American Press Institute reported that 70% of consumers used multiple forms of media at the same time. Diane Levin, a professor of education at Wheelock College in Boston, told the Tribune that multi-screening was “producing robots… The more screens children have now, the more they’ll need, and the less they’ll be able to master their world.”

Fast forward to 2013, and the robot apocalypse has been postponed. Instead, Ofcom reports that multi-screening has returned us to the Winky Dink era. Its Communications Market Report 2013 is headlined, ‘The reinvention of the 1950s living room’.

One of the unexpected by-products of multi-screening is that kids are staying in the room while the TV is on, even if they have something better to do on another device. Ofcom research shows that 91% of UK adults watch the main TV in the living room at least once a week – although 53% regularly multi-task while watching TV.

This isn’t necessarily the permanent distraction that the “43-hour day” predicted: 25% communicate about the TV show while watching, which the report calls “meshing”. Not surprisingly, young people are almost twice as likely to “mesh”.

More breadth on the nature of multi-screening comes from The New Multi-screen World, an American study completed by Google in 2012, surveying 1,611 participants in partnership with Sterling Brands and Ipsos. Participants filled in mobile text diaries, participated in online bulletin boards and interviews in LA, Boston and Austin, and also logged traditional and digital media consumption in a diary over a 24-hour period.

It defines two types of multi-screening. The first is “simultaneous”: doing two things (which might be complementary, or not) at once. It paints a picture of more divided attention than the Ofcom results: in the survey, 77% of the time when we use a TV, we use another device as well, and 78% of simultaneous use is not complementary. It quotes Bradley, one of the participants, who sounds a little like a 2012 version of Sarah Knight: “I’ve never understood why I do it, but I just do it in the middle of a TV show, and start searching… you feel like you don’t stay as engaged with the show that you’re watching,” he says.

Distraction stations

For traditional broadcasters or advertisers, it appears there’s a lot of work to do to keep our attention, which is more easily grabbed by online content. TV prompted only seven per cent of phone searches, and 17% of searches were prompted by commercial messages on TV. That’s less than the 22% that were prompted by internet video.

‘Sequential’ use, in which a single task is carried from device to device – for example a search is prompted by TV, started on a smartphone at the time, and then the purchase is concluded on a laptop – is even more prevalent. Nine out of 10 respondents are engaged in sequential multi-screening, and 98% of them complete the process that day, according to Google’s research. Small screens are less “sticky” than a TV, but not dramatically so – the average time spent per interaction was 17 minutes on a smartphone, 30 minutes on a tablet, 39 minutes on a PC or a laptop, and 43 minutes in front of the TV.

This is more encouraging for advertisers, but also harder to track – attributing marketing success to the device on which the “last click” was made in this environment clearly gives no insight into what prompted the “first click” – or any of the clicks in between.

Finally, Google presents a world in which devices are rapidly assuming equivalent status – while there are clearly “best” devices, “best” depends on the context, not just on the type of media or the task. The ability to carry a task from device to device is expected.

Putting these studies together, we see a population that spends its time doing more than one thing, but is easily distracted; it’s also one that carries tasks between devices. The first is a problem for broadcasters and advertisers.

The second is an opportunity, but hard to quantify. It’s not clear how to act in response to either trend – because less than a decade ago, it existed mainly in the nightmares of concerned parents.

Screening behaviour

Perhaps we have embraced multi-screening because we all do it, though in different ways: while Google’s research paints a broad picture of behaviour, it doesn’t break the aggregates down further, to capture distinct patterns of behaviour for different segments.

In its ‘ITV Lives’ research, now in its third wave, ITV attempts to segment behaviour by the type of viewer, and also by the target audiences of its programme makers and advertisers.

“Quite a lot of research focuses on the entire adult population: but there are definitely groups out there who use technology in different ways,” says Glenn Gowen, ITV’s head of commercial research.

ITV Commercial commissioned Kantar Media and Work Research to do the work, the results of which are presented as a tool for planners and buyers, which allows them to cut the results according to target demographic or region. Its key finding is a simple segmentation of multi-screeners into Social, Connected, Busy and Traditional Lives. While ‘Social Lives’ are the group that is heavily influenced by social media, ‘Traditional Lives’ households are not so technically savvy – but they still multi-screen.

The results are not as simple as you might imagine. While the Social category is most likely to ‘like’ a show on Facebook, the Traditional participants are the group most likely to enter a competition related to a programme, which implies that multi-screening is not a single behaviour, but a whole set of offline preferences and behaviours translated to a new medium.

It is also heavily dependent on context: the social media behaviour of live sport or quiz shows has predicable highs and lows which may be supported by advertisers during the event – but in the second wave of research, ITV Lives looked at a range of programmes, from The X Factor to Downton Abbey, and found very different patterns for drama, where second screening is complementary before and after the episode, but a distraction during it, even though there is clear evidence that second screening is driving a live television audience. For example, Broadchurch, one of ITV’s word-of-mouth successes, had its largest live audience for its final episode.

Gowen counsels care on how advertisers and broadcasters use the opportunity to coordinate their multi-screen messages.

“The Twitter behaviour for Broadchurch, for example, has massive spikes at the beginning of the programme, massive spikes at the end, and nothing in between. If we provide content, it will have to be complementary,” he warns.

In common with other research on the topic though, we see that the new medium is fast becoming pervasive. ITV Lives reports that a third of its survey population owns a tablet, an increase of 152% since February 2013. Behaviour is changing even more quickly: the numbers using a tablet while watching TV increased 225% in the same period – roughly in line with the findings of Ofcom’s research.

Watching the watchers

One of the stickiest problems with multi-screening is measuring what’s actually happening. Knowing that a second device is being used isn’t useful unless we know what prompted its use, and what the outcome of that use is likely to be. As with the Google or ITV Lives research, this creates a hybrid of quant and qual: the quant can try to measure what was done, the qual to work out why.

The most thorough recent example of this was Screen Life: The View from the Sofa, a study carried out for Thinkbox by COG Research. The study examined more than 700 hours of film from the living rooms of 20 households in the UK, and employed psycho-physiological analysis, a technique that COG calls “digital ethnography”, and online research from a base of 1,000 people with TV and online access.

Lindsey Clay, managing director of Thinkbox, wanted to analyse multi-screening in detail because she believes that diary studies, such as the Google research, often miss important insights. “This type of methodology is flawed. When you ask people what they do, you find they are very poor witnesses to their own behaviour. And television seems to be one of those things about which we can’t honestly report what we do.”

The research had three stages: the initial survey of 1,000 people established the amount and frequency of multi-screening behaviour, but it also helped to identify the households where multi-screening was common, and to segment multi-screening behaviour into groups.

In the follow-up study in homes, CCTV cameras filmed the TV viewing experience for two weeks using a facial recognition technology known as Quividi: it could show who was watching, which devices they were using, and where their attention was focused. With this evidence in hand, 50 households – including most of the CCTV homes – were asked to reflect on what they had done, and why.

Thinkbox found that TV is still our social focus, and that multi-screening drives us to watch more live TV. This is consistent with the findings of ITV Lives and shows, Clay explains, that multi-screening is often a new way to have the TV-based conversations that have been identified in a single-screen world.

The detailed analysis of the video footage shows that multi-screening tends to reinforce TV viewing rather than distract from it – and that even the multi-tasking behaviour may not be problematic for advertisers or programme makers.

First, multi-screeners stay put. “Multi-screening is more likely to keep you in the room during commercial breaks. That was a very surprising finding,” Clay says, “Multi-screeners are less likely to change channels in an advertising break, and less likely to leave the room.” In the research, only 19% of multi-screeners left the room or changed the channel in the break. When there was no multi-screening, the figure was 29%.

Second, multi-screeners find a way to share. The ethnographic research uncovered the insight that households wanted to be together, to have a communal experience, and multi-screening in that case is a compromise: for example, playing along with a quiz show can make it more interesting for one half of a couple.

Third, multi-screening is a distraction that doesn’t detract from attention or recall. The research also gives insight into the problem of attention: if you’re tweeting, are you taking in what happens on the screen? The behaviour that Thinkbox calls “meerkatting”, of constantly switching attention between screens, leads to some equally surprising conclusions.

“Part of the reason that TV works so well is that it is processed in a low attention state, and so it goes direct to long-term memory,” Clay explains. “When we tracked what multi-screeners were doing, their eyes are constantly flicking between screens. But we found that multi-screening is no more distracting, if you are watching television, than any other activity. In fact, we found that it is less distracting than having someone else in the room and talking to them while you are watching.”

Taking this further, Thinkbox has also recreated the Saatchi & Saatchi ‘ironing board’ experiment. In 1981, to study recall of radio advertising, 317 housewives were told they were testing a new brand of starch. While ironing in their homes, they were exposed to a faked radio programme, and afterwards asked to recall the advertising. The conclusion was that recall in this low attention state for “soft” advertising was much better than expected.

The multi-screening experiment asked subjects to wait in a room where TV advertising was playing. One group was told that they could use the other screens in the room while they waited. The other was asked not use any other screen. Creative recall of the advertising was not significantly affected by using a second screen. The tentative conclusions: multi-screeners watch more advertising, give it at least as much attention, and recall it just as well.

Tweet returns

If multi-screening isn’t disastrous for advertisers, how big an opportunity can it be when they have our attention? While qualitative research can be extremely insightful about how we act, we clearly have unreliable insight into what provoked us to change our own behaviour if we have been influenced by several messages on several devices. Multi-screening exposes us to many different incentives to act, plus their interactions, in many different contexts. Trying to figure out what persuaded us to buy, to watch, or to choose is not trivial.

Establishing correlation is a start. SecondSync, for example, analyses the Twitter conversations around TV. It can tell us the most-tweeted about TV programme, and show who was tweeting, how often, and when.

But trying to answer the simplest follow-up question – when there’s more tweeting about a programme, does that mean more people will watch the show again? – is an econometric obstacle course (see Cause and effect).

Some recent research pushes the boundaries of statistical techniques. For example, in August, Nielsen claimed a world first: “…findings, which, for the first time, provide statistical evidence of a two-way causal influence between broadcast TV tune-in for a programme and the Twitter conversation around that programme”.

Or, in other words, more tweets equals more viewers, and more viewers means more tweets. To get there, “The time series analysis methodology used for this study was developed by Nobel Prize-winning economist Clive Granger, and is widely used in the fields of econometrics, physics,
and neuroscience.”

As world firsts go, this is a headscratcher: it’s hard to base a business decision on it. It doesn’t solve the three problems of marketing that analytic specialist MarketShare identifies: attribution, optimisation, and allocation.

Attribution, in MarketShare’s definition, is the basic problem of working out what has worked. Optimisation decides the scenarios for planning purposes. Allocation is the real-time redistribution of resources.

The company is applying itself wholeheartedly to the problem of attribution in a multi-screen world. “You need to start by asking, how am I defining what a screen is?” says Heath Podvesker, executive vice-president, MarketShare EMEA. His argument is that even digital billboards are now a form of screen: and that it is important, and now possible, to measure everything.

Cloud storage and big data make this sort of measurement possible. “Any form of digital distribution has a very high volume of data, from cookies to clicks. It’s terabytes and terabytes of data,” he says, “Management has always claimed it was drowning in data. It’s going to get a hell of a lot worse.”

MarketShare has two processes: it builds a standard top-down model of the process, which uses the available data to predict that, if you allocate more here, you get this much more there.

Multi-screening makes this model arguably more complex, but not dramatically so, considering that it already measured activity on digital platforms. That is used alongside a continuously updated bottom-up model, which is showing the results of what you are doing in almost real time.

The key is in modelling and measuring the interactions between devices, and not assuming that every interaction is based around one type of marketing – what MarketShare calls “swim lane measurement”. That has been compounded by the habit of attributing success to the “last click”, rather than analysing the process that led to it – which Google has shown is far more intricate than simple techniques can measure.

“We all know that last-click attribution analysis is flawed, but that’s what we had,” says Podvesker. “Now I can have a wonderful understanding of how my television drives my search, or how paid search is driving display advertising; and so that’s a quantitative picture of what’s happening. This provides me with a structure and direction that I need to plan and deploy my budget.”

The essential insight in multi-screen analytics is to capture “assist rates” – that is, a piece of collateral that customers see, but don’t act on at that moment. Yet those assists don’t just come from activity across the various screens, even if that activity is optimised and coordinated according to strategy. MarketShare’s analytics will take in data on weather, season, competitive activity and the economy. Multi-screening is, in this context, a subset of the complex interactions which cause us to buy – but at least it is one that can be measured.

The next question is, if it can be measured, can it be successfully analysed? Because, if so, there is plenty of opportunity to guide and optimise that customer’s multi-screening behaviour.

Multiple opportunities

The questions of how to optimise activity, and how to allocate resources, remain. Both, in Podvesker’s view, rely on an integrated marketing effort which has the ability to act on what it discovers works best.

On one hand the maths PhDs who build the models MarketShare uses are solving multiple simultaneous equations that are breathtaking in their complexity.

On the other, at this stage in the development of multi-screening, the optimising behaviour need not be so complicated. “It’s very early days for this. We’ll look back at some of the work we did in a few years, and think it was terribly unsophisticated,” says Clay at Thinkbox.

Some of the early initiatives can take broad insights and apply them internally: ITV Lives, for example, is being used internally to help plan online activity and design competitions.

MediaCom (see So what’s the plan?) is using what it knows to build media plans that enhance the effect of multi-screening, and makes the point that this type of behaviour requires little creative input from clients beyond a willingness to act on what we already know.

On the other hand, other brands are taking the multi-screen experience and using it to drive activity. Interactivity specialist Shazam has been one of the most successful. It has developed from a handy widget on your phone to a device to coordinate second screen activity automatically to the TV message.

Shazam recognises audio by matching a few seconds against its database. At first this was used simply to let users answer the question, “What’s that song?”; now, by recognising the audio in an advertising message on TV, Shazam can direct smartphone or tablet users to extra content, or free samples, or long-form video from the advertiser.

Today Shazam has displayed its “call to action” in the corner of 10 billion TV spots in 12 countries. It has been part of 300 TV ad campaigns for 150 brands, including Red Bull, HTC, Sony and Coca-Cola, and has just had a $40m investment from the world’s richest man, cellphone magnate Carlos Slim, to expand its activity.

“Interactive TV has been talked about for years, but it was always dependent on the set-top box,” claims EVP of marketing, David Jones. “Shazam completely bypasses all those constraints: it recognises audio, regardless of the provider, the set-top box, anywhere in the world. It’s a universal solution.”

There are 350m active Shazam users; those who use it on the iPad (and, from September 2013, the iPhone) have an enhanced experience as Shazam listens passively and places tags on its home screen for the advertising messages they hear while second-screening: in effect, it saves their commercial messages for later. Shazam internal research shows that 55% of users return to their saved tags.

Shazam’s advertising revenue doubles every six months. A tie-up with ad agency Y&R will mean that the Shazam tag is introduced as part of the creative process, not by media agencies – so that, for Shazam’s clients, second screening “will be baked in from the first day” in Jones’s description.

A side result is that Shazam can already feed back exact data to its clients on how and when viewers act on the multi-screen offer.

Multi-screening gives quants a headache – but it also gives creative marketers a chance to increase the effect of their activity.

In September 2012, Paddy Power used advertising to ask customers to tweet the messages that they wanted to send to the European Ryder Cup team. The best tweets were given to a fleet of five aircraft. They wrote ‘You can keep Piers Morgan’ and ‘Europe has better hair’, among other messages, in the sky above the course. The messages, of course, were then featured on television, and retweeted.

In using the sky, Paddy Power had grabbed the biggest screen of all.

Most multi-screen marketing will be more mundane, but as analytics improves, so these can become measurable improvements in marketing.

There’s reason to be enthusiastic about multi-screening – and so far, it hasn’t turned our children into robots.

• First published in Issue 3 of Impact Magazine. Our thanks to special report sponsors Hall & Partners