NewMR Virtual Festival blog
The Brits have Dragons’ Den, the Americans have Shark Tank – researchers get the Insight Innovation Competition, ostensibly a sideshow of the Festival of NewMR 2011 but in practice one of the biggest draws (personally speaking, at least).
The format will be familiar to fans of the two TV shows mentioned above. Six businesses were given 10 minutes each to pitch their concept at a panel of judges, three of whom stumped up $20,000 to invest in the best idea.
Walking off with the prize was Decooda, a company that started out analysing social media but has since developed an MR Assistant that automates the coding of open-ended responses within surveys.
Here’s Decooda explaining how it works:
“Clients start by uploading their raw data file, which may include branching and skip logic questions and any related metadata. We automatically parse the file into questions and their associated responses and metadata. Next, the client can select the number of categories they want responses to be assigned, so that responses with multiple attributes can be auto-coded into multiple categories. Our 100% automated natural coding process then clusters responses into groups of related documents. These document sets can then be easily manipulated to the analysts liking by merging, renaming or deleting clusters.”
Text analytics also allows the open-ends to be scored on sentiment, emotion and context. You can read Decooda’s full pitch here.
Second was the iPhone survey app PollBob, which presented a bold vision for the future of location-based quick polls at a price point that might appeal to small business owners. But a small user base – 6,000 people spread across North America – and a host of rivals, including Opinionaided and Wayin, weighed against it.
As part of yesterday’s Festival of NewMR, Jon Puleston of GMI pulled together 32 judges (including me) to vote for the things “that have had (or are having) the most transformative impact on market research”.
The Research Transformation Awards covered a broad range of concepts, techniques, methodologies, books and other forms of communications channels – including publications, a category in which I’m proud to say we won.
You can read the full list of nominees and winners over on Puleston’s Question Science blog, but on behalf of the team here at Research I wanted to say a big thank you to those who voted for us. It means a lot.
Congratulations also to the many wonderful researchers who contribute their ideas and opinions to our magazine and website in the form of quotes, blogs and articles – it’s your award too.
Never mind ROI – Mark Earls was out to generate a better Return On Effort at the Festival of NewMR 2011 with a presentation on the importance of pattern spotting in large data sets. The aim of this technique is to quickly understand the nature of the market a company is dealing with and whether consumer behaviour within it is based on independent choice or social influence.
“What we are talking about is not boiling the ocean,” he says. “It’s spotting patterns in population data sets” – retail sales data being a prime example. It’s another way of going about understanding individuals, Earls says. “Market-level patterns reveal what lies beneath.”
Markets driven by independent choice and social influence have distinct sales distribution curves – respectively, the short tail and the long tail. In the former, a large number of items have a decent share of market. In the latter, a very small number of items account for most of the market volume but with a large number having a small share.
In long-tail markets social copying is at work, says Earls – and this forms the subject of his new book, I’ll Have What She’s Having. In these instances there are two sources consumers might look to (whether consciously or not) to shape their purchase decisions: experts when there’s very few items to choose from, and peers when they are overwhelmed with choice.
Earls reckons that social influence has a hand in purchase decisions in a lot more markets than one might assume. Crucially, though, he says much traditional market research is better suited to understanding purchase behaviours in those markets where choices are arrived at independently, either through a process akin to guesswork or careful consideration of options.
Understanding social influence is within the capability of the research industry, Earls reckons, but those already doing “great stuff are taking inspiration from outside the industry”.
It was a suitable note to end on, as BrainJuicer CEO John Kearon followed Earls with a talk about how the works of psychologists Daniel Kahneman and Paul Ekman is being used to help researchers understand the role of emotion in consumer decision making-and how to measure it.
“One of the biggest mistakes we’re all guilty of in business is that we over-intellectualise marketing,” says Kearon. We – humans – all “think so much less than we think we think”.
“We need to get better at measuring emotions. It drives most decisions,” he says. That’s why the best-performing ads are stuffed full of emotion – ads like the “Just One Cornetto” campaign which lingers on in the memory long after it last ran (in the 1990s, according to Wikipedia).
An ad with a strong message that scores well on traditional testing measures is still far from being as effective as it could be, says Kearon. “If you have a strong message, you cannot have a four- or five-star advert – you are getting people to think and stopping emotion from taking over.”
Speaking of getting people to think, Kearon kicked off his talk with a request for Festival delegates to imagine a world where market research has been made illegal, where surveys have been banned. “How would organisations make decisions?” he asked – before leaving us all hanging. “That’s for another time,” he said. Or, if you like, for pondering in the comments thread below.
“We have made more mistakes than anyone else,” said Communispace CEO Diane Hessan, kicking off her talk at Festival of NewMR 2011 about what can go wrong within market research online communities.
Here are her five things that haven’t worked over the years:
- The bigger is better syndrome: Community size is the wrong metric to focus on. “The perfect size for us, having done about 600 of these communities, is about 300 to 500 members,” Hessan said. “It sounds small, but the metric that matters is engagement. If the community is too big participation actually goes down.” Communispace has a 64% rule for participation rates, which works out at an average of 7.3 contributions per active member each month. Compare that to Facebook Fan Pages, Hessan said. Recent research suggests that for every 100,000 fans a page has only an average of 54 people will actually ‘like’ a post and only nine will comment. “Engagement trumps sample size.”
- Survey-itis: Surveying too much can be an easy mistake to make and a major problem. “Over the years we have had communities where we have just surveyed people but members told us this was just downright boring,” said Hessan. “As you increase the variety of activities you increase participation.” A study of 57 Communispace communities found that those with more balanced facilitation generated a much higher number of average monthly contributions. “Until 2009 the preponderance of what we were doing was just discussion boards and surveys,” said Hessan. “But there are so many examples of what you can do above and beyond traditional market research. We have had great success with mobile ethnography, heat-mapping, collaging, whiteboarding, video chats, co-creation. The net here is that by using less researchy, more humanist approaches, success has gone up.”
- The garbage pail: Hessan’s term for doing things with a community that are not worthwhile, or succumbing to the temptation to use it for everything – like working out 2+2 on a calculator because the calculator happens to be to hand.
- The Trojan horse: sometimes people want to use a community to listen to consumers, but others want to create advocates, Hessan said. “But consumers know if you are only pretending to listen to them, just so they will go out and buzz about you… that has never, ever worked for us.”
- The traditional research mindset: Concepts like the importance of anonymity, of keeping things neutral and structured and ensuring the client is invisible to consumers don’t apply in communities, says Hessan. “What works is being open, transparent and having a conversation.”
By Jeffrey Henning
Annie Pettit, chief research officer of Conversition Strategies, regaled the NewMR Virtual Festival with an allegorical love story.
Once upon a time there was a man named Mr. Survey. He loved to help clients get the right answers. But Mr. Survey was sad. His survey response rates were horrible; he remembered the good old days when response rates were 60% without incentives but now they were 5% with incentives. The surveys were too long to fit to between dinner and cleaning up the kitchen, and his respondents were always telling him his surveys weren’t relevant.
Mr. Survey had many great qualities. He knew exactly how to measure information quantitatively. He could gather frequencies accurately; he could inquire about specific SKUs; he knew how to ask how many minutes you walked your dog. He knew how to do attribute batteries and personality batteries and how to study early product adoption.
But he needed some help. To gather all the information his client needed about athletic shoes, he would have to write a one-hour survey, and he knew he couldn’t do that to his respondents. He had four problems with the survey and didn’t know what to do.
Fortunately for Mr. Survey, Ms. Social Media Research thought he was cute, and, being a girl, she could listen to millions of conversations and never get bored. She could identify all kinds of conversations. She knew how to measure them. And, well, she thought Mr. Survey was cute!
To impress Mr. Survey, she introduced him to all the conversations about athletic shoes. She went to social networks and blogs. She checked video sites and photo sites and commenting sites and question-and-answer sites and review sites. She went everywhere and listened to everyone.
Ms. Social Media Research helped Mr. Survey with each of his four problems.
- Mr. Survey didn’t know which retail outlets his consumers cared about. Where do people buy shoes? What retailers are important to consumers? Ms. SMR found all the retail outlets that were discussed by Mr. Survey’s customers and prioritised them in order of volume of discussion.
- Mr. Survey didn’t know which brands his consumers cared about. Mr. Survey had 500 brands of shoes to track; far more than could be done in a survey. Ms. SMR found all the brand names that were discussed by his customers and identified the five most important brands.
- Mr. Survey still needed data on the other 495 brands. No respondent would answer a matrix question with 500 brands, but Mr. Survey still needed information on the less important brands. Ms. SMR gathered the social media sentiment scores for all the brands, and presented it to him on a five-point scale so that it looked the way he was used to seeing it. That way Mr. Survey could keep his questionnaire short.
- Mr. Survey needed a lot more context about shoes. Mr. Survey was cute but not terribly imaginative: he couldn’t imagine all the unique types of conversation that included his shoes. Ms. SMR did lots of research: people were talking about basketball shoes and golf shoes, but not about lacrosse shoes or bowling shoes. But Ms. SMR didn’t stop at shoes. She was pretty darn nosey about everything, and she found that people talked about shoes and shopping, shoes and music, shoes and food, and so on.
Ms. SMR gave Mr. Survey data on every aspect of shoes. Just like the good old days when Mr. Survey would get 300 pages of data tabulations on his desk, but this time with social media data. Mr. Survey could not believe his eyes. Mr. Survey was in love!
Mr. Survey could not hold it any longer. Ms. SMR had so many wonderful characteristics. “I want to give you product quantities and frequencies, I want to give you census and target population generalisability, I want to give you precise attribute battery responses. Will you marry me? You are my better half!”
Ms. SMR accepted. “I want to give you unimaginable sample sizes. I want to give you flavor around your key measure. I want to give you unending competitive brand lists. I want to give you serendipity and crazy ideas. I will marry you!”
From that day forward, Mr. and Mrs. Social Media Research-Survey conducted market research together. And they lived happily ever after.
Republished with permission from the Voice of Vovici blog.
By Jeffrey Henning
At last week’s New MR Virtual Festival, Tamara Barber of Forrester Research, in her presentation “MROC: now we’re telling you what it is”, discussed the future of market research online communities. “MROCs are here to stay, but only the most strategic will survive,” said Tamara. “They have been whizz-bang shiny objects for researchers in the past, but the ones that survive will be those that can actually help clients.”
MROCs have been called lots of things: insights communities, online insight villages, online knowing festivals, online insights communities, think boxes, innovation communities, even idea spaces. Whatever you call them, they are a community used for the purposes of gathering insight. Even if MROC sounds like something from the Flintstones, a la Terri Sorenson [and Jane Mount!], said Tamara, Forrester is going to stick with calling them MROCs: “We came up with the term three years ago to describe these,” said Tamara, “so we will stick with it.”
What is an MROC? Forrester’s definition: “A market research online community is a captive interactive group of people online, joined together by a common interest, systematically harvested for market research purposes over time.” For Forrester, MROCs are private communities. “They are not just used on an ad hoc basis when you have a question but are used regularly. MROCs are different than online focus groups because they are used over time. And, while you may be able to do surveys in a community setting, MROCs are really qualitative.”
Tamara shared the results of some ongoing Forrester research into what’s next for MROCs.
- The size debate is going away - While past discussions have obsessed about small vs. large online communities, “today more people are really talking about engagement: how you want people to engage.” Measuring engagement really reveals the health of the community. Yes, the bigger communities let you “slice and dice” participants, but what matters is that members are engaged and contributing. “Instead of focusing on size, focus on engagement.”
- Communities becoming mobile - Vendors are optimizing communities for the mobile web and are building smart-phone applications for the iPhone and Droid. Community members can then participate in many of the same activities on their phones rather than on their computers. Mobile solutions help brands “connect with people in the moment, when they are involved with a certain stimulus in the real world.”
- Communities becoming even more social - While MROCs have always been about the social activity, now communities are reaching out to the public social networks that members already engage with: inviting members to do exercises on Twitter, or to access the community through applications on Facebook or MySpace “so that you don’t have to leave Facebook to participate in the community.”
- Communities are engaged earlier in the business-planning process - “As I said at the beginning, MROCs are here to stay but only the most strategic will survive,” Tamara said. “Communities are moving away from providing one-off answers to questions: those situations where the CEO has a burning question, and the survey isn’t back yet, so let’s hit the community.” Community research is now becoming a formal part of the process for new product development. MROCs are helping facilitate the launch of many successful new products.
Tamara will be sharing more trends in an upcoming independent Forrester white paper. “The bottom line,” she concluded, “is that, regardless of what you call them, MROCs are evolving and proving their value. It is in an exciting place to work for certain!”
Republished with permission from the Voice of Vovici blog.
By Jeffrey Henning
Games as a source of research innovation were a popular topic at the NewMR Virtual Festival. For instance, check out “Gamifying surveys – continue or quit?” for Brian Tarran’s recap of Tom Ewing’s presentation “Game On”. Brian Fine, CEO of Australia Online Research, kicked off the day’s discussion of games with his presentation, “‘Serious Games’ has a place in future MR”.
Serious games are a genre of games where the primary purpose is something other than entertainment: for instance, training, education, marketing and simulating real world events. “To date most serious games are for education and training, but market research could be part of the next wave,” said Brian.
What would be the goals of using serious games? To improve response rates and engagement levels. Traditional surveys have low response rates among 18 to 24 year olds, for instance; “river” recruitment needs to engage potential respondents in new ways.
Serious games seem especially suited to researching youth. “Kids learn by trying and making mistakes. Their brains are making patterns, and learning to analyze patterns,” said Brian. “Playing a game exercises your brain until you master the pattern: once you master it, then it becomes boring. The real challenge is when you are always at the edge of your ability.”
For examples of the serious games market researchers should aspire to, Brian gave some common and uncommon examples:
- Second Life - “Though waning in popularity,” Brian said, “one interesting phenomenon is Second Life, with its virtual landscapes, some based on cities, some fictional. It’s a place for people to explore, organize, meet people, even fall in love. Players can portray themselves like comic characters or can create an avatar that is realistic.”
- Darfur is Dying - We go from life to death, with Darfur is Dying, a game designed to build awareness of the experience of 2.5 million refugees in Sudan. The player’s goal is to keep the simulated refugee camp functioning in the face of possible militia attacks.
- Peacemaker - Next we go from war to the possibilities of peace. Peacemaker is another “game for change”, enabling players to take the role of either the Israeli prime minister or the Palestinian president. Settings range from calm, tense or violent, and show you virtual news events using real news footage. Will you bring the region into disaster or win the Nobel Peace Price?
- America’s Army - The next serious game that Brian covered was designed to encourage military recruitment: America’s Army provides an authentic military experience, from deployment to different missions.
- FoldIt - The FoldIt game encourages players to “solve puzzles for science”. The number of ways to fold a protein is infinite, and among those infinite possibilities may be proteins to help in the fight of diseases such as HIV and cancer. The game takes advantage of human puzzle solving capabilities to identify possible proteins for scientists to test.
- Re-Mission - From fighting cancer in the future, we move to fighting cancer now. The game Re-Mission is designed to help improve the quality of life of young people with chronic illness. The game offers patients health information and has 20 missions to choose from.
So these are not the types of games we typically think of. How can market research include games? One way is for surveys to adopt more engaging, more game-like user interfaces. Another opportunity is to embed games within surveys: such games would need to be short, of course, given the setting. For instance, one “game” that could fit within a survey is to incorporate a prediction market. Researchers would need to be aware of the potential bias that such game-enabled interfaces might provide. However, serious games present an opportunity for researchers to experiment and – yes – play!
Republished with persmission from the Voice of Vovici blog.
By Jeffrey Henning
At the NewMR Virtual Festival, Diane Hessan of Communispace said one of the most frequently asked questions she gets is, “What are the critical successful factors for a community?” Having built over 400 of them, Diane says, “We have probably made more community mistakes than anyone in the world!” Based on those experiences, and on extensive research on research that Communispace has done, Diane has identified 8 myths about successful research communities.
Myth 1: A community is just a fancy word for a panel. “Using the terms interchangeably creates a lot of confusion! They are actually quite different,” Diane said. “A panel is a large group of consumers that a brand would access about once a month to do a survey. The members of the panel don’t typically interact with one another. In contrast, think of a community as a virtual room where customers can talk with one another.” In fact, a community fosters three channels of communication:
- The client company is asking questions of community members. Each time it might be a survey, a bulletin board, a brainstorming session, a request for members to send in videos or photos, or to participate in mystery shopping, journals or diaries.
- Members can give unsolicited advice to the client. “Answers to the questions we forgot to ask!”
- The members of the community talk with each other. “In the third channel, the one the researchers love the most,” Diane said, “members talk with each other and the result is more exploratory about their lives, dreams and values.”
Myth 2: Communities can be valuable for almost any decision. There is a myth that you can just ask members about anything, figure out what they are saying, and move forward. Communities are great for new product service development work, existing brand and product feedback (including of competing brands), general market research and longitudinal research. Research communities are not appropriate for:
- Go/no-go decision making
- Volume forecasting
- Response rate estimation
- Predictive modelling
- Advanced quantitative studies
- Satisfaction tracking studies
- Ad tracking
- Market size estimation
- Brand equity studies
- Awareness and usage studies
Myth 3: Bigger is better. There are lots of different types of online communities, for social media, for marketing purposes, where the success metric is how many registered users you have. This is not the case for research communities. “The purpose of the community is to understand, to engage. You don’t need one million members. You need members who are engaged and responsive and willing to open up their lives.” You can generate 3,500 contributions a month from a community with 350,000 visitors, 1% of whom participates once, or from a 400-person private community with 55% participation rate and 16 contributions per active member,. “Measure engagement rather than members! The real Holy Grail is intimacy.”
Myth 4: Make sure you pick the best technology. “There is an endless list of companies with great technology and with communities where no one participates much,” Diane said. Technology is important: you have to have a rich feature set; you need an ability to run with subsets; to profile community members, but engagement is the most important thing.
Myth 5: Communities are largely for “brand fans” (or high involvement products). A great use of a community is to understand non-customers or customers who only occasionally use your product. You can study brand advocates, of course, but communities have many other uses. You can have a vibrant community on low involvement products – say, toothpaste – where members have an engagement with one another. Communities are very vertical.
Myth 6: Community members are biased. “After all, you have a relationship with these people. After time, they love you and will be excited about anything you do, right? Our research team has done extensive research about this and finds that community members become slightly more critical over time.” As the relationship strengthens, community members are more likely to tell you why they are unhappy with something.
Myth 7: The more you pay, the higher the response rate. “Let me give you some context – the average member is asked to engage in 8 to 10 research activities a month, so we give members some sort of gift as a thank you for this time. They get a gift certificate of about $10 a month if they participate at a certain level that month.” Communispace has tested this extensively, and has found that participation doesn’t go up when you pay more. The number one reason that people participate in communities is that they want “a continuous and loud voice in the future of company”.
Myth 8: ROI is hard to get. It is not impossible to measure the ROI from communities. Research communities are “better, faster, cheaper” and they have many potential returns: reduce costs, build customer loyalty, kill bad ideas fast, conduct more effective marketing, get to market faster, develop more creative solutions, decrease risk, try new products, bring in fresh perspectives.
Republished with permission from the Voice of Vovici blog.
By Jeffrey Henning
Ray Poynter, managing director of The Future Place and organizer of The NewMR Virtual Festival, kicked off the festival with a presentation on “How can we turn online discussions into insight?” He suggested three strategies: netnography, discussion creation and mass techniques.
- Netnography - To see where academia meets real practice in the real world, check out Robert Kozinets book Netnography. Kozinets outlines the practical steps that you can follow. To him, netnography always involves observation and participation. It is not enough to simply monitor what others are saying online: you need to participate in the conversations and echo back your analysis to other participants to make sure you really understand what you’ve heard (what Kozinets calls member checking). Once you’ve collected a corpus of discussions to analyze, there are a wide range of techniques you can use: discourse analysis, conversation analysis, hermeneutics, grounded theory, semiotics, NLP, word counting and mathematical models.
- Discussion Creation - If the Australian tax office can have a conversation with their “customers”, Ray argues, then all organizations can have conversations with their stakeholders: their customers, employees, partners. Look to social media to create conversations, and look to novel services for new opportunities to create conversations (for instance, Foursquare). “Market researchers wouldn’t want to create a conversation and feed it into Marketing,” Ray argues, “but if Marketing creates a conversation then that should feed into Market Research.”
- Mass Techniques - The third approach is mining vast quantities of data, reporting on trends. How many people mention a brand name? A movie star’s name? What are the trending topics in a community or about a brand? Mass techniques leverage content to see changes in its volume, to search for trends, to perform sentiment analysis and to identify influence. Some of these mass techniques are quite inaccurate; for instance, FreshMinds looked at how bad most sentiment analysis software is. Others have looked to identify the role of influence on social media conversations.
Social media research ethics are still being discussed and parallels are being sought. For instance, “looking at the ethics of mystery shopping - what are the rules there and are they applicable?” Preserving anonymity is really difficult, when someone else can use a search engine to identity the commenter behind a verbatim quote. Already firms like Google, Facebook and Nielsen have run into privacy debates.
The final issue in turning online discussions into insights is confidence. “In quantitative research, confidence is measured statistically: 90% certain with a plus or minus 3%,” Ray said. “Now we have to say: ‘Some people believe this, lots of people believe that, nearly everyone we were able to find believes in these.’ To achieve confidence, some sort of member checking is required, going back to some of the people to see to verify that you have correctly understood what they meant.
Ray summed up by saying, “This is going to be BIG! But there will be disasters!” He advised that “software is better at finding corpora than analyzing them” and that researchers need to know what model they are applying to their data and need to educate clients about the confidence they can place on the findings you report.
Republished with permission from the Voice of Vovici blog.
By Jeffrey Henning
For the NewMR Virtual Festival, Alastair Gordon, of Gordon & McCallum, presented “Will New MR become Old MR? The Challenge of Turning Radical Methods into Sustainable Businesses”. Alistair defines New MR as:
- Listening, not questioning
- Engagement and interaction, not passivity
- Social media focus, not just online data collection
- “Deep, realistic, emotional response,” not artificial, rational response
- Communities, not samples
Many researchers believe New MR will drive out Old MR the way the telephone drove out the telegraph, using that specific analogy. That may be, Alistair argued, but the reality of the analogy is rather more complex: while the telephone was invented in 1876, telegraph usage continued to grow; in fact, the heyday of the telegraph was long after the invention of the telephone. In fact, in the late 1920s, central bankers were still using telegrams to coordinate their response to the onset of the Great Depression. “It took multiple hits to kill the telegraph,” said Alistair: widespread household adoption of the telephone, reductions in calling costs, the invention of the fax, for instance. It will take multiple hits to kill “Old MR” and Old MR’s heyday may yet be in the future.
Alistair shared some quotes from a magazine article about “the new market research”:
- “Forget focus groups and…surveys… Anything worth knowing about your customers, traditional market research can’t tell you anymore.”
- “You don’t necessarily care if 20% feel one way and 80% think another…You want a range of views.”
- “We are in the midst of watching a real shift away from survey-based research.”
Quite bold statements. Made more bold when Alistair revealed that they are from a July 1998 Inc. Magazine article. Enthusiasm for New MR is understandable, but… “It’s not inevitable that new techniques will supplant the old ways. Consumer insights help, but it’s only part of the picture. Being better is good, yet a rapid change to New MR requires massive superiority.”
Alistair argued that, in pitching New MR, we often forget the emotions and client need states of market researcher customers. “MR clients are driven by need states just like consumers. They have emotional needs, which effect how we package New MR.” Essentially we need to package New MR to fit into these need states. On their own, New MR methods and theories can help research clients, but such advantages often seem merely incremental.
For New MR to become Old MR, Alistair argues that New MR and Old MR should forge a synthesis. He gave two examples:
- A methodology using online qualitative research, a customer advisory panel and SMR text analytics. “You could build a superb customer research program if you put them all together, but any of these methodologies on its own won’t be sufficient. The telephone alone didn’t defeat the telegraph.”
- A methodology using neuroscience/modeling analytics, mobile studies and a brand-tracking study. “Align these with the traditional survey, which offers a lot of breath for brand awareness, but if the survey could be triggered by mobile studies and use elements of neuroscience…!”
There are three ways that New MR can become Old MR:
- Fading into the background. “Good ideas, over-hyped, but the great excitement gradually fades into ‘business as usual’.”
- Cherry picking. “The scalable, easier to integrate aspects of New MR are adopted by the top five agencies, leaving smaller companies with ‘scraps’ and niche markets.”
- Genuine revolution. “New MR recognizes its own limits, plus the potential for integration of other methods.”
Alistair concluded with a rosy vision of the future: “New applications arise, based around precise business solutions and clients’ ‘emotional’ needs, leading to a wave of amalgamations and cooperative partnerships among New MR firms that drive a genuine MR revolution.”
Republished with permission from the Voice of Vovici blog.
Researchers have many problems to contend with in their never-ending quest to uncover great consumer insights – a lack of time, a lack of resource, inadequate tools and approaches. But the biggest issue facing the industry, according to Virtual Surveys’ Rich Shaw, is a lack of ‘joy’.
“We’ve managed to reduce vast swathes of our industry to soul-destroying, mechanical processes,” lamented Shaw. “Joy and passion are important because they are the basis of creativity and innovation.”
He makes a business case for joy in the workplace, arguing that as data becomes more easily obtainable, the economic value of information is reduced. Value instead comes from creative analysis of that information, and finding innovative applications for the data.
Shaw says researchers need to embrace the ‘hacker ethic’ – a philosphy that values things like free information and decentralisation.
Quoting the book The Hacker Ethic by Finnish philosopher Pekka Himanen, Shaw says hackers do what they do because they are passionate about something, rather than being duty-bound to do it. Money is important, of course, but not the only motivation – happiness is paramount. A key belief is that people should be able to determine how they use their time as much as possible.
An idea worth running by the HR department, perhaps?
Kantar’s Tom Ewing should have had most researchers pressing the mental ‘pause’ button with his NewMR presentation this morning exploring the idea that surveys are videogames. Thinking about it for any length of time, as Ewing has done, leads one to draw a number of comparisons.
His stand-out point was that researchers, like games designers, are world-builders, crafting artificial environments for people to progress through, hopefully to the end screen without getting bored. So can we leverage game design skills to improve respondent motivation and engagement – a concept known as “gamification”?
Ewing makes the case that in many ways, researchers are already doing so. “Games motivate their players by establishing goals which create the intrinsic motivation to succeed,” he says. “Research projects also have goals – task and survey completion – and associated rewards, even if they’re not always intrinsic ones.
“But simple in-game achievements aren’t always enough,” he says. “The best games are designed to reward imagination and creativity in players, and we’re seeing new MR methods emerge – like crowdsourcing and similar social idea generation techniques – which aspire to doing the same.”
There are, however, aspects of game design that currently sit at odds with the way research is conducted. Ewing emphasised the strategy element core to most games: that is, that players are placed in a world and told to figure out a strategy for making their way through it. Research projects, he says, actively discourage strategy. “Should this always be the case?” he asks. “After all, the feeling of mastering the environment is one of the most engaging and motivating in games.”
Might researchers also consider abandoning the linearity that typifies most projects? Sandbox games are some of the most popular on the market, presenting players with an environment where they have “more leeway to explore, play freely and carve out their own path”. Ewing says that with the advent of longer-term research communities “we’re seeing sandbox projects become a reality”.
“But we can still learn a lot from their balance of compulsory and optional tasks,” he says. “Most sandbox games give players the opportunity to explore within a loose structure of missions. Completing these missions is not the same as completing the world. Is this a model we can imagine working in a research community, as opposed to the rigid task-based model we often see used now?”
He warns, though, that introducing game mechanics can have unintended consequences. Different players have different playing styles – some, for instance, will actively seek to bend and break the imposed rules – and as such, some surveys or communities will appeal to certain types of people more than others.
(Helpfully, Ewing has posted his entire presentation on his blog here.)
BrainJuicer’s John Kearon spoke this morning about the company’s Digividuals, the research robots built to match certain demographic specifications which then scoure the web for people like them, “gathering their thoughts, blogs, photos, music and videos and adopting them as the robot’s own to produce a living, breathing single-person synthesis of the many thousands of people who fit the profile”.
Kearon told NewMR delegates that a freeware version of Digividuals would be made available next year, perhaps as early as January.
Start channelling your inner Asimov.
The debate was over before it even started. Kicking off the NewMR Virtual Festival webinar pitting online versus in-person qualitative research, advocates of both modes agreed that a combination of the two would be relied upon by clients in years to come, rather than one totally displacing the other.
It is inevitable, of course, that offline qual will lose ground to online in the coming years, just as face-to-face and telephone gave up market share in quant work. But both sides in the debate accepted that the choice of online and in-person approaches would remain, and which was used would depend on research objectives.
And so the panellists were left to argue over the amorphous point of which was best at meeting client needs for speed, quality, cost and depthand for creating respondent engagement. Here sharp divisions emerged.
Visionslive.com’s Andreiko Kerdemelidis, for the onliners, talked up the cost benefits, leaving Simon Patterson of QRi Consulting to point out in defence of the offliners that in-person costs have been driven down so much over the years there’s really little difference between the two approaches.
Patterson’s contention, though, was that the little more you pay for in-person qual brings with it a significant improvement in quality of output. Meeting people and talking with them face-to-face offers “unique benefits”, he said, not least the opportunity to discern non-verbal indicators of a person’s underlying emotions which they may have difficulty articulating – if they are even aware of them.
Tom De Ruyck of InSites Consulting, for the onliners, questioned this advantage, saying it was “an illusion to think that either online or in-person methods can get into people’s minds”. Online, he said, with its applications for sharing text, pictures and video, offers “more angles” from which to observe consumers to produce “a more holistic vision”. Chatting with them for two hours in a focus group can’t compare, said De Ruyck.
Both he and Julie Wittes Schlack of Communispace had earlier remarked how online communities allow for long-term relationships to be created with respondents and how this, over time, would lead to deeper engagement with the research and ultimately more in-depth consumer understanding.
“I would agree that [online] you can lose some of the nuance of a live interview,” said Wittes Schlack, but that loss is “more than compensated” by the longitudinal relationship.
But to compare online communites to offline focus groups is to compare “apples to pears” – synchronous to asynchronous studies – said moderator Joanna Chrzanowska. Independent quallie and in-person advocate Geoff Bayley here spied a weakness, and pounced.
Turning their argument against them, Bayley said the onliners were making the point that depth comes over the long-term, when the issue is that clients want speed and depth. Razor Research’s Eleanor Atton chimed in, damning online with the faint praise that it “can be uniquely powerful for getting snapshots of people’s lives”.
Later, as the debate turned to the issue of resistance to online methods, Atton challenged De Ruyck’s claim that traditionalists are loth to use online qual because “people fear the unknown”. She said that, to the contrary, she herself uses online methods and sees them as a complement to her in-person work – a point similarly made by Patterson at the outset of the debate.
Contentiously, Atton said: “We would challenge the onliners that they have a fear of offline.” Despite a spirited defence, it seems, the offliners couldn’t sway those listening. Chrzanowska was reluctant to call a clear winner from the online poll conducted at the conclusion of the debate, though it looked like a majority sided with the view that qual’s future was an internet-enabled one.