FEATURE1 March 2010

All in good time

Semiotician Chris Arning ponders what might happen to the MR industry when we have a web that can measure meanings and decode concepts.

Currently most data on the internet can only be read and understood by people. If you type a query into a search engine, it presents you with results that contain the words you typed. But did you know that every keyword used on the internet has an average of 2.7 meanings attached to it? Thought not. Neither do most basic search engines. Instead they work by blindly trawling through documents and clumsily pulling up only exact matches, which is a haphazard way to go about things.

The semantic web is one of the names being given to the next evolution of the internet – Web 3.0, if you like. Semantic simply means ‘related to meaning’, so the semantic web is one based on the meanings that underlie data, using technology that can understand language and context. It’s an internet that can think for itself and knows what it’s talking about.

Organisations as diverse as the BBC, the UK government, the CIA, Reuters and Google are already investing in making their data more searchable and combinable in ways that can add value. Semantic advertising, too, is already with us, allowing pages to match ads to editorial with greater precision and sensitivity. But this is only the tip of the iceberg - semantic technologies promise to revolutionise the way companies and consumers use data.

At the moment people are intrigued by the buzz around semantics, but unsure what the fuss is about or why they should get involved. There are privacy concerns and a need to educate business on what it’s all about, but the likelihood is that some form of semantic web will evolve in the next decade. This is, first, because we are drowning in a sea of data that demands more sophisticated systems of organisation, but also because the more semantic technology gets used, the more useful it becomes.

What does it mean for market research? I have picked out five outcomes that can be extrapolated from technologies available today, mostly from the interaction between semantics and other emergent trends. There may be opportunities for market research businesses to add revenue streams and improve quality – but there will also be threats.

1. Semantics and the (very) long tail

Imagine that a wine buff could bypass large retailers and big winery brands altogether and find a tiny vineyard in Chile, Georgia or even the UK that could bottle up a case of the perfect grape variety and send it over for dinner that weekend. At the moment these wineries are not online and have no incentive to go online, because they would never be found.

Semantic search engines are more sensitive – they can combine granular data sets in a way that allows them to pick up on finer-grained aspects of products. This will highlight niche products that were previously invisible to search engines. The web of data could drive the monetisation of niche artefacts and less popular product lines – the ‘long tail’ as described by Chris Anderson in his book of the same name. Making machine-readable data available to the semantic web could provide a boost to whole cottage industries. This could, in return, drive consumer demand for uniqueness, quirkiness and greater authenticity.

New value propositions could come to the fore and bypass price comparison sites as gateways to purchase. Martin Hepp at the University of Munich has devised a system that allows for ‘deep comparison shopping’ by making more individual products available online. This is important because it means that semantic extraction can do a better, more thorough job of matching consumer preferences to miscellaneous offerings. This may change the way companies market, which will in turn necessitate more subtle and sophisticated ways of segmenting the audience for more niche service offerings.

2. Semantic CRM

Imagine that in the future all shoppers are armed with portable reader devices or perhaps have them integrated into their mobile consoles. As they browse shelves they zap RFID-tagged items and instantly bring up a dashboard of information pertaining not just to the product type but to that very pack, second from the left on the third shelf down.

Semantic identification allows consumers to check a product’s trajectory through the supply chain, provenance of ingredients, names of key suppliers, food miles travelled and carbon footprint. As the world’s resources run out and the triple bottom line becomes more of a hot topic, offering transparency and proprietary data could become a discriminator in retail. In addition (and a German retailer is already piloting this) the device will give them real-time recommendations on products that go well with this item or other promotions.

But the data flows both ways: as long as consumers give their consent, millions of these interactions every day could provide a treasure trove of detailed behavioural data and buying preferences that would give a further boost to customer relationship marketing. It also gives a company another opportunity to engage with a valued customer.

3. Semantic usurpers

Imagine a world where market research agencies no longer exist, a world where leaner, sharper, hungrier players have more readily grasped the implications of the semantic revolution. There are organisations out there that have already developed expertise at interpreting data with the massive scalability that semantics brings. It would not involve a massive leap for them to apply their skills to market intelligence.

Instead of sitting hunched over a coding grid scrawled over with text, let a semantic ‘reasoner’ take the grunt work out of analysis. This is an exciting application for semantics. Large reasoning engines programmed with sophisticated algorithms and vast processing power could trawl through tagged data in a matter of seconds to determine if factor A (say, the purchase incidence of analgesics) is governed by a factor B (say, proximity of a given person to aircraft flight paths). Or it might be that you want to merge two research reports from different agencies. Perhaps a hitherto-unidentified sub-segment of the population can be seen to over-index on purchase of a certain item or show a tendency towards a certain set of relevant attitudes. Bingo: an automated insight.

Ontoprise, a German semantic consultancy, already performs operations for chemical companies that need to determine technical compliance across millions of factory parts. The same principle could eventually be applied to usage and attitudinal data to return surprising inferences visually, in a trice.

News wire services could also start to encroach on our territory. Open Calais is an initiative by Thompson Reuters based on semantic technologies designed to parse, sift and sort through news stories in order to deliver a bespoke news feed. Krista Thomas, VP of communication, calls one open-source platform a ‘content triage’ streaming news stories into different categories for easier browsing. This principle could be applied to online insight gathering – dynamic siphoning of insights straight from the data cloud to clients’ inboxes.

4. Semantic recruitment

Imagine field agencies that live on your desktop and require no coaxing phone calls. No more rogue respondents or embarrassing groups that were so off-spec you thought the participants belonged in the toilet paper group next door.

The semantic web allows models – ‘ontologies’ – to define and describe an area of interest and how its elements relate to one another. One such ontology, called FOAF (Friend of a Friend), allows an online author to describe all the relationships he has with others through a lattice of semantic links. Nodes can be classified to distinguish between siblings, colleagues and acquaintances and to show the strength of connections, and whether they are reciprocal. Thus a given user could be profiled for research purposes according to professional expertise, interests, social networks and closeness of bonds.

The other benefit for research agencies would be the potential for online recruitment. There is no reason that consumers could not opt in to fill out attitudinal and behavioural data that links in to their URI (‘uniform resource identifier’) and other semantic profiles. This would enable a recruiter to determine whether someone would have a pre-disposition to a certain mode of behaviour or be susceptible to this or that offer. It would also allow group or panel quotas to be filled through referral via an ontology such as FOAF.

But the reliance of semantic data on URIs – unique codes attached to bits of information to identify them – could prove to be a double-edged sword for MR. While it would allow us to clean up sampling, when applied to human beings URIs could spell the end for easy anonymity online, which raises difficult civil liberties questions.

5. Semantic spies

Imagine that you have an online valet that not only furnishes you with shortlists of items it thinks you want to buy but can also arrange hospital appointments and meetings with travellers you might share interests with for an upcoming trip. Now imagine that market research has a way of spying on this online valet and understanding how he thinks.

The holy grail for the semantic web enthusiast is a world of tiny online bots – intelligent agents running errands for human users. Semantic web expert Steve Ardire gives the hypothetical example of looking for a companion with whom to converse while on business in Ireland. They have to live in Dublin, be within a kilometre of the hotel and want to discuss James Joyce. An intelligent agent quickly identifies all the matches by pulling data from personal network profiles, GPS and LibraryThing (a site where you can share your book catalogue) and returns a number of matches for you to search through.

The implications of ordinary consumers having these ‘digital valets’ would be huge for market research, effectively passing the purchase process to machines. Once these bots learn their master’s preferences over time and are trusted to perform more tasks, wouldn’t market research be expected to somehow interpolate itself in the procedure, in order to find out how the valet bots reasoned and why certain products did not make their shortlist? The response is to develop ‘snitch bots’ that could track and report upon the bots’ activity and the metadata they throw up on their navigations through the web.

The semantic revolution
The semantic web promises to revolutionise marketing and market research. Michael Marshall, creator of search engine marketing tool SemLogic, writes: “The semantic web will be better for consumers because they will more easily find products and services that provide precisely what they need. Additionally, the corporate community stands to benefit by spending less energy and time pursuing the wrong prospects and marketing to the wrong channels”. Marketers are already looking to data to provide return on investment.

It would seem that the marketing function as a whole will need to incorporate IT into its heart. On the one hand, data will be more meaningful in itself and new inferences will be up for grabs but, on the other, effective interpretation of these inferences may require new skills. So what is the position of market research in this new world?

The migration of enterprise to semantic standards will bring opportunities and threats to market research, and those who ignore the challenge will become vulnerable. MR agencies should be prepared to bolt on semantic capability, partner up in syndicates and affiliate with semantic enterprises in order to remain competitive.


How the semantic web works

?The semantic web works by identifying ‘entities’, and giving them a ‘uniform resource identifier’ (URI), a unique number tied to that particular thing. In reality, the entity that a URI refers to might be a person, company, place, product, book, webpage or something more abstract like an idea, concept or theme. Typed tags are then attached to these entities to describe their attributes and relationships with each other in a format called RDF (resource description framework). The idea is to make data across documents of all types anywhere on the internet interoperable – making the web a huge dynamic database.

 URIs and RDF tags allow publishers to describe the relationships between things within documents. In order to work with this information, computers need to be taught how to interpret and cross-reference the concepts involved – which is where ontologies come in. Ontologies are simply models used to teach machines how to classify entities and their interrelationships.

For instance, an ontology could be programmed to define Star Wars as a film. Another ontology defines a ‘director’ as a person who has a connection with a film. This allows all data on George Lucas (who has been identified as a person, rather than a song title or restaurant name) to be inferred as a relevant piece of data, and bundled into the results to any query on Star Wars or blockbuster movie directors of the 1980s, which can then be culled from Wikipedia, IMDb or any Star Wars fan site. Instead of just giving us a bunch of results containing the words star and wars, the semantic web works out what those words mean and how they relate to all the other information out there, and as a result bring up much richer data.

There are already thousands of ontologies governing all sorts of relationships, and the more layers there are, the more sense the semantic web can make out of all that data.

Chris Arning is a freelance consultant specialising in semiotics. His background is in qual research and brand consultancy, and he was most recently head of semiotics at research agency Flamingo International.

2 Comments

14 years ago

What a radical view of the future. Chris, if half of your projections become true, the world of true one-to-one marketing will be here. An excellent thought piece, Frank

Like Report

14 years ago

Thanks for mentioning GoodRelations! For those interested in making E-Commerce on the Web of Linked Data a reality, please find developer links below. The most important thing to understand that the vision described in this article is not from the far future, but will rather materialize in the next few months. All bits and pieces are there! Project page: http://purl.org/goodrelations/ Resources for developers: http://www.ebusiness-unibw.org/wiki/GoodRelations Webcasts: Overview - http://www.heppnetz.de/projects/goodrelations/webcast/ How-to - http://vimeo.com/7583816 Recipe for Yahoo SearchMonkey: http://www.ebusiness-unibw.org/wiki/GoodRelations_and_Yahoo_SearchMonkey

Like Report