NEWS13 May 2010
All MRS websites use cookies to help us improve our services. Any data collected is anonymised. If you continue using this site without accepting cookies you may experience some performance issues. Read about our cookies here.
NEWS13 May 2010
US— A study described by its authors as “encouraging” – but which may be regarded differently by market researchers – has found that an analysis of messages posted on Twitter yielded similar results to opinion polls tracked over a two-year period.
A team from Carnegie Mellon University’s School of Computer Science and Tepper School of Business in Pittsburgh, Pennsylvania, looked at surveys of consumer confidence and presidential approval published by various organisations during 2008 and 2009 and found that they correlated to a ‘sentiment ratio’ based on words and phrases used in tweets in the same period.
“While our results vary across datasets,” they write in a paper published at a social media conference later this month, “in several cases the correlations are as high as 80%, and capture important large-scale trends.”
“The results highlight the potential of text streams as a substitute and supplement for traditional polling,” they concluded.
The team also looked at whether Twitter data was predictive of poll results, but found that results were very different at different times. This could be a result of the changing nature of Twitter, which grew vastly in usage during the period, the researchers said, but it also suggests more work is needed to understand how events and the passage of time influence how words are used.
A 72% correlation was found between Twitter sentiment and job approval ratings in the period after Barack Obama took office in 2009, although there was “no substantial correlation” with results of voting preference polls in the lead-up to the election. It remains unclear, the team said, how the names of election candidates should fit into a model of voting intentions.
“We find that a relatively simple sentiment detector based on Twitter data replicates consumer confidence and presidential job approval polls,” the researchers said. “While the results do not come without caution, it is encouraging that expensive and time-intensive polling can be supplemented or supplanted with the simple-to-gather text data that is generated from online social networking. The results suggest that more advanced NLP (natural language processing) techniques to improve opinion estimation may be very useful.”
They said that sentiment analysis could be useful for modelling traditional survey data and as a stepping stone toward “larger and more sophisticated applications”.
The paper, by Brendan O’Connor, Ramnath Balasubramanyan, Bryan Routledge and Noah Smith, is available online here.
4 Comments
Scott Jackson
15 years ago
For an interesting (and surprisingly research-informed) discussion on this story check here: http://science.slashdot.org/story/10/05/11/2245236/Using-Twitter-Data-To-Approximate-a-Telephone-Survey?from=rss
Like Reply Report
Paul Vittles
15 years ago
Of course there will be a broad longitudinal relationship but at the point when it really matters, and as a predictive tool, it's probably useless. I recall getting my first car with an onboard computer. It could calculate how many miles the car could do on the fuel in the tank...until, that is, it got to about 50 miles out and then it stopped working because it wasn't sensitive enough when the fuel level was low. That's right, at the very point it became potentially useful it was useless. Well done again to the UK pollsters on such a fantastically accurate exit poll. An accurate prediction at the very point when we needed to know the result. I enjoyed watching the politicians on early shift saying that the exit poll was wrong, a waste of time, you should abandon it and save money, sack your pollsters, etc - and then watching it pan out virtually spot on.
Like Reply Report
Christine
15 years ago
Any clues on where the paper is buried?
Like Reply Report
Annie Pettit
15 years ago
I'm not sure why this news is surprising. Why shouldn't opinions from people predict people's opinions? This is what market researchers do. Like any survey research, data sourced from social media which is then processed according to strict fundamentals of marketing research techniques should be able to provide predictive information. Annie Pettit, PhD, Chief Research Officer Conversition Strategies @LoveStats
Like Reply Report