Report this comment to a moderator

Tracking online word-of-mouth: The people vs machines debate

Let me comment the subject from the engineering point of view. It seems to follow from the great discussion above that there are three main requirements as to the monitoring online media: real-time quickness, accuracy and understanding the meaning of the discourse monitored. As I see, we all agree, that the third needs specific human abilities and involvement in the monitoring process. So, the problem is how to construct human-operated Monitor accurate enough and fast enough. What does it mean "accurate enough"? What level of accuracy we need? Some experts seem to overestimate the weight of accuracy, and compete to attain it as close as possible to 100%. However, accuracy is costly. It costs both work and time. 100% accuracy is nonsense from the economic and technical point of view. Problem of optimum accuracy is similar to the old problem of quality assurance in manufacturing sector. Industrial statisticians made great progress when they replaced 100% quality control with the Statistical Process Control (SPC), based on the study of only the small fraction of pieces (samples). Some experts of sentiment analysis of online media texts urge to collect 100% of utterances of positive or negative opinions in the texts examined (sorry for simplification) in order to attain the highest possible accuracy. They apply strong computer applications to do that and attain the accuracy of 70-90%. This is not necessary. It is not so hard to attain 97% accuracy of the sentiment's changes measurement (on 0,95 confidence level) using SPC approach, provided that the results of sophisticated discourse analysis (human work!) are implemented in the Monitor. In this way the Monitor can be built as human-operated and computer-aided tool and process for real-time monitoring.

12-Jul-2009

11:38 am