Expert Analysis: Is Sentiment Analysis an 80% Solution? - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Software // Information Management
News
3/29/2010
04:13 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Expert Analysis: Is Sentiment Analysis an 80% Solution?

Sentiment-analysis technologies aren't perfect. But what critics are missing is the value of automation, the inaccuracy of human assessment, and the many applications that require only "good-enough" accuracy.

Applications Beyond Monitoring

"Sentiment is nice to know, but up until today I haven't heard a single commercial application for it," writes Mikko Kotila. He's like Miranda in Shakespeare's Tempest, who grows up on an island with only her father, Prospero, for human company. Miranda sees survivors of a shipwreck and exclaims, "O brave new world / That has such people in't!" Prospero's response: "'Tis new to thee."

There are many sentiment-analytics applications with proven ROI; others will prove themselves soon if they haven't already. You just need to look beyond social/media monitoring and measurement to business analytics and operations, to functions that such as customer experience management. I'll cite just a few examples, provided by vendors. These examples are focused, but they're representative of experience shared by many users.

First, ROI isn't always measured in dollar terms. Clarabridge reports that for customer Gaylord Hotels, "Automated analysis of survey comments showed that customer experience was measurably enhanced when bell services staff accompanied lost guests to their destinations within a resort, as opposed to merely pointing them in the direction they needed to go. This insight led management to incorporate the process improvement into what Gaylord calls the 'Service Basics.'"

Other commercial applications range from innovations such as an "emotional tone checker" from Lymbix, which targets corporate communications functions, and a similar application from Adaptive Semantics that automates comment moderation for user-generated, online content; to use by consumer-goods companies such as Unilever for marketing-campaign analysis, as I describe in a 2008 article, Sentiment Analysis: A Focus on Applications.

As I say, there are many applications of sentiment analysis that are already delivering business value. For those many, disparate applications, how accurate is accurate enough? Requirements depend on business needs, and "good enough" is a very valid concept. If automated sentiment analysis can help spot one dissatisfied customer who would otherwise have been missed, that's a start. There's always room for improvement, however, and higher accuracy is certainly on the to-do list, for researchers and vendors as hinted at above, and also for practitioners who apply the technology.

Improving Accuracy

My 2008 article describes how consultancy Anderson Analytics applied text technologies as one of three parts of a "triangulation" process seeking to understand consumer sentiment surrounding the Unilever's Dove-brand pro.age marketing campaign. It's a truism, common wisdom, that you can usually boost analytical accuracy by applying multiple methods. You can link text-extracted information to transactional and operational records, for instance, to determine what service center staff our French Toshiba critic interacted with, or for a survey, you can collect sentiment both using numerical rating scales and free text. And of course, you can also explore steps to improve accuracy with particular techniques.

"Coming to a Theater Near You! Sentiment Classification Techniques Using SAS Text Miner," a paper by Jake Bartlett and Russ Albright of SAS Institute, describes the authors' attempt to improve accuracy in sentiment analysis of a collection of movie reviews. They looked at four variations on out-of-the-box use of SAS Text Miner and found they could reduce misclassifications by up to one-third, to 14.4%. Now of course, you don't just throw techniques at problems and hope they will help. Bartlett and Albright cite weightings, synonyms, and part-of-speech tagging and more complex extraction and feature manipulation techniques, but they suggest that part-of-speech tagging may not be effective if you don't have well-formed, grammatical text.

Learning More

I've presented what I hope is a systematic, balanced appraisal of the state of the accuracy and usefulness of automated sentiment analysis. If you'd like to learn more, there are a number of places you can continue. Check out:

Finally, a plug: The Sentiment Analysis Symposium, April 13, in New York, will look at a broad set of sentiment-analysis approaches, business applications, solutions, and benefits. Please consider joining us for the conference.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
3 of 3
Next
Comment  | 
Print  | 
More Insights
Commentary
The Best Way to Get Started with Data Analytics
John Edwards, Technology Journalist & Author,  7/8/2020
Slideshows
10 Cyberattacks on the Rise During the Pandemic
Cynthia Harvey, Freelance Journalist, InformationWeek,  6/24/2020
News
IT Trade Shows Go Virtual: Your 2020 List of Events
Jessica Davis, Senior Editor, Enterprise Apps,  5/29/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Key to Cloud Success: The Right Management
This IT Trend highlights some of the steps IT teams can take to keep their cloud environments running in a safe, efficient manner.
Slideshows
Flash Poll