Who wins when automated and human sentiment analysis fight?

Monday, May 10th, 2010

Over the last couple of weeks, the social media space has engaged in extensive battles over whether human or automated sentiment analysis is better. It’s a difficult question to answer and may in fact begin with this question:

Photo credit: Grafixar from morguefile.com

“Why assume that automated systems are more accurate judges of human emotions than human beings?”

Human sentiment analysis has many advantages, the first one undoubtedly being that it is more valid than automated sentiment analysis. This is no surprise. This is of no contention. Human beings are better at sentiment analysis than automated systems. After all, it is humans who make automated systems work to begin with and humans who work day to day to improve the systems. End of story.

Of course, the story does not end there. Humans are fabulous at understanding the humor of their best friends but not quite as good as interpreting the humor of strangers. Humans understand spelling and grammar mistakes… usually. We understand grammatical errors… usually.  Humans never get tired or bored or inconsistent after coding millions of records for weeks on end. Wait… that’s not right.

The automated sentiment analysis systems that humans have carefully created over the last few decades were built to handle some of these human disadvantages. Forget human capabilities of 250 words per minute and start thinking automated capabilities of thousands of words per second. Forget how my interpretation differed from yours because I’m tired and you didn’t get the joke, and consider the 100% reliability of automated systems even after they’ve been working for ten days straight with no coffee. If you think about it, the accuracy of automated systems boosted with human intervention is quite impressive.

There is no perfect valid sentiment analysis system but there is a system that is right for you. Do you need high accuracy for a small amount of data? Then human systems are right for you. Do you need good accuracy for millions of records? Then automated systems are right for you.

So who would win the fight? I’d say there was never a fight to begin with.