5 Ways to Fool An Automated Sentiment Analysis System

February 9, 2011 | Comments Off

Perfect is a wonderful thing. We seek it out in every aspect of our lives. The perfect meal, the perfect house, the perfect car.  But every meal seems to come with onions (which I hate), and my house doesn’t have 20 foot fresco ceilings (which I love), and my car is severely lacking in an automated driver. I guess what I’m saying is no matter how much we seek perfection, it’s impossible to find.

Sentiment analysis is the same. We really, really want perfect but it’s impossible to achieve. Any time we try to evaluate the validity of a sentiment system, we are met with failure because seeing just a few incorrect scores immediately convinces us the entire systems has failed. So, to make sure we’re clear on how to fool an automated system, here are some quick tips for you to try.

  1. Use brand new, like, totally wicked slang that U know is FTW but most people DK.
  2. Use, in any form, a type of phrasing, which, on account of those whom others would believe they are more intellectual persons, might possess phrasing that could, potentially, be viewed as awkward grammatically
  3. Use uncorrect spelling which effects you’re interpretation of they’re opinion
  4. Use phrasing that is not, you know what I mean, direct
  5. Analyze any of the 15% of conversations that humans are unable to score correctly. Those conversations include ones where the person doing the scoring doesn’t know the newest slang, or doesn’t understand a complicated word, or  misread/misinterpreted the grammar. In other words, any of the first 4 items on this list.

The point is that when humans score sentiment manually, it can’t be perfect and when automated systems score sentiment, it can’t be perfect. People will get the scoring wrong about 15% of the time and machines will get it wrong at about 25% of the time.  But, what machines lack in quality, they more than make up for in quantity. For, 1 million records scored at 75% accuracy is far more useful than 100 records scored with 85% accuracy.

We're sorry, but comments are closed.

No Responses to  “5 Ways to Fool An Automated Sentiment Analysis System”

By submitting a comment here you grant Conversition Social Media Market Research a perpetual license to reproduce your words and name/web site in attribution. Inappropriate comments will be removed at admin's discretion.