Posts Tagged ‘social media research’

The Sharks and Icebergs of Social Media Measurement

Friday, March 25th, 2011

It’s all very deceiving. Social media measurement is superquick and feeds everyone’s desire for instant gratification. Want to measure something right now, lickety split? Here you go!

Unfortunately, it’s not that simple, nor do you want it be that simple. Behind every “instant analysis” are hundreds and thousands of hours of tedious, painstaking work. Let’s take a brief climb up the social media measurement iceberg and check out where the data quality sharks are hiding.

Taking much of the time, the bulk of the iceberg, and many of the data quality sharks is sentiment analysis. There’s no point in being amazed at how quick your measurements were generated if those numbers have no bearing on the true sentiment of a brand.  The data quality sharks biting at the validity of your data include things like failing to measure  “the bomb” correctly and forgetting to measure “FTW.” If these sharks aren’t addressed, your amazingly quick measurement will be wrong.

The next chunk of the iceberg that rarely gets seen is the work that goes into developing variables. The data quality sharks here include recognizing that “tasting success” and “tastes in music” aren’t discussions of how deliciously yummy something is. Behind the scenes, people must spend icebergs of time making sure those invalid mentions are identified and ignored. Otherwise, your instant measure of whether people are happy with the new flavour of Doritos will be wrong.

Now we’re actually ready to gather YOUR data. As always, you can go online and quickly gather a ton of information about Ford or Sears or Harley, but your data quality sharks try to stuff in mentions of “my mayor, Mr. Rob Ford” and “sear the meat quickly” and “my dog Harley.”  If you don’t take the time to kill those data quality sharks, your instant analysis will, once again, be wrong.

The last chunk of work goes into removing spam and irrelevant data. Of course, you can easily set up an automated system to detect and delete spam but you must always be on the lookout for new and improved spam. This data quality shark sneaks in when you least expect it. No checking means no validity.

And now, we’ve climbed to the top of the iceberg, to the last tiny percent of data work that is visible to the outside world. Enjoy your magical, instant measurement. It wasn’t that instant after-all, was it!

Related links

Forgive me, for I have sinned

Wednesday, March 23rd, 2011

Forgive me for I have sinned. Without permission, I used company property for my own personal gain.

Earlier this year, I saw some ads on the subway system for a glasses and contact lens company. The ad proclaimed, “FREE GLASSES.” I proclaimed, “HOAX.”

But it wasn’t a hoax, it was just a very effective and provocative advertising campaign. Glasses cost me at least $400 every time so free glasses were extremely tempting. But, I also didn’t want want to be mentioned on the evening news as “Annie was one of 300 stupid people who fell for the glasses scam.”

So, after seeing ads for really cheap glasses from the same company a few more times, I broke down. I, um… borrowed company property. I used Conversition’s proprietary social media measurement system to collect a few thousand tweets and blogs, and forum and photo comments that mentioned the company and their ads. You see, thousands of independent opinions from people who have already conducted business with the company couldn’t be wrong.

The system cleaned the data of irrelevant spam, scored the sentiment on a continuous scale, and coded all of the comments into the one thousand, most relevant variables. Just a couple hours after I finally decided I wanted in on cheap glasses, I had valid and reliable, quantitative and qualitative social media data at my fingertips. And here’s what I learned.

  • When people talked about whether they would recommend ClearlyContacts, 54% of those recommendations were positive, as opposed to middle of the road neutral or negative.
  • 44% of comments about pricing were positive
  • 37% of comments about the choice of products were positive
  • 57% of comments about the customer service were positive
  • 50% of comments about the shipping were positive
  • Overall, about 46% of people liked this phantom internet company and less than 6% did not like the company.

.
Of course, you do have to listen to the people who aren’t happy if you want a balanced opinion, so I also checked out what the biggest complaints were.

  • 11% of comments about the frames were negative
  • 10% of comments about the lenses were negative
  • 8% of comments about the check out process were negative

.
Given survey research norms call for about 10% to 15% of results to fall into the negative side of things, these negative comments were not out of the ordinary and were nothing for me to be concerned with. And since norms dictate that about 30% of results fall into the positive side of things, I felt even better.

Social media research did its job well and proved to me that the company was legit! So I bought a way cool pair of glasses for way cheap. Actually, because they were so cheap, I bought two pairs. Like ‘em?

.___________..________.
.
.
Read these too

Global Voices, Local Choices: A Social Media Research Case Study

Monday, March 21st, 2011

Every country has their own brands and services. The thing about the internet though, is that we can read about services that are available in other countries. No matter whether the options are available to us, we can see who is being offered better, or worse, products than we are and demand those things where we live. This case study, in the telephone/television/internet space, illustrates consumer opinions towards several services, some of which are available only in Canada, only in the US, or in both countries. Let’s compare AT&T, Rogers, Comcast, T-mobile, and Virgin.

.
.
Related links
Justin Bieber vs. Katy Perry vs. Metallica: A social media research case study
Fail #1 Google, Fail #2 Superbowl Tibet, and Groupon Hangs in the Wings
Which pain relief medication for which pain? An SMR case study #MRX
Snickers Bars and Breakfast: A social media case study
Battle of the Burgers: Big Mac vs Whopper

Which pain relief medication for which pain? An SMR case study #MRX

Friday, March 11th, 2011

Your local pharmacy is chock full of pain relief medications and every box says it’s best for something different. This case study illustrates how real people think about five brands, including Tylenol, Advil, Aleve, Aspirin, and Excedrin. We explore the psychographics associated with each brand as well as what types of pain lead people to use specific brands.

Listen to what hundreds of thousands of people have to say in the online space.

.
Related links

Quick and Dirty or Slow and Careful: Conversition’s #NewMR Presentation

Wednesday, March 9th, 2011

Social media analysis is so simple. Do a keyword search on Google, find some data mentioning your brand name, whip up some cool analytical results. But wait – have you ever wondered why different people end up with such different results when they research or monitor the same topic? This presentation, created for the #NewMR festival on Listening, will share just a few of those reasons with real examples. Mmmm…. real data!

.
Related posts

.
Conversition on LinkedIn

The #Klout of @Klout

Wednesday, March 2nd, 2011

Not many folks liked the popularity contests of highschool, but we still like to be popular, whether in the face-to-face space or the online space. Klout is one of the available measures of online influence and it takes into account the size of your engaged audience, the likelihood that people will retweet or message or like you, and how influential your engaged audience is.

The folks at Klout have even built an extension for the Chrome browser that lets you see the Klout score of each tweeter in your stream. According to the screencap of our twitterstream, these research folks are doing pretty well with scores ranging from 42 to 66.

But let’s not worry about what Klout does, or how it does it or even whether it is actually accurate it is. Let’s consider, instead, people’s perceptions of the tool. Do we love it or hate it?

First of all, the obvious notes – This dataset does not include tweets or blogs or messages sourced from Klout or KloutPerks. And, the measurement was not confounded with Klout terminology such as “specialist” or “influenced” or “perk.” And lastly, we’re basing our results from a random sample of about 5000 messages that were written in 2011.

Now that we’re all on board, let’s look at some numbers.

.

Overall, do people like Klout?
Overall, people are fond of Klout. Around 40% of people like it while only 6% of people dislike it. But, that leaves an awfully large group of people who are undecided, unsure, or couldn’t care less. Common ways to talk about Klout include  “best,” “love,” “thank,” “amazing,” “impressive,” and “fun” but we also speak of “sadly,” “dislike,” “wtf,” “rant,” “scary,” and “shame.”
.
Among people recommending tools, is Klout recommended?
We’re doing pretty good here! About 66% of people would recommend Klout. In fact, they say they not only “recommend” Klout, but they “endorse it,” “cheer for it,” and are “addicted to it.” That does leave another 30% of people who are pretty neutral about the whole thing, as well as a 4% who would never recommend it.
.
Among people talking about favorites, is Klout their favorite influence tool?
In this case, we have a ton of people talking about Klout being their favorite tool. Being the favorite of 67% of people is pretty darn good. But, of course, about 32% of people don’t really care, and a tiny percent say it’s not their favorite at all.
.
Among people talking about accuracy, do people feel that Klout is accurate?
Lastly, and possibly the most important item for a small number of researchers who are most concerned with validity, we come to accuracy. In this case, only 14% of people gave Klout a positive score. And, more than 22% gave Klout a negative score.  While many people say Klout is “accurate,”  even more say it is “wrong” or it’s a “fail.”

So, it seems that regardless of accuracy being less than desirable, people are still fond of the tool.  Our Klout score is 50. What’s yours?
.
.
Related links
#MRX MRA IMRO SMR Guidelines #10: Qualifications
Fail #1 Google, Fail #2 Superbowl Tibet, and Groupon Hangs in the Wings
The Fantasy of Representative Samples
WeightWatchers or Jenny Craig: Social Media Research Knows the Answer
Who am I? Contributor, author, participant, responder?
Super Bowl = Beer + Wings + Hooters

Snickers Bars and Breakfast: A social media case study

Tuesday, March 1st, 2011

The breakfast wars have begun! It doesn’t matter if breakfast was never on the menu. It is now. This social media research case study illustrates how three restaurants have taken on the breakfast category in three different ways: successful, questionable, and unnoticeable. Guess which word reflects Tim Hortons, McDonalds, and Subway!

.
.
Related links

Battle of the Burgers: Big Mac vs Whopper
Justin Bieber vs. Katy Perry vs. Metallica: A social media research case study
Fail #1 Google, Fail #2 Superbowl Tibet, and Groupon Hangs in the Wings
Word of Mouth 2.0: Old Spice, Double Rainbows, and the Starbucks Logo

Is Groupon Still on a Time-out? Social Media Research Reveals All #MRX

Friday, February 25th, 2011

Alright, so Groupon screwed up with their Super Bowl commercial. Tweeps were furious, many of them indicating they planned to unsubscribe from the group coupon service. But, as most of us know, actions speak louder than words and it made us wonder – did the commercial REALLY have any negative impact on Groupon?

The first chart below illustrates the volume of online conversations related to Groupon starting at January first and running until today. You can clearly see the spike when the commercial aired and people tweeted their thumbs off with fury. Then, you can see how the volume of conversations slowly returned back to a normal level.  In terms of just plain volume of conversations, it seems that we got over the offense pretty darn quick.


Obviously, though, the volume of conversations about a brand has pretty much nothing to do with how people feel towards a brand.  Videos and commercials and memes can be become viral because we hate them as well as because we love them.

With that in mind, we measured the sentiment towards Groupon over the same time frame. The following chart illustrates the percentage of comments about Groupon that were positive (the top black line) and the percentage of comments that were negative (the bottom blue line). Again, you can instantly see when the Super Bowl commercial aired. Up until that day, about 40% to 50% of conversations about Groupon were positive and about 3% to 7% were negative. Then, suddenly, only about 25% of conversations were positive and about 15% were negative.

But wait, we’re still angry with Groupon aren’t we? So why, pray tell, has the percentage of positive messages returned to 40% and negative messages to 5%? Sounds to me like the tweeters were just shooting off at the mouth. Were you one of them?


.
.
Related links

Join Us at the NewMR Virtual Festival – Listening is the New Asking #MRX

Thursday, February 24th, 2011

We are pleased to once again be presenters at a NewMR virtual festival! This virtual event looks at different aspects of buzz listening, blog monitoring, and computer assisted qualitative data analysis. You can read more about the event here.

The Event will be held as three two-hour sessions, each session being targeted at a different time zone. Annie Pettit will be speaking at the end of the third session which is 3pm to 5pm New York Time (EST) or 8pm to 10pm (GMT). .

Annie Pettit, PhD is the Chief Research Officer of Conversition Strategies. She has more than 15 years of experience as an online market researcher and specializes in data quality and social media research. Annie is a member of the CASRO, MRA, and ESOMAR social media research committees. Annie was previously the VP of Online Panel Analytics at Ipsos. Her expertise in research methods and data quality has been highlighted through numerous conference presentations, including ARF, CASRO, MRA ,MRIA, NetGain, and IIR. She has also published numerous articles in both professional and refereed magazines and journals.

.

Presentation Summary

Quick and Dirty or Slow and Careful: Social Media Data Speaks

Social media research seems like a quick and easy solution to surveys. The data is instantly available to anyone whether you are a researcher and whether you have access to any specialized tools. But, this paper will demonstrate, with real data, how your research results can be negatively affected if you don’t take care at various stages of the research process. We will evalute how results are affected when care isn’t taken to develop the initial search set, to create the variables being measured,  and to select the websites being reviewed.

RW Connect: Privacy and Ethics in Social Media Research #MRX

Wednesday, February 23rd, 2011

Privacy and ethics in social media research continue to attract the attention of established research organizations. Read about ESOMAR’s efforts to contribute to the discussion via a social media research committee that includes Annie Pettit, quoted below.

Privacy and ethics in social media research

Research Challenges & Issues — By Manfred Mareck on February 22, 2011 4:10 pm

“Most consumers are aware that their online conversations could be monitored but a small percentage may not understand that this is the case. A small percent of two billion online users is a lot of people who would be surprised, and possibly embarrassed or offended, that their information is being shared in an arena outside of what they originally intended”, says Annie Pettit, chief research officer at Toronto-based Conversition Strategies and member of the guideline project team. “Whilst ultimately consumers should always protect themselves this is not the ethical standard that market researchers can align themselves with and we must always work diligently on the contributors’ behalf.”