Upcoming ESOMAR 3D presentation: Tell me what you want, what you really, really want

Tuesday, July 26th, 2011

3D DIGITAL DIMENSIONS 2011
(ONLINE + SOCIAL MEDIA + MOBILE) RESEARCH
MIAMI / 26 – 28 OCTOBER

.
TELL ME WHAT YOU WANT, WHAT YOU REALLY, REALLY WANT
CREATING DESIRED RESULTS FROM SOCIAL MEDIA RESEARCH

Annie Pettit, Conversition and Research Now, Canada
.
This presentation will teach you how to generate the social media research results you desire regardless of what the true results are. I will demonstrate how to gather social media data from the internet using inappropriate sampling methods, and how to select the wrong pieces of data and code it incorrectly. The topics of sampling, weighting, data quality, sentiment analysis, and text analysis will be highlighted so that you can understand the full range of options for mistreating data. The ultimate goal will be to create set of data that reflects our predispositions towards a topic as opposed to reality.

Attendees are required to come prepared with a sense of humour  (i.e., I will be speaking in jest!)

MRA IMRO SMR Guidelines #1 Advantages and Disadvantages: The Conversition Response

Monday, November 1st, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is the first of a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

jdurham from morguefile
.
#1 Advantages and Disadvantages of Social Media Research
.
Without exception, every research method has unique advantages and disadvantages. Nonprobablity sampling is an issue affecting every form of market research, including social media research. Just as surveys normally can’t identify every member of a population and force a random sample to complete the survey, social media research cannot listen to people who do not contribute to social media nor those whose data is behind password protected walls. But we’ve developed methods for understanding and compensating for such weaknesses.
.
One of the unique weaknesses of social media research is the lack of individualized demographic and and geographic data. This means we must find new and different ways to learn about the unique characteristics of people talking about brands. Inferred demographics and qualitative psychographics are just some of the solutions. The lack of demographics also means that we must find new ways of sampling and weighting our results, such as using website sources, to ensure that results are properly generalized to external populations.
.
As researchers, we have a collective history that has taught us how to address and compensate for methodological issues. As business managers, these issues mean that assigning costs to business units can be more difficult – how does one invoice a local project to a global unit?
.
On the the other hand, every method has unique advantages. Some of the most distinguishing advantages of social media research includes its amazing ability to discover how people talk about a brand in an uncontaminated environment – no questions, no probing, no elicitation, just pure, raw meaning. Is the chatter highbrow or lowbrow, quick and dirty or descriptive and well-thought out?
.
In addition, the penetration of social networks, forums, video and photo sites, and other types of websites that allow users to interact with other people means that the quantity of usable data can be astounding. Well known consumer brands such as Microsoft, Nike, Starbucks, and Coca-cola have an enviable problem of working with millions of records. Other less popular brands, though, can still take advantage of social media research by evaluating competitor data and industry data.
.
Because of the volume of data and the millions of people contributing knowledge about a brand, the breadth of available data in social media research can be astounding. Where surveys are limited to 20 or 30 minutes and perhaps 40 questions, social media research is like offering a ten hour survey with thousands of directly relevant questions. The variety of topics that people talk about online is far greater than can be incorporated in a traditional survey.
.
There are far more advantages to social media research but don’t take our word for it. Try it for yourself.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

The Fantasy of Representative Samples

Tuesday, September 21st, 2010

Photo credit: chaka from morguefile.com

Come clean now. Did you ever really think there was such a thing as representative samples in market research?

Door to door surveys: Not everyone has a home. Not everyone answers the door. Not everyone agrees to complete the surveys.

Mall Intercepts: Not everyone goes to the mall. Not everyone passes by the interviewer. Not everyone agrees to complete the survey.

Online Surveys: Not everyone is online. Not everyone joins survey panels. Not everyone completes the surveys.

Social Media Research: Not everyone is online. Not everyone is on Facebook/Twitter/YouTube. Not everyone contributes content.

People are just as a stubborn as mules!

Let’s visit the concept of probability sampling. This theory was created by mathematical geniuses like Pascal, Gauss, Fisher, and Pearson who used statistics to better understand things like games of chance (coin tosses)  and genetics (molecules, cells). Let’s think now – when was the last time a penny, a skin cell, or a molecule of carbon dioxide ever had a choice about whether to participate in research?

Because human beings have free will, unlike coins and cells, they can choose whether to participate in research. Hence, the requirement for equal and independent opportunities to participate in research actually means equal and independent opportunities to be selected for and complete the research. This, market researchers never have.

But this does not mean all is lost. What this proves is that researchers are pretty incredible people. Even though we know our samples can never be representative and can never meet statistical requirements for probability sampling, we have learned through decades of careful study how to create knowledge out of noise. We know how to work around the rules and around the limitations to generate information. We know how to apply error rates, confidence intervals, and gut-instincts to end up with valid and reliable results.

Do we need the representative samples that random probability sampling provides? It sure would be nice,  but it’s not the entire equation.
.
.
Related links
Social media monitoring vs social media research: Can you see the difference?
How important is sampling? Well, how important is gay marriage?
Tracking the Mood of Americans: Use Twitter if you want to prove they’re happy

Quirks: Thoughts on sampling and weighting in social media research

Wednesday, August 4th, 2010

Social media research has officially burst onto the scene and we are pleased that Quirks has devoted the entire August issue to it. Included in this month’s magazine is an article on sampling by Annie Pettit, our Chief Research Officer. The abstract follows and if you want to read more, simply click here and head on over to page 42.

Article Abstract
When analyzing data from social media, the choice of a sampling plan depends on the purpose of the research and how those results will be used afterwards. The author discusses three options and some of the issues surrounding the use of data from social media.

Article ID: 20100806

Social media monitoring vs Social media research: Can you see the difference?

Monday, July 26th, 2010

Photo puravida from morguefile

Many people are curious about the difference between social media monitoring and social media research. The distinction is clear, and fairly easy to see if you have experience with market research.
.

Social media monitoring

• The act of reviewing and tracking social media data
• May include tracking the volume of data meeting specific criteria, possibly tracking the sentiment of that data, determining which websites are producing greater or lesser volumes of data, identifying individual who are prone to discussing a brand, interacting with individuals contributing the social media data

.

Social media research

• The application of scientific methods to social media data
• Like surveys, focus groups, and other established research methods, social media research incorporates the scientific principles that turn data into valid and reliable data sources, suitable for explaining past and current behaviour, and predicting future behaviour
• Established methods may include several of the following: developing research objectives, defining problems, defining and selecting relevant target groups and samples, applying sampling, weighting and standardized data quality methods, applying validation methods, evaluating data reliability
• Outputs include Usage and Attitudes studies, Brand Trackers, Ad Trackers, Segmentation studies, and other research traditionally using survey or focus group data

.

.

Tracking the Mood of Americans: Use Twitter if you want to prove they’re happy

Sunday, July 25th, 2010

An article in the New York Times this week discussed a research project that is attempting to track the mood of Americans using Twitter as the data source. The project involves researchers from Northeastern University College of Computer and Information Sciences and Harvard Medical School. It is certainly reasonable that a group of scientists can develop algorithms that accurately predict the mood of Americans. However, Twitter data is not simply and instantly predictive of the general population of Americans. Given that only 7% of people who are online even use Twitter, it is risky, and can easily lead to wrong conclusions.

Want to see a real example? No problem.

Let’s look at consumer opinions related to one specific product, the iPad.

1. First, we gathered thousands of opinions from across the internet, from blogs, microblogs, forums, question and answer sites, personal sites, all of which mentioned the iPad. Sites like YouTube, Blogger, Twitter, and thousands more were included.
2. Then, we categorized all of the conversations into two groups, 1) everything from Twitter and 2) the entire internet space.
3. Next, we determined the level of emotion for every online conversation. Specifically, we determined whether the emotion of the conversations was extremely happy, somewhat happy, neutral, somewhat unhappy, or extremely unhappy.
4. Finally, we created the pretty little charts that you see on the right of this page.

What’s the first thing you notice from these charts?

Not one single chart has two bars that look the same. What is the percentage of tweets that reflect an extremely happy opinion? 15%. What is the comparable number for the entire internet? 5.6%. I hope it’s not just me, but 15% doesn’t look like 5.6%, not even if the 5.6% is averaged up to 6%. There is a big difference in the percentage of people who have extremely happy opinions on Twitter vs the entire Internet.

The same trend is apparent when we look at the percentage of people who are extremely unhappy with the iPad. 11.3% of tweeple are extremely unhappy compared to just 1.9% of the entire internet space. All five of the charts lead to the same conclusions. Twitter results do not equal Internet results.

It’s not 1 to 1

Clearly, the relationship between Twitter data and total internet data is not 1 to 1. It’s impossible to gather Twitter data,  analyze the sentiment, and be confident that it represents a wide, more general audience.

Perhaps people on Twitter have more extreme opinions than everyone else; perhaps they are less likely to guard their remarks so that the more extreme opinions are shared; perhaps Twitter opinions are in fact the closest to the average American opinion. Whatever the reason, it is undeniable that the mood on Twitter is unlike anywhere else.

Prepare to be wrong. Prepare to explain contradictions. Generalize Twitter mood at your own risk.

Links that might interest you:
iPad on EvoPlay
New York Times article
Conversition on Facebook