The Fantasy of Representative Samples
Tuesday, September 21st, 2010
Come clean now. Did you ever really think there was such a thing as representative samples in market research?
Door to door surveys: Not everyone has a home. Not everyone answers the door. Not everyone agrees to complete the surveys.
Mall Intercepts: Not everyone goes to the mall. Not everyone passes by the interviewer. Not everyone agrees to complete the survey.
Online Surveys: Not everyone is online. Not everyone joins survey panels. Not everyone completes the surveys.
Social Media Research: Not everyone is online. Not everyone is on Facebook/Twitter/YouTube. Not everyone contributes content.
People are just as a stubborn as mules!
Let’s visit the concept of probability sampling. This theory was created by mathematical geniuses like Pascal, Gauss, Fisher, and Pearson who used statistics to better understand things like games of chance (coin tosses) and genetics (molecules, cells). Let’s think now – when was the last time a penny, a skin cell, or a molecule of carbon dioxide ever had a choice about whether to participate in research?
Because human beings have free will, unlike coins and cells, they can choose whether to participate in research. Hence, the requirement for equal and independent opportunities to participate in research actually means equal and independent opportunities to be selected for and complete the research. This, market researchers never have.
But this does not mean all is lost. What this proves is that researchers are pretty incredible people. Even though we know our samples can never be representative and can never meet statistical requirements for probability sampling, we have learned through decades of careful study how to create knowledge out of noise. We know how to work around the rules and around the limitations to generate information. We know how to apply error rates, confidence intervals, and gut-instincts to end up with valid and reliable results.
Do we need the representative samples that random probability sampling provides? It sure would be nice, but it’s not the entire equation.
.
.
Related links
Social media monitoring vs social media research: Can you see the difference?
How important is sampling? Well, how important is gay marriage?
Tracking the Mood of Americans: Use Twitter if you want to prove they’re happy
Comments Off
Category conversition | Tags: Tags: conversition, door to door, mall intercept, online surveys, probability, sampling, social media research,
Social Networks : Technorati, Stumble it!, Digg, de.licio.us, Yahoo, reddit, Blogmarks, Google, Magnolia.
