. TELL ME WHAT YOU WANT, WHAT YOU REALLY, REALLY WANT CREATING DESIRED RESULTS FROM SOCIAL MEDIA RESEARCH
Annie Pettit, Conversition and Research Now, Canada .
This presentation will teach you how to generate the social media research results you desire regardless of what the true results are. I will demonstrate how to gather social media data from the internet using inappropriate sampling methods, and how to select the wrong pieces of data and code it incorrectly. The topics of sampling, weighting, data quality, sentiment analysis, and text analysis will be highlighted so that you can understand the full range of options for mistreating data. The ultimate goal will be to create set of data that reflects our predispositions towards a topic as opposed to reality.
Attendees are required to come prepared with a sense of humour (i.e., I will be speaking in jest!)
Translation: Is do-it-yourself social media research good (FTW=for the win) or bad (FTL=for the loss)?
DIY research of any sort is a highly contentious topic. Though it offers significant financial savings, those savings can be completely lost if the user doesn’t have significant skills in research design and analysis. Insufficient knowledge about data quality practices will lead to wrong results. Insufficient knowledge about data validation practices will lead to wrong results. Insufficient knowledge about question design, focus group moderating, sampling, weighting, scaling, norms, and many other topics will lead to wrong results. Through training and experience, expert researchers have gained an intimate knowledge of the myriad ways that apparently good data can be contaminated and confounded leading to irrelevant and invalid results.
The decision about whether to use DIY SMR is no different than the decision to use DIY surveys. The data source may be different, the data may be processed differently, but all of the research methodology that goes into making survey results valid is the same that goes into making social media research valid. (Hey, how come we never talk about DIY focus groups?)
. What is DIY SMR?
There are different levels of do-it-yourself social media research. In its most simplest form, it can involve an individual manually reading through results from sources such as google, facebook, or twitter and analyzing each conversation one by one. Someone might conduct their own rudimentary sentiment and content analysis to determine the type of emotion and the content of the conversations. Generally, the volume of data is small because of the manual limitations of collecting and coding data.
At its most complicated level, DIY SMR involves using a third party source such as an SSR feed to collect the data, and possibly gaining access to a scoring system. A larger quantity of data can be analyzed this way.
If we put aside all of the issues concerning research quality, DIY SMR might work for you if:
you are interested in the flavor of online conversations
you want to discover new ideas
you want some ideas to beef up a survey
small sample sizes are sufficient
you don’t require precision or generalizability
you have significant research experience
.
On the other hand, DIY SMR will not work for you if
you need precise results
you need to generalize valid results to a larger population
you need to compare your results to normative category data
you need to track your brand results reliably over time
you need to compare your results to survey and focus group data
you are inexperienced with research methodologies
.
In the end, you need to make the decision that is right for you and your research objectives. As long as your intent to obtain valid and reliable answers to your SMR objective, you will make the right choice.
Almost anywhere you turn, someone is offering social media tracking, monitoring, measurement, evaluation, or some other form of analysis of social media data. How do you know whether you’re getting quality goods? Here are six things to checkmark before you get started on the journey.
Search Quality: What are the restrictions put around the data you are seeking? Are there methods in place to ensure the right data is being selected in and the wrong data selected out? If your brand is “Target, ” you need to make sure that the data is all about clothing and consumer goods (select in) and not about target practice (select out). Ask whether the data collection processes allow complex “and” and “or” searches so that data can be easily excluded and deliberately included.
Search Population: Is data being gathered from across the entire internet or just the top sites? There are pros and cons to each method. The top sites often account for up to 80% of all of the relevant data, but who’s to say whether the other 20% reflects a unique group of consumers whose voice could change how you think about your brand. You should at least know which process is being used.
Data Volume: Being blessed with millions of online records is a sweet luxury that only a few brands can achieve. But, unlike the survey world where 500 is a great sample size, this just doesn’t cut it in social media research. Most brands fall somewhere in between these two extremes, generating from hundreds to thousands of records each month. If your brand generates just a few hundred records every month, you might be more suited to a qualitative approach to SMR and some efforts towards building a greater online presence. Brands generating thousands of records each month can take full advantage of both quant and qual approaches.
Scoring Quality: There are many different methods for scoring the sentiment of online conversations. What systems are being used? Is the scoring a manual process, automated, or combination of the two? Is it dictionary based or mathematical based? How do the systems accommodate the rapidly evolving English language? How do the systems account for new and emerging slang? And all the while, you need to remember that no system, not even a human being, can achieve perfect scoring. In this world, perfect isn’t 100%, it’s only 85%.
Coding Quality: Data isn’t useful until it’s categorized into meaningful chunks of data. Knowing that overall sentiment towards a brand is “Very Positive” does nothing to help you decide whether you need to build your product in a different color, shape, or size. But this isn’t an easy process. When Earl Grey Tea gets categorized into a color, you have no hope of generating valid insights from your results. Ask about the process of data quality in the coding process. Find out whether Charlie Brown is a color.
Coding Flexibility: Your brand is unique like no other brand. Your research objectives are like no other brand. There’s no reason to assume that the coding structure any other brand uses should be the same as what you use. Beyond the obvious requirements of purchasing, recommendations, trial etc., you have specific needs. Be sure to ask about how the coding can be customized to meet your unique requirements.
.
.
Many people are curious about the difference between social media monitoring and social media research. The distinction is clear, and fairly easy to see if you have experience with market research.
.
Social media monitoring
The act of reviewing and tracking social media data
May include tracking the volume of data meeting specific criteria, possibly tracking the sentiment of that data, determining which websites are producing greater or lesser volumes of data, identifying individual who are prone to discussing a brand, interacting with individuals contributing the social media data
.
Social media research
The application of scientific methods to social media data
Like surveys, focus groups, and other established research methods, social media research incorporates the scientific principles that turn data into valid and reliable data sources, suitable for explaining past and current behaviour, and predicting future behaviour
Established methods may include several of the following: developing research objectives, defining problems, defining and selecting relevant target groups and samples, applying sampling, weighting and standardized data quality methods, applying validation methods, evaluating data reliability
Outputs include Usage and Attitudes studies, Brand Trackers, Ad Trackers, Segmentation studies, and other research traditionally using survey or focus group data
Last but not least, here is our third installment of social media research data quality blunders. Whether you prefer Coke or Pepsi, this one is sure to tickle your fancy.
.
After gathering thousands of verbatims from all across the internet, we created a word cloud that would help us determine what are the other types of beverages and brands of beverages that Coke drinkers mention a lot. Obviously, Diet Coke and Cherry Coke are bigs hits, as are Pepsi and Minute Maid. But if you’ve read the last two blogs, you know where we’re going with this.
What exactly would the word cloud look like if we only used words that didn’t actually mean Coca-Cola the delightful bubbly beverage? Prepare to look away if you’re easily shocked. Somehow, if we don’t put any effort into collecting quality, relevant data, we collect a lot of data referencing illegal drugs. Cocaine, marijuana, heroin, and meth rise to the top of the data. I can’t say for sure, but my guess is that Coca-Cola wouldn’t care to have this data affect their results. But that’s just my opinion.
On to part 2 of data quality! Ok, so BP was a bad example. Obviously, a lot of different brands and people and things will have the initials BP. It’s an isolated case. But is it? Here is another great example of how data quality begins at the very beginning of any social media research project.
The scenario is the same. We gathered thousands of verbatims from thousands of websites and created a word cloud of all things related to Apple. The usual suspects are all there. Competitive brands like HP, Hitachi, and Toshiba are well represented. And, since the iPad is the greatest discovery in all of mankind EVER, it is the most prominent feature of the cloud.
.
But wait. Isn’t there more to Apple than just computers? Here is a second word cloud we created from the very same data. No manipulations and no sneaky subsampling. What’s with all this apple pie, apple cider, and apple cinnamon deliciousness? This is simply another great example of poor quality workmanship inviting terribly incorrect confounds.
Is social media research fast? Sure it is. It’s even faster if you ignore the annoying stage of data quality.
Unlike other types of research, social media research has the potential to provide brands with millions of relevant datapoints. That is, of course, as long as sufficient work is put into gathering the right data. What exactly is the right data? If you do Google “BP,” about 200 million records will be found. And given the severity around the situation in the Gulf, the first few hundreds of pages are all about British Petroleum. We carefully gathered thousands of records that specifically related to British Petroleum and created a quick word cloud of the types of companies that were mentioned. Not surprising, energy and finance companies were top of the list.
However, just two months ago, if you had googled “BP,” chances are you would have returned very different data. Perhaps it would have been about Brad Pitt or Basis Points or Blood Pressure, who’s to say? Well, we created a second word cloud that was based on sloppy data collection. Any mention of BP in social media was collected regardless of the context. As you can see in the cloud, BP most commonly stands for Blood Pressure. This is a huge confound that absolutely must be avoided.
Imagine how the research results would be affected if blood pressure data was included with British Petroleum data. We would be astonished that fishermen were using Zocor to try and calm down after watching Inglourious Basterds.
