Almost anywhere you turn, someone is offering social media tracking, monitoring, measurement, evaluation, or some other form of analysis of social media data. How do you know whether you’re getting quality goods? Here are six things to checkmark before you get started on the journey.
Search Quality: What are the restrictions put around the data you are seeking? Are there methods in place to ensure the right data is being selected in and the wrong data selected out? If your brand is “Target, ” you need to make sure that the data is all about clothing and consumer goods (select in) and not about target practice (select out). Ask whether the data collection processes allow complex “and” and “or” searches so that data can be easily excluded and deliberately included.
Search Population: Is data being gathered from across the entire internet or just the top sites? There are pros and cons to each method. The top sites often account for up to 80% of all of the relevant data, but who’s to say whether the other 20% reflects a unique group of consumers whose voice could change how you think about your brand. You should at least know which process is being used.
Data Volume: Being blessed with millions of online records is a sweet luxury that only a few brands can achieve. But, unlike the survey world where 500 is a great sample size, this just doesn’t cut it in social media research. Most brands fall somewhere in between these two extremes, generating from hundreds to thousands of records each month. If your brand generates just a few hundred records every month, you might be more suited to a qualitative approach to SMR and some efforts towards building a greater online presence. Brands generating thousands of records each month can take full advantage of both quant and qual approaches.
Scoring Quality: There are many different methods for scoring the sentiment of online conversations. What systems are being used? Is the scoring a manual process, automated, or combination of the two? Is it dictionary based or mathematical based? How do the systems accommodate the rapidly evolving English language? How do the systems account for new and emerging slang? And all the while, you need to remember that no system, not even a human being, can achieve perfect scoring. In this world, perfect isn’t 100%, it’s only 85%.
Coding Quality: Data isn’t useful until it’s categorized into meaningful chunks of data. Knowing that overall sentiment towards a brand is “Very Positive” does nothing to help you decide whether you need to build your product in a different color, shape, or size. But this isn’t an easy process. When Earl Grey Tea gets categorized into a color, you have no hope of generating valid insights from your results. Ask about the process of data quality in the coding process. Find out whether Charlie Brown is a color.
Many people are curious about the difference between social media monitoring and social media research. The distinction is clear, and fairly easy to see if you have experience with market research.
.
Social media monitoring
The act of reviewing and tracking social media data
May include tracking the volume of data meeting specific criteria, possibly tracking the sentiment of that data, determining which websites are producing greater or lesser volumes of data, identifying individual who are prone to discussing a brand, interacting with individuals contributing the social media data
.
Social media research
The application of scientific methods to social media data
Like surveys, focus groups, and other established research methods, social media research incorporates the scientific principles that turn data into valid and reliable data sources, suitable for explaining past and current behaviour, and predicting future behaviour
Established methods may include several of the following: developing research objectives, defining problems, defining and selecting relevant target groups and samples, applying sampling, weighting and standardized data quality methods, applying validation methods, evaluating data reliability
Outputs include Usage and Attitudes studies, Brand Trackers, Ad Trackers, Segmentation studies, and other research traditionally using survey or focus group data
Last but not least, here is our third installment of social media research data quality blunders. Whether you prefer Coke or Pepsi, this one is sure to tickle your fancy.
.
After gathering thousands of verbatims from all across the internet, we created a word cloud that would help us determine what are the other types of beverages and brands of beverages that Coke drinkers mention a lot. Obviously, Diet Coke and Cherry Coke are bigs hits, as are Pepsi and Minute Maid. But if you’ve read the last two blogs, you know where we’re going with this.
What exactly would the word cloud look like if we only used words that didn’t actually mean Coca-Cola the delightful bubbly beverage? Prepare to look away if you’re easily shocked. Somehow, if we don’t put any effort into collecting quality, relevant data, we collect a lot of data referencing illegal drugs. Cocaine, marijuana, heroin, and meth rise to the top of the data. I can’t say for sure, but my guess is that Coca-Cola wouldn’t care to have this data affect their results. But that’s just my opinion.
On to part 2 of data quality! Ok, so BP was a bad example. Obviously, a lot of different brands and people and things will have the initials BP. It’s an isolated case. But is it? Here is another great example of how data quality begins at the very beginning of any social media research project.
The scenario is the same. We gathered thousands of verbatims from thousands of websites and created a word cloud of all things related to Apple. The usual suspects are all there. Competitive brands like HP, Hitachi, and Toshiba are well represented. And, since the iPad is the greatest discovery in all of mankind EVER, it is the most prominent feature of the cloud.
.
But wait. Isn’t there more to Apple than just computers? Here is a second word cloud we created from the very same data. No manipulations and no sneaky subsampling. What’s with all this apple pie, apple cider, and apple cinnamon deliciousness? This is simply another great example of poor quality workmanship inviting terribly incorrect confounds.
Is social media research fast? Sure it is. It’s even faster if you ignore the annoying stage of data quality.
Unlike other types of research, social media research has the potential to provide brands with millions of relevant datapoints. That is, of course, as long as sufficient work is put into gathering the right data. What exactly is the right data? If you do Google “BP,” about 200 million records will be found. And given the severity around the situation in the Gulf, the first few hundreds of pages are all about British Petroleum. We carefully gathered thousands of records that specifically related to British Petroleum and created a quick word cloud of the types of companies that were mentioned. Not surprising, energy and finance companies were top of the list.
However, just two months ago, if you had googled “BP,” chances are you would have returned very different data. Perhaps it would have been about Brad Pitt or Basis Points or Blood Pressure, who’s to say? Well, we created a second word cloud that was based on sloppy data collection. Any mention of BP in social media was collected regardless of the context. As you can see in the cloud, BP most commonly stands for Blood Pressure. This is a huge confound that absolutely must be avoided.
Imagine how the research results would be affected if blood pressure data was included with British Petroleum data. We would be astonished that fishermen were using Zocor to try and calm down after watching Inglourious Basterds.
