#MRX MRA IMRO SMR Guidelines #7: Research Contributor Awareness – The Conversition Commentary

Wednesday, December 22nd, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #7 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

slowfoot from morguefile

Are the participants aware that their user-generated content is under observation?

No.

The simple answer is no.

Not all people using social media realize that the comments and opinions they share online are under observation. It might seem counter-intuitive but as researchers of human behaviour, we need to remember that not everyone comes from the same social environment that we have. Given that you’re reading this blog, chances are you are fully immersed in the social media space and know that there is no such thing as private social media. But you’re an exception to the rule.

Human behaviours fall on a normal curve. Some people fall on the end of the curve where they understand that any sense of privacy was lost many years ago and that is probably where you sit. Other people fall on the end of the curve where they believe that the Nigerian prince truly does need financial assistance to cash in his multi-million dollar inheritance. There may be many fewer people on this side of the curve, but that doesn’t mean we can disregard them. As researchers, we are obliged to take care and protect people who may not fully appreciate all the implications of the internet. We do, after all, respect their opinions and so we should also show respect for the conditions under which they shared those opinions.

Most people, however, fall somewhere in between. They accept that the internet is public. They know that many people will read their comments on YouTube. But, they probably haven’t read the Facebook Terms of Service and they probably aren’t aware of which pieces of data are being shared widely. They probably aren’t thinking that when they obtain a password for a chatgroup about dyslexia there may be a researcher in the group who also easily obtained a password. And they certainly aren’t expecting that a completely disrespectful comment they made during extreme frustration would end up in an official report on the desk of the CEO of a global corporation with their name and twitter ID right beside it.

Research contributors are not all aware.  It’s our duty to respect that.

.
.
Related links
MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR
MRA IMRO Guide #5: Responsibilities of Social Media Data Users

#MRX MRA IMRO SMR Guidelines #6: Social Media Research Skills

Friday, December 17th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #6 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

ppdigital from morguefile

What additional knowledge, skills and abilities will a corporate researcher need to learn in order to improve their level of competency with SMR?

.

The leap from traditional survey and focus group skills to social media research skills is not necessarily a very large one. Many traditional skills, including sampling, weighting, norms, scaling, handling bias, accounting for statistical assumptions, and other key topics of marketing research are directly relevant and remain essential. The basic skills required of researchers are completely unchanged.

There are, of course, nuances and in this respect, researchers who have specialized in qualitative methods may find themselves slightly ahead of the game. Most qual researchers already have a good understanding of sentiment analysis and content analysis as these are essential components of the focus group, personal interview, and observational research methods. Qual researchers who already have extensive experience in manual versions of sentiment and content analysis may find they need to become more familiar with the slight differences of their automated equivalents including how grammatical errors, slang, and other normally only-human-understood elements are interpreted.

Researchers who have chosen to specialize in the quantitative side of things will have to take a larger leap. In fact, quantitative researchers are familiar with sentiment analysis but it is normally presented in the form of Likert scales where responders slot their opinions into pre-specified boxes such as “Strongly Agree“ or “Somewhat Disagree“. In the social media research case, however, quant researchers will need to spend some time learning about the processes for placing opinions into those Likert scale boxes on behalf of the responder (sentiment analysis). And, where quantitative researchers are used to creating categories of topics via a survey to which responders then provide an answer, they will have to learn how to reverse engineer that process. Thus, they will have to learn how to read an opinion and assign it to specific questions or categories such as “How likely are you to purchase this item?”  on behalf of the responder (content analysis).

If you keep your eyes open, you’ll notice just how familiar you already are with this thing called social media research.
.
.
Related links
MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR
MRA IMRO Guide #5: Responsibilities of Social Media Data Users

#MRX MRA IMRO SMR Guidelines #5 Responsibilities: The Conversition Commentary

Thursday, December 16th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #5 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

Alvimann from morguefile
.
Within businesses and organizations, how will SMR activities be tracked and aggregated, and whose responsibility is it to handle each of those functions?

This seemingly innocuous question has huge punch behind it. It first requires one to distinguish between social media research and other uses of social media data such as customer relationship management and public relations.

First, each use of social media data requires different skills. Those who intend to use it for social media research need to be knowledgeable about and skilled in such things as sampling, weighting, standards, norms, and scaling. These are industry specific skills that are learned over time through education and practice, and cannot just be picked up casually. As such, if you plan to use social media data for research purposes, make sure you have a researcher on hand who can filter out the noise of monitoring and filter in the science of research.

Second, users taking a research point of view towards the data must understand the ethics associated with that use of the data. Remember, as researchers, we have agreed to respect requests for privacy from our research contributors whether they are participating in surveys, focus groups, or social media research. Though the gathering of social media data brings with it links to an individual’s facebook, twitter, or youtube page, this does not give researchers the unequivocal right to contact and engage with people. This perspective is completely different than that taken by the CRM and PR business so there is definitely room for error and misunderstanding here. Fortunately, numerous professional groups are attempting to work through this issue including the MRA IMRO team.

In the end, as long as you remember to involve people with the right skills and solid ethical considerations, you will be on the right path.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #4 Reliability: The Conversition Commentary

Monday, November 8th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #4 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

deanjenkins from morguefile
.
How reliable are SMR results?
.
Reliability and validity are topics of particular interest to Conversition because they are staples of the market research industry that are not always well understood by market researchers and users. These quality measures apply to all methods of research, including survey and focus group research, and for best results must be considered jointly.

To begin, reliability refers to results which can be replicated across numerous occasions. For instance, if several different people were to conduct the same research, each person would achieve the same results. Or, if the same study was conducted across several time periods, each study would each achieve the same results. No matter who or where or when, every time the study is performed, the results should be the same. (Unless of course you are conducting a pre/post or time series study where you are looking for different results each time.)

Because different survey panels have different incentive and recruitment strategies, high reliability across survey panels is generally not expected nor appropriate. The same follows for social media research. Every vendor has different methods of data collection and treatment, and they each follow more or less stringent standards and processes. But, within each social media research vendor, an acceptable level of reliability over time and among product categories should be achievable.

Now, remember that reliable results do not quality make. One can easily achieve the same wrong results over and over again by using the same bad survey or bad focus group or bad SMR over and over again. This brings us to validity, the significant other of reliability. Validity refers to results which reflect exactly what was intended to be measured.

When we ask people to select their favourite item from a list, we achieve validity when people read the entire list instead of choosing an item at the top of the list. We achieve validity when people tell us which political candidate they honestly plan to vote for instead of giving us the name of most socially desirable candidate. Of course, people are not robots and these validity issues pop up all the time, but, we have learned many research techniques to solve these problems.

Validity in social media research comes down to the treatment of data. Data quality measures must ensure that the right data is being selected for analysis. As such, data for Apple Computers must not include data for apple pie. Similarly, data for British Petroleum (BP) must not include data for Basis Points or Blood Pressure or Boston Pizza.

Data quality practices extend beyond simply gathering the right set of data. They must be applied to other data treatments as well including sentiment analysis and content analysis. Thus, sentiment analysis must distinguish between dope that is smoked illegally and dope that is hip, cool, and totally rad. And, content analysis must distinguish between the orange fruit and the orange color and the Planet Orange charity so lovingly built by the folks at ING.

For all of these purposes, validity can be evaluated with a fairly simple process.

1. Randomly select 1000 records from across different topics, dates, and data sources.
2. Score each record yourself.  a) What brand name does it reflect? b) What sentiment score does it deserve? c) What variable does it reflect?
3. Run the data through a second system whether it be your automated processes or a second person.
4. Match the two sets of results together.
5. Calculate the percentage of results that agree. a) What percentage of the data was actually about the intended brand name? b) What percentage of sentiment scores matched? c) What percentage of variables were correct?

.

The most important component of this validation work is that the two sets of data are scored blindly. In other words, I don’t know how you scored them, and you don’t know how I scored them.

Reliability and validity are essential components of all quality market research methods, including social media research. You can have one without the other, but without both you really have nothing. You need to ask your social media research provider how they address validity and reliability. Are these words essential components of their work? Do they have processes in place? Are those process grounded in solid research standards?

Go forth and inquire. It’s time.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #3 Data Fusion: The Conversition Commentary

Thursday, November 4th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #3 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

jdurham from morguefile
.
How does SMR interact with other forms of traditional and non-traditional research, including online, offline, in-person, and qualitative and quantitative?
.
This is one of our favorite topics at Conversition. We love to show clients how data fusion of results from traditional market research and social media research generates more powerful results. Note that we say fusing research, not replacing research because there is just no replacement for surveys and focus groups.

One of the first questions many clients have is whether social media results are the same as survey results. Given that we are market researchers with years of first hand experience transitioning offline studies to online versions, we completely understand how new methods affect results. But, social media research isn’t intended to duplicate survey research – it builds and broadens the entire set of results.

For those new to social media research, one of the best ways to get a better understanding of SMR is to run both a survey and SMR at the same time. You can take advantage of the specificity that surveys provide, as well as the descriptive narrative and breadth of topics that social market research provides. Pay attention to the types of survey questions that can and can’t be mirrored in the SMR. Pay attention the the types of survey questions that explode into new questions and serendipitous results in the SMR.

Another interesting way to combine the best of both worlds is to use social media research to inform the contents of a survey. When surveys become too long, we must choose something to leave out and long lists are a great place to start. One way to shorten these long lists is to gather thousands of records from SMR, cull out the top brands or flavours or SKUs, and include just the top items in the survey.

Social media research also gives researchers a much broader and more open perspective of how users communicate about a brand. Once the basic usage and attitude information has been gathered, social media research can be used to flush out those results. The most obvious way is to discover how people talk about brands to gather psychographic information – what word choices do they make, do they use slang, do they use correct grammar.

Additionally, social media allows for the discovery of creativity – how do users incorporate photos, videos, and music, into their approach to a brand.

Social media research is also a great way to conduct pre-post research for new products. Where focus groups allow researchers to observe people using brand new products before those products go to market, social media research allows you to evaluate product instantly as they are introduced to the market.

These are just a few of the ways that traditional research and social media research can be incorporated into a multi-mode method of market research. Try it. You’ll like it.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #2 Datasources: The Conversition Commentary

Wednesday, November 3rd, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #2 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

lespowell from morguefile

What data sources are typically used in SMR?

We’ve found it’s easiest to understand social media research if you use a familiar type of research as the starting point, in this case survey research. When conducting a survey, one of your main tasks is to identify all the places from where you will source your survey responders. In recent years, we’ve come to rely on survey panels as the main source, though many other sources do exist. You could use permission based email lists or consumer registration lists. You could visit various locations such as shopping malls or community centers to ask people in person if they would like to participate in a survey. Regardless of from where the survey respondents are sourced, or from how many different places they are sourced, you as a researcher have made a decision about the selection and combination of respondent sources.

The same can be said for social media research. It may seem as though the internet is a single source, but it is comprised of many unique communities. About three quarters of a billion people are familiar with the Facebook community, a place to catch up with fun commentary, silly pictures, and unique games. But there are many different communities which attract smaller, more specialized subgroups of people.

Video sites such as YouTube and MetaCafe attract people who like to make, watch, and comment on videos. Traditional news sites such as CNN or BusinessWeek attract a different type of person to the community, people who are interested in politics, world affairs, and financial news. Microblogging sites, such as Twitter or Identi.ca attract users who like pithy commenting without extraneous flair. Blogging sites, such as Blogger, WordPress, or Posterous attract yet another group of people, those who have ideas and opinions about a wide range of topics that they want to share with other people. Along with many other types of websites, each of these types attracts a different audience with different desires and intentions and purposes. And with that comes unique sets of demographics and psychographics.

No matter how you slice and dice the internet, for there is no single correct way other than that which best suits your individual research objective, the range of available websites virtually reflects a shopping mall of communities. Clothing stores, electronics stores, toy stores, hardware stores, grocery stores, discount stores, department stores, financial stores, the service desk, the parking valet, and every other type of store, or website, is set up and ready to welcome interested members of its community.

While I may be most interested in the baking and gardening stores (websites), my mom is more interested in the clothing and shoe stores (websites), my dad is more interested in the hardware and grocery stores (websites), and my SO is more interested in the electronics and comic book stores (websites). And those are the social media communities to which you will find each of us contributing.

Understanding the layout of websites on the internet, whether it be the type of website (e.g., blogs, forums, and news sites) or the type of content (e.g., entertainment, attire, and electronics), is the first step towards having a solid foundation to begin your own social media research.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

MRA IMRO SMR Guidelines #1 Advantages and Disadvantages: The Conversition Response

Monday, November 1st, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is the first of a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

jdurham from morguefile
.
#1 Advantages and Disadvantages of Social Media Research
.
Without exception, every research method has unique advantages and disadvantages. Nonprobablity sampling is an issue affecting every form of market research, including social media research. Just as surveys normally can’t identify every member of a population and force a random sample to complete the survey, social media research cannot listen to people who do not contribute to social media nor those whose data is behind password protected walls. But we’ve developed methods for understanding and compensating for such weaknesses.
.
One of the unique weaknesses of social media research is the lack of individualized demographic and and geographic data. This means we must find new and different ways to learn about the unique characteristics of people talking about brands. Inferred demographics and qualitative psychographics are just some of the solutions. The lack of demographics also means that we must find new ways of sampling and weighting our results, such as using website sources, to ensure that results are properly generalized to external populations.
.
As researchers, we have a collective history that has taught us how to address and compensate for methodological issues. As business managers, these issues mean that assigning costs to business units can be more difficult – how does one invoice a local project to a global unit?
.
On the the other hand, every method has unique advantages. Some of the most distinguishing advantages of social media research includes its amazing ability to discover how people talk about a brand in an uncontaminated environment – no questions, no probing, no elicitation, just pure, raw meaning. Is the chatter highbrow or lowbrow, quick and dirty or descriptive and well-thought out?
.
In addition, the penetration of social networks, forums, video and photo sites, and other types of websites that allow users to interact with other people means that the quantity of usable data can be astounding. Well known consumer brands such as Microsoft, Nike, Starbucks, and Coca-cola have an enviable problem of working with millions of records. Other less popular brands, though, can still take advantage of social media research by evaluating competitor data and industry data.
.
Because of the volume of data and the millions of people contributing knowledge about a brand, the breadth of available data in social media research can be astounding. Where surveys are limited to 20 or 30 minutes and perhaps 40 questions, social media research is like offering a ten hour survey with thousands of directly relevant questions. The variety of topics that people talk about online is far greater than can be incorporated in a traditional survey.
.
There are far more advantages to social media research but don’t take our word for it. Try it for yourself.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

MRA IMRO Social Media Research Guide has arrived!

Wednesday, October 27th, 2010

The MRA has just announced the release of the much anticipated document. With contributions from Annie Pettit and other market research experts, it was a journey of a couple of months to get it to this point and it will continue to evolve in future iterations. You can download the document here.

We highly encourage you to download and review the document, but in the meantime, here are a couple points of particular note:

• Best practices call for researchers to respect the coded crawling terms of every Web site they visit. Where Web sites are coded to indicate that crawling is not permitted, those Web sites should not be crawled even if it is technically possible. Researchers must not join Web sites under the pretense of being a member so that they then have access to crawl a Web site that prohibits such crawling otherwise – this condition holds for both automated and manual crawling. Where researchers do join groups, they must immediately make it explicit that they are there for the purposes of marketing research.
• Validity refers to the degree to which results reflect truth or reality while reliability reflects the degree to which results can be replicated if someone else were to conduct a similar study. Because research suppliers have different methods, standards of quality, and processing rules, research consumers must conduct their own validity and reliability analysis of any potential supplier to ensure the quality of work is sufficient. As with all types of marketing research, the validity and reliability of social media research varies greatly.

We’ d love to hear your thoughts.

IMRO guidelines for research based on social media data

Tuesday, July 20th, 2010

A committee has been formed and brainstorming is underway!

With the realization that research using social media data is quickly gaining in popularity, the IMRO division within the MRA has put together a committee to help define and guide users and providers in this space. Over the next couple of months, a team of 17 or so researchers, including Annie Pettit (@LoveStats) of Conversition Strategies, and led by Jim Longo (@LongoMR) of iTracks, will be putting their heads together to build a short document that will be made publicly available.
.
The committee will address a number of issues including what is social media research, how should users evaluate it, what are the essential components for consideration, as well as other key topics. The goal is not to provide rules that must be followed but rather to ensure users and providers are aware of all of the issues when considering and conducting social media research.
.
Your opinions are important. Share your thoughts, questions and concerns with us by leaving comments here, emailing Annie (LinkIn) or by emailing Jim Longo.