#MRX MRA IMRO SMR Guidelines #8: Citing References

Wednesday, January 5th, 2011

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #8 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

kevin_p from morguefile

How are sources cited in research reports and on research Web portals? Are the citations different based on the source, e.g., Twitter, Blogger, forums?

Marketing research is not the same as academic research that takes place in a library. Though it is possible to gather social media data and specifically identify the website link, and even the individual person creating the information, that doesn’t mean it’s appropriate to do so. Social media data is research data that should be treated as similarly as possible to other forms of research data.

In surveys and focus groups, we promise participants that we will not share their names or other personally identifiable information with clients. We never include email addresses, phone numbers, or personal data in client reports. This is probably why people share so many intimate details with us – they didn’t write their name on the survey and their name won’t be in the research report.

However, the mere process of collecting social media data means that individuals are identifiable. Their name is usually right beside their research contribution. Sometimes, their age, gender, and address are right beside their contribution as well. But, as always, just because we have it doesn’t mean we should share it in reports.

It is common practice to provide links in social media portals. This allows researchers to click through the link and read comments in context. It ensures that opinions can be fully understood before they are aggregated into a representative research result.  It is a method for validating the finding and the accuracy of the end result. The purpose of market research is not to respond to individual people nor even to evaluate individual people. The purpose of market research is to aggregate data to achieve a broad understanding of a community. As such, there is no need to include personally identifying information in the next stage of the research process, the research report.

Research reports should not include personally identifiable information such as links or names. But, it is entirely appropriate for reports to indicate that “15% of results were sourced from Twitter” or that “86% of Foursquare users are extremely satisfied with the brand whereas only 26% of Blogger users are extremely satisfied.”  But to go one step further and name names just isn’t in keeping with the spirit of market research.

There are, of course, exceptions to the rule. Chances are that public figures expect their opinions to be shared in an identifiable way. Massively popular blogs like Mashable and Techcrunch expect people to directly quote and link to them. Politicians like Obama and Stephen Harper expect to be directly quoted. (I bet you’re wondering who Stephen Harper is.) And, even yours truly expects that people will quote this blog directly.

Exceptions to the rule do not make the rule. If you do not know, without a doubt, that the person would expect their comments to be quoted, then adhere to the rule of privacy. No citations in reports.

.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR
MRA IMRO Guide #5: Responsibilities of Social Media Data Users

Top Market Research Resources

Wednesday, December 29th, 2010

Your best bet for market research news is to start with the organization(s) in your country. They will have the most up-to-date legal and ethical opinions on issues that are unique to your region. But once you have that covered, you can learn a lot about your industry by keeping in touch with the various organizations around the world. Most of them welcome memberships from the global community particularly as our research focuses on the global community. Here are just a few of the English organizations that may be of interest to you.

….

.
CASRO – Council of American Survey Research Organizations
ESOMAR – European Society for Opinion and Marketing Research
MRA – Marketing Research Association (US)
IMRO – Interactive Marketing Research Association (US, part of MRA)
MRIA – Marketing Research and Intelligence Association (Canada)
AMSRS – Australian Market and Social Research Society
MRS – Marketing Research Society

.
There are also many other independent websites to help you stay on top of the MR scene. You can’t possibly participate in all of them as there are far too many and they are spread across Facebook, LinkedIn, and private sites, but not to worry as you’ll find a lot of duplication among them. You’ll probably catch most things with just these few sites. Say good-bye to free time!
.
All Top Market Research – Blog listings for any topic, including MR
MrWeb – Great site for MR news updates, plus frequent email updates if you wish
Market Research World – MR news updates, plus email updates
Research Live – Great site for MR news updates, plus frequent email updates if you wish

#MRX MRA IMRO SMR Guidelines #7: Research Contributor Awareness – The Conversition Commentary

Wednesday, December 22nd, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #7 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

slowfoot from morguefile

Are the participants aware that their user-generated content is under observation?

No.

The simple answer is no.

Not all people using social media realize that the comments and opinions they share online are under observation. It might seem counter-intuitive but as researchers of human behaviour, we need to remember that not everyone comes from the same social environment that we have. Given that you’re reading this blog, chances are you are fully immersed in the social media space and know that there is no such thing as private social media. But you’re an exception to the rule.

Human behaviours fall on a normal curve. Some people fall on the end of the curve where they understand that any sense of privacy was lost many years ago and that is probably where you sit. Other people fall on the end of the curve where they believe that the Nigerian prince truly does need financial assistance to cash in his multi-million dollar inheritance. There may be many fewer people on this side of the curve, but that doesn’t mean we can disregard them. As researchers, we are obliged to take care and protect people who may not fully appreciate all the implications of the internet. We do, after all, respect their opinions and so we should also show respect for the conditions under which they shared those opinions.

Most people, however, fall somewhere in between. They accept that the internet is public. They know that many people will read their comments on YouTube. But, they probably haven’t read the Facebook Terms of Service and they probably aren’t aware of which pieces of data are being shared widely. They probably aren’t thinking that when they obtain a password for a chatgroup about dyslexia there may be a researcher in the group who also easily obtained a password. And they certainly aren’t expecting that a completely disrespectful comment they made during extreme frustration would end up in an official report on the desk of the CEO of a global corporation with their name and twitter ID right beside it.

Research contributors are not all aware.  It’s our duty to respect that.

.
.
Related links
MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR
MRA IMRO Guide #5: Responsibilities of Social Media Data Users

#MRX MRA IMRO SMR Guidelines #6: Social Media Research Skills

Friday, December 17th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #6 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

ppdigital from morguefile

What additional knowledge, skills and abilities will a corporate researcher need to learn in order to improve their level of competency with SMR?

.

The leap from traditional survey and focus group skills to social media research skills is not necessarily a very large one. Many traditional skills, including sampling, weighting, norms, scaling, handling bias, accounting for statistical assumptions, and other key topics of marketing research are directly relevant and remain essential. The basic skills required of researchers are completely unchanged.

There are, of course, nuances and in this respect, researchers who have specialized in qualitative methods may find themselves slightly ahead of the game. Most qual researchers already have a good understanding of sentiment analysis and content analysis as these are essential components of the focus group, personal interview, and observational research methods. Qual researchers who already have extensive experience in manual versions of sentiment and content analysis may find they need to become more familiar with the slight differences of their automated equivalents including how grammatical errors, slang, and other normally only-human-understood elements are interpreted.

Researchers who have chosen to specialize in the quantitative side of things will have to take a larger leap. In fact, quantitative researchers are familiar with sentiment analysis but it is normally presented in the form of Likert scales where responders slot their opinions into pre-specified boxes such as “Strongly Agree“ or “Somewhat Disagree“. In the social media research case, however, quant researchers will need to spend some time learning about the processes for placing opinions into those Likert scale boxes on behalf of the responder (sentiment analysis). And, where quantitative researchers are used to creating categories of topics via a survey to which responders then provide an answer, they will have to learn how to reverse engineer that process. Thus, they will have to learn how to read an opinion and assign it to specific questions or categories such as “How likely are you to purchase this item?”  on behalf of the responder (content analysis).

If you keep your eyes open, you’ll notice just how familiar you already are with this thing called social media research.
.
.
Related links
MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR
MRA IMRO Guide #5: Responsibilities of Social Media Data Users

#MRX MRA IMRO SMR Guidelines #5 Responsibilities: The Conversition Commentary

Thursday, December 16th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #5 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

Alvimann from morguefile
.
Within businesses and organizations, how will SMR activities be tracked and aggregated, and whose responsibility is it to handle each of those functions?

This seemingly innocuous question has huge punch behind it. It first requires one to distinguish between social media research and other uses of social media data such as customer relationship management and public relations.

First, each use of social media data requires different skills. Those who intend to use it for social media research need to be knowledgeable about and skilled in such things as sampling, weighting, standards, norms, and scaling. These are industry specific skills that are learned over time through education and practice, and cannot just be picked up casually. As such, if you plan to use social media data for research purposes, make sure you have a researcher on hand who can filter out the noise of monitoring and filter in the science of research.

Second, users taking a research point of view towards the data must understand the ethics associated with that use of the data. Remember, as researchers, we have agreed to respect requests for privacy from our research contributors whether they are participating in surveys, focus groups, or social media research. Though the gathering of social media data brings with it links to an individual’s facebook, twitter, or youtube page, this does not give researchers the unequivocal right to contact and engage with people. This perspective is completely different than that taken by the CRM and PR business so there is definitely room for error and misunderstanding here. Fortunately, numerous professional groups are attempting to work through this issue including the MRA IMRO team.

In the end, as long as you remember to involve people with the right skills and solid ethical considerations, you will be on the right path.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #4 Reliability: The Conversition Commentary

Monday, November 8th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #4 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

deanjenkins from morguefile
.
How reliable are SMR results?
.
Reliability and validity are topics of particular interest to Conversition because they are staples of the market research industry that are not always well understood by market researchers and users. These quality measures apply to all methods of research, including survey and focus group research, and for best results must be considered jointly.

To begin, reliability refers to results which can be replicated across numerous occasions. For instance, if several different people were to conduct the same research, each person would achieve the same results. Or, if the same study was conducted across several time periods, each study would each achieve the same results. No matter who or where or when, every time the study is performed, the results should be the same. (Unless of course you are conducting a pre/post or time series study where you are looking for different results each time.)

Because different survey panels have different incentive and recruitment strategies, high reliability across survey panels is generally not expected nor appropriate. The same follows for social media research. Every vendor has different methods of data collection and treatment, and they each follow more or less stringent standards and processes. But, within each social media research vendor, an acceptable level of reliability over time and among product categories should be achievable.

Now, remember that reliable results do not quality make. One can easily achieve the same wrong results over and over again by using the same bad survey or bad focus group or bad SMR over and over again. This brings us to validity, the significant other of reliability. Validity refers to results which reflect exactly what was intended to be measured.

When we ask people to select their favourite item from a list, we achieve validity when people read the entire list instead of choosing an item at the top of the list. We achieve validity when people tell us which political candidate they honestly plan to vote for instead of giving us the name of most socially desirable candidate. Of course, people are not robots and these validity issues pop up all the time, but, we have learned many research techniques to solve these problems.

Validity in social media research comes down to the treatment of data. Data quality measures must ensure that the right data is being selected for analysis. As such, data for Apple Computers must not include data for apple pie. Similarly, data for British Petroleum (BP) must not include data for Basis Points or Blood Pressure or Boston Pizza.

Data quality practices extend beyond simply gathering the right set of data. They must be applied to other data treatments as well including sentiment analysis and content analysis. Thus, sentiment analysis must distinguish between dope that is smoked illegally and dope that is hip, cool, and totally rad. And, content analysis must distinguish between the orange fruit and the orange color and the Planet Orange charity so lovingly built by the folks at ING.

For all of these purposes, validity can be evaluated with a fairly simple process.

1. Randomly select 1000 records from across different topics, dates, and data sources.
2. Score each record yourself.  a) What brand name does it reflect? b) What sentiment score does it deserve? c) What variable does it reflect?
3. Run the data through a second system whether it be your automated processes or a second person.
4. Match the two sets of results together.
5. Calculate the percentage of results that agree. a) What percentage of the data was actually about the intended brand name? b) What percentage of sentiment scores matched? c) What percentage of variables were correct?

.

The most important component of this validation work is that the two sets of data are scored blindly. In other words, I don’t know how you scored them, and you don’t know how I scored them.

Reliability and validity are essential components of all quality market research methods, including social media research. You can have one without the other, but without both you really have nothing. You need to ask your social media research provider how they address validity and reliability. Are these words essential components of their work? Do they have processes in place? Are those process grounded in solid research standards?

Go forth and inquire. It’s time.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #3 Data Fusion: The Conversition Commentary

Thursday, November 4th, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #3 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

jdurham from morguefile
.
How does SMR interact with other forms of traditional and non-traditional research, including online, offline, in-person, and qualitative and quantitative?
.
This is one of our favorite topics at Conversition. We love to show clients how data fusion of results from traditional market research and social media research generates more powerful results. Note that we say fusing research, not replacing research because there is just no replacement for surveys and focus groups.

One of the first questions many clients have is whether social media results are the same as survey results. Given that we are market researchers with years of first hand experience transitioning offline studies to online versions, we completely understand how new methods affect results. But, social media research isn’t intended to duplicate survey research – it builds and broadens the entire set of results.

For those new to social media research, one of the best ways to get a better understanding of SMR is to run both a survey and SMR at the same time. You can take advantage of the specificity that surveys provide, as well as the descriptive narrative and breadth of topics that social market research provides. Pay attention to the types of survey questions that can and can’t be mirrored in the SMR. Pay attention the the types of survey questions that explode into new questions and serendipitous results in the SMR.

Another interesting way to combine the best of both worlds is to use social media research to inform the contents of a survey. When surveys become too long, we must choose something to leave out and long lists are a great place to start. One way to shorten these long lists is to gather thousands of records from SMR, cull out the top brands or flavours or SKUs, and include just the top items in the survey.

Social media research also gives researchers a much broader and more open perspective of how users communicate about a brand. Once the basic usage and attitude information has been gathered, social media research can be used to flush out those results. The most obvious way is to discover how people talk about brands to gather psychographic information – what word choices do they make, do they use slang, do they use correct grammar.

Additionally, social media allows for the discovery of creativity – how do users incorporate photos, videos, and music, into their approach to a brand.

Social media research is also a great way to conduct pre-post research for new products. Where focus groups allow researchers to observe people using brand new products before those products go to market, social media research allows you to evaluate product instantly as they are introduced to the market.

These are just a few of the ways that traditional research and social media research can be incorporated into a multi-mode method of market research. Try it. You’ll like it.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRX MRA IMRO SMR Guidelines #2 Datasources: The Conversition Commentary

Wednesday, November 3rd, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is #2 in a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.

lespowell from morguefile

What data sources are typically used in SMR?

We’ve found it’s easiest to understand social media research if you use a familiar type of research as the starting point, in this case survey research. When conducting a survey, one of your main tasks is to identify all the places from where you will source your survey responders. In recent years, we’ve come to rely on survey panels as the main source, though many other sources do exist. You could use permission based email lists or consumer registration lists. You could visit various locations such as shopping malls or community centers to ask people in person if they would like to participate in a survey. Regardless of from where the survey respondents are sourced, or from how many different places they are sourced, you as a researcher have made a decision about the selection and combination of respondent sources.

The same can be said for social media research. It may seem as though the internet is a single source, but it is comprised of many unique communities. About three quarters of a billion people are familiar with the Facebook community, a place to catch up with fun commentary, silly pictures, and unique games. But there are many different communities which attract smaller, more specialized subgroups of people.

Video sites such as YouTube and MetaCafe attract people who like to make, watch, and comment on videos. Traditional news sites such as CNN or BusinessWeek attract a different type of person to the community, people who are interested in politics, world affairs, and financial news. Microblogging sites, such as Twitter or Identi.ca attract users who like pithy commenting without extraneous flair. Blogging sites, such as Blogger, WordPress, or Posterous attract yet another group of people, those who have ideas and opinions about a wide range of topics that they want to share with other people. Along with many other types of websites, each of these types attracts a different audience with different desires and intentions and purposes. And with that comes unique sets of demographics and psychographics.

No matter how you slice and dice the internet, for there is no single correct way other than that which best suits your individual research objective, the range of available websites virtually reflects a shopping mall of communities. Clothing stores, electronics stores, toy stores, hardware stores, grocery stores, discount stores, department stores, financial stores, the service desk, the parking valet, and every other type of store, or website, is set up and ready to welcome interested members of its community.

While I may be most interested in the baking and gardening stores (websites), my mom is more interested in the clothing and shoe stores (websites), my dad is more interested in the hardware and grocery stores (websites), and my SO is more interested in the electronics and comic book stores (websites). And those are the social media communities to which you will find each of us contributing.

Understanding the layout of websites on the internet, whether it be the type of website (e.g., blogs, forums, and news sites) or the type of content (e.g., entertainment, attire, and electronics), is the first step towards having a solid foundation to begin your own social media research.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

MRA IMRO SMR Guidelines #1 Advantages and Disadvantages: The Conversition Response

Monday, November 1st, 2010

MRA recently released version 1 of the MRA/IMRO Guide to the Top 16 Social Media Research Questions, a tool to help newcomers and vendors communicate with each other about this new datasource and method. Conversition was a key contributor to this document which is now available on the MRA website.
.
This blog is the first of a series of 16, each one addressing Conversition’s viewpoint on one of the items in the guidelines. We welcome your questions and comments, and look forward to further discussions on this exciting new trend in the market research industry.
.

jdurham from morguefile
.
#1 Advantages and Disadvantages of Social Media Research
.
Without exception, every research method has unique advantages and disadvantages. Nonprobablity sampling is an issue affecting every form of market research, including social media research. Just as surveys normally can’t identify every member of a population and force a random sample to complete the survey, social media research cannot listen to people who do not contribute to social media nor those whose data is behind password protected walls. But we’ve developed methods for understanding and compensating for such weaknesses.
.
One of the unique weaknesses of social media research is the lack of individualized demographic and and geographic data. This means we must find new and different ways to learn about the unique characteristics of people talking about brands. Inferred demographics and qualitative psychographics are just some of the solutions. The lack of demographics also means that we must find new ways of sampling and weighting our results, such as using website sources, to ensure that results are properly generalized to external populations.
.
As researchers, we have a collective history that has taught us how to address and compensate for methodological issues. As business managers, these issues mean that assigning costs to business units can be more difficult – how does one invoice a local project to a global unit?
.
On the the other hand, every method has unique advantages. Some of the most distinguishing advantages of social media research includes its amazing ability to discover how people talk about a brand in an uncontaminated environment – no questions, no probing, no elicitation, just pure, raw meaning. Is the chatter highbrow or lowbrow, quick and dirty or descriptive and well-thought out?
.
In addition, the penetration of social networks, forums, video and photo sites, and other types of websites that allow users to interact with other people means that the quantity of usable data can be astounding. Well known consumer brands such as Microsoft, Nike, Starbucks, and Coca-cola have an enviable problem of working with millions of records. Other less popular brands, though, can still take advantage of social media research by evaluating competitor data and industry data.
.
Because of the volume of data and the millions of people contributing knowledge about a brand, the breadth of available data in social media research can be astounding. Where surveys are limited to 20 or 30 minutes and perhaps 40 questions, social media research is like offering a ten hour survey with thousands of directly relevant questions. The variety of topics that people talk about online is far greater than can be incorporated in a traditional survey.
.
There are far more advantages to social media research but don’t take our word for it. Try it for yourself.
.
.
Related links

MRA IMRO Guide #1: Advantages and Disadvantages of SMR
MRA IMRO Guide #2: Datasources of SMR
MRA IMRO Guide #3: Data Fusion and SMR
MRA IMRO Guide #4: Reliability of SMR

#MRA_FOC Live Blogging by Conversition’s Annie Pettit

Sunday, October 31st, 2010

The MRA First Outlook Conference is from November 2 to 4 in Orlando, Florida this year. Our very own Annie Pettit will be live blogging the event so stay tuned to her LoveStats blog for all the highlights. You will also be able to catch her blog on the MRA website.
.
In addition, Annie will be joining Patrick Glaser of the MRA and Jim Longo of iTracks in a panel discussion about the newly released MRA IMRO Social Media Research Guide. Annie was instrumental in developing the guidelines and is excited to discuss them with social media research skeptics, fans, and experts alike.
.
We look forward to seeing each of you there!