by: George Silverman
In an article last Monday in the Wall Street Journal, reporter Emily Steel described the growing trend of using online social networks — both existing and company-encouraged — for marketing research. It’s a very dangerous trend, as I point out in my letter to her. Many companies are headed for disaster if they give undue weight to the opinions expressed on their online networks.
Emily,
Very much enjoyed your article The New Focus Groups: Online Networks.
http://online.wsj.com/article/SB120027230906987357.html?mod=djemMM
However, it didn’t cover the major pitfalls, of which there are many. (Full disclosure: I am a marketing consultant who runs face-to-face focus groups and telephone focus groups. I’m a founder of the Qualitative Research Consultants Association and member of its Professionalism Committee, although I am speaking officially for neither.)
I have rejected the methodology of online groups for reasons enumerated in detail in the following article:
http://www.mnav.com/onlinetablesort.htm
In brief, the written word does not allow for the reading of emotions that live focus groups do, and the reading of these emotions is absolutely necessary for the interpretation of the results. (E.g., How enthusiastic are they? Are they hesitant? Are their remarks ironic and sarcastic? Are they coming from their heads or hearts? Are they mildly annoyed or royally pissed off?)
However, these kinds of standing panels have a problem that I didn’t discuss in the article. It is one of sample bias. As you describe, people are continually dropping out of the group and being replenished. This severely skews the kind of people who remain in the panel, in ways that are virtually impossible to account for in interpreting the findings. It is well known that participators are radically different than non-participators and ex-participators. For instance, probably the more enthusiastic and/or more lonely people (including social misfits) tend to stay in. So, you keep the enthusiasts, for whom the panel becomes a part of their social life (as mentioned in your article). They are sometimes less prone to criticize, but sometimes more prone to criticize. The point is, one never knows. But as this sample becomes more and more distorted and unrepresentative of real customers, you have a disaster waiting to happen.
On the other hand, these panels are a wonderful source of ideas and a way to make sure that certain actions and wording do not antagonize loyal customers. But they are notoriously unpredictive of success in the marketplace. The Edsel automobile and New Coke are but two of many examples of going to the wrong people and asking the wrong questions. Many of the .com failures were guided by discussions on company forums, forgetting that most real people to not hang out on forums, especially for prolonged periods of time.
I hope that, as a reporter, you will follow this phenomenon. You will have many juicy disasters to cover. When you ask, “What were they thinking and why were they thinking it?” I hope that you will keep this letter in mind when they tell you “That’s what our customers told us they wanted.”
Original Post: http://wordofmouth.typepad.com/george_silvermans_word_of/2008/01/using-online-ne.html
