by Stefan Kolle on 17 July, 2007 - 17:09
To follow up my previous post on the Net Promoter Score (NPS) being under fire, I would like to engage the readers of this blog in a discussion on the use and value of the NPS in itself. I will be listing some (rather random) thoughts and musings on the NPS to get your insights and ideas.
So we break all our (unwritten) rules at the same time - but we think this issue is important enough to focus some attention on, and we really need your feedback, so please read on.
The interesting bit is, I may have misunderstood NPS all along – which is why it made such perfect sense. As explained in my previous, I think it is the wider net, the capture of non-customer promoters (advocates, influencers, whichever you want to call them) that provides true insight into what is going on for my brand. If indeed, as Tim says, the NPS (and therefore comparative research) should only focus on actual purchasers, it would lose half it’s value for me. I’m finding myself in a bit of a paradox here.
Let’s put this in a broader perspective – that of the industry’s search for the holy grail, i.e. proper Word of Mouth (WoM) metrics, and how to calculate WoM value. (See also below)
The key value of the question asked – “would you recommend” over the often used “would you buy (again)” to me is that it catches out a lot of false positives/negatives. The example I gave in my discussion with Tim is that of me being very satisfied with my past experience with a particular brand of car, but not buying it again for the simple reason that my boss has given me a company car…. This would be a false negative in a “would you buy again”-question, but a strong positive in a NPS setting. It captures my value as a brand advocate to the company.
Even though I had an immediate and instinctive positive reaction to the NPS, there have been a number of things that kept bothering me all along. For one, the strictness of the categories. I would say that we should look at the categories more differentiated – both culturally and operationally. What do I mean? Not everyone is as likely to give a good experience a 10 score, and especially cultural differences weigh in heave – in some countries it is just unthinkable to give such a high score. But, as the main point is to do intra-country comparisons, I can live with this.
But I would also like to see a better differentiation within the categories. For instance, I think there is a massive distinction between a 5 and a 0. The 0 might go out there and pro-actively discourage people from buying with a certain company (picketing in front of their headquarters), while the 5 is most likely to go ‘mwoah’ when asked by me. Put an 8 (who, mind you, is seen as neutral) next to him who goes ‘yes, I had a good experience with that company’ and I will still turn into a customer – even though I only talked to a neutral and a detractor.
Also, the ‘why’ to me does play a strong role, as without that question we may have to deal with a lot of false negatives. Some people might answer ‘no, I am not likely to recommend’. If you ask why – ‘because nobody ever asks me’. I kid you not, this is an answer I have seen a lot. And this is one of the points where the value of a multi-brand analysis comes in for me. I might give a 9 score to Brand X – and still not recommend, because I’m giving a 10 score to Brand Y.
The main point of the current attacks on the NPS is about it’s predictive value towards revenue/shareprice growth. As interesting as that point in itself may be, it is more the profitability of individual customers and how to capture and improve that which to me has the most interest.
A promoter of the company is likely to generate additional sales, pay premium prices, buy more units and cost less in servicing. See the example Dell calculation here : http://www.netpromoter.com/winners-sinners/economics.php
This all aside, however, I think the true value of the Net Promoter Score as a tool within companies lies simply in the focus on the customer it generates.
Where many marketers are now buried under reams of data, KPI’s, customer satisfaction studies, brandvalue analysis, etc ad infinitum, this is replaced by one single and easy to handle and understand metric. It focuses the organization on concrete results, on “how will we delight” instead of generic customer satisfaction indexes. It creates a dollars-and-cents conversation due to the measurable value of an individual promoter to the company. And again it focuses the organization on precisely that point – to get more profitable customers (insert dollar value here) I need to improve specifically X, Y and Z. “Here you go, dear finance director – my new marketing initiative will generate this ROI for the company, because of 500 detractors being turned into promoters, generating 1000 X, 500 Y and 750 Z. “
To summarise my position - there are a lot of things wrong or improvable about the NPS - but it is the best we have, and warts and all is still a very valuable tool to achieve certains goals within a company.
So – what do you think? A couple of questions to kick off the discussion:
A. Is the value of NPS in single or multi-brand research?
B. Does it help you (and your organization) focus, or do you see it as a further distraction?
C. Do you focus on the money value of promotors of your company?
D. What else can we learn from or do with the NPS?
E. Do the flaws of the NPS discredit it as a tool?
We’d like to turn this into an ongoing discussion, so please respond – here, or with a blogpost and let us know.
This blog reflects the personal opinions of individual contributors and does not represent the views of Futurelab, Futurelab's clients, or the contributors' respective employers or clients.
RSS
Alain Thys says:
18 Jul 2007, 11:38
Stefan
to continue our phone conversation online :-) I thin k you hit closest to home with the "focus comment". As a CEO or Senior executive you're confronted with a gazillion KPI's which gives everyone in the organisations a large amount of excuses to everything under the sun, except "delight" the customer. focusing everyone on "one metric" may not be academically correct (but who cares), it does get the point across and eliminated internal excuses. It also is extremely helpful in getting various "silos" in the organisation to pay attention to the same thing.
Stefan Kolle says:
21 Jul 2007, 13:46
Jan, a very nice point you make (which also contains the answer I would give to Larry):
"Ultimately NPS measures the balance of mental power in the market"
We are still very far away of being able to measure or calculate all the variables we would need to have true insight into the behaviour of the whole market. But as a quick test of the watertemperature, the NPS is very useful.
With so many major companies investing seriously in NPS capture and conversion, I am sure we will see more finetuned solutions come about in the next years, whereby we start capturing those additional elements, such as propensity to promote.
But one thing I am still adamant about - the full value of NPS only comes through multi-brand research across buyers and non-buyers.
Jan Van den Bergh says:
20 Jul 2007, 09:09
When I read Reicheldâs article in HBR many years ago I promoted myself immediately to the status of Reicheldian.
Since Iâm a seasoned advertising man who left old media already in 1994 I daily experience the mediapower of recommendation. Also here in China. People who recommend a brand to other people are the ultimate marketing communication tool for brand. And theyâre free. If the people who recommend are highly connected ( to e.g more than 50 friends) itâs even better than when theyâre only connected to very few. The highly connected are the most valuable.
People who recommend sth have positive experiences with a brand and they are trusted when they talk honestly about these experiences. Theyâre finally like ads you can trust. The only ads by the way ⦠you REALLY trust!
In our practice we try to find out who these ambassadors are. We try to identify them. They neednât be customers of course. Since 2 yrs Iâ m a fanatic promoter of Jaguar although I never had one. From the financial bottomline Iâm worthless for Jag. From a communication value point of view Iâ m worth a Jagâº. I hope to get one for free.
Stefan is right saying it is also the âwider net, the capture of non-customer promoters (advocates, influencers, whichever you want to call them)â you have to take into account and not only the actual customers. It's indeed the overall buzz (or lack of buzz) you have to take into account. Itâs the good versus the bad messages. The better the final score, the brighter the future for a brand.
Word of Mouth in itself is also the result of many many inputs. Some very âquick and dirty first impressionsâ. Some very deep. Some recent. Some old. Thatâs why you do not have be too strict with the categories. And the âagreementâ of using 9 and 10 minus 0 till 6 is just an agreement. Like mathematics. Since the netpromoter score is only a vague indication, a vague prediction of what is and what will come.
You can only âhopeâ that within a given cultural consumers more or less interpret it the same way and give a 9 or 10 when they really actively recommend. May be it would have been better using ++, +, 0, - and - - and reduce the 10 categories to five easy to understand. But having ten possibilities gives more greyscalesl in the end result.
The 0 (zero) I already I interpret like you wrote it here: â â¦might go out there and pro-actively discourage people from buying with a certain company (picketing in front of their headquarters),â I already interpret this as active negative word of mouth. And I think Reichheld does that too.
And indeed the why could also be important and the multi-brand choice is also important. I can recommend Jag, saab en bmw at the same time. And give a 10 to 3 brands - so what? Why not? It is indeed about â the profitability of individual customersâ the marketing communications ROI of an indidual human medium
I donât think that such a net promoter will yield bad data and bad decisions. The data are exact. The decisions afterwards are taken by human beings. The data give you an idea about where you are. Like putting your finger in a hot water bath before you jump in. Either itâs ok. Or too hot or too cold. The reason for the three different situations can be multiple. Of course. You can run separate research to find out. And the solutions can be different too. But you have at least the data. Of course if you add too cold water when it was too hot, then you can be obliged to add again more hot water.
.
Is satisfaction measurement enough? Of course not ! I can be very satisfied but talk to nobody about my pleasure. Then â from a communication point of view - Iâm useless. Thatâs like having no ad. Ultimately NPS measures the balance of mental power in the market. And itâs this balance (positive or negative) that indicates among others purchase intent, loyalty and word of mouth.
Look at Yahooâs study a few weeks ago. Thereâs a clear difference between those who actively recommend and those who less actively recommend. Thatâs what Reichheld captures. Yahoo's study http://www.clickz.com/showPage.html?page=3626006
is of interest because of some interesting quantifiable data it presents.
It found, for example, advocates have a 2:1
conversion impact across categories. In plain English, that means the researchers found an advocate talking up the virtues of her new car
had at least one friend make the same purchase, to the tune of $718,000 in sales. A similarly satisfied non-advocate was responsible for
only $502,000 in additional conversions. In consumer electronics, the numbers are even more staggering. For purchases over $300,
advocates moved an additional $1.6 million of merchandise, compared to $570,000 attributed to their less vocal, less influential peers.
Reicheld doesnât capture the size of the peer group in which you launch all these brand conversation. That still a weakness. Not all the 10âs are the same.
The influence of human media on other humans has a REACH and FREQUENCY aspect. Like all other media.
Larry Freed says:
19 Jul 2007, 16:17
It is an interesting question you pose. First, I agree with your statement "the true value of the Net Promoter Score as a tool within companies lies simply in the focus on the customer it generates." The challenge is that net promoter will yield bad data - and with bad data comes bad decisions, garbage in, garbage out.
Another great thought you had is the search for the proper word of mouth metric. The question that net promoter suggests only measures positive word of mouth - are you likely to recommend. If you are not likely to recommend does not mean you are negative, or a "detractor". It simply means that you won't recommend. We need to measure both positive and negative word of mouth. Net promoter is totally flawed in its calculation of negative word of mouth.
So let's summarize what is wrong with net promoter.
1. Detractors are not detractors.
2. The "bucket" approach (0-6 detractors, 7-8 neutral, 9-10 promoters) leads to horrible precision in the measurement. The value of a 1 is different then a 5, a 6 is different then a 7, etc. This violates measurement 101. The net promoter calculation is statistically flawed.
3. The evidence provided that tells us net promoter works is flawed. Look close at the data. The studies done compare net promoter with prior period revenue growth. That would at best suggest revenue growth drives net promoter, not the other way around. Even the case studies in Reichheld's book are flawed. Enterprise Rental car does not ask the recommend question, they ask a satisfaction question. Yet Reichheld tries to pull the wool over our eyes and present it as a net promoter case study.
4. Net promoter does not cause or predict growth. Tim's paper helps substantiate that along with much other research that has been done.
So, one more point -- ultimately we want to measure purchase intent, loyalty and word of mouth. But what drives those things? Satisfaction (when measured correctly, such as the American Customer Satisfaction Index). So we want to accurately measure purchase intent, loyalty and word of mouth -- and we want to measure what drives those. So we can take action! Far more then one question!
Add your comment