Ever since Fred Reichheld heralded Net Promoter Score (NPS) as the “One number you need to grow” in his 2003 Harvard Business Review article, businesses around the world have adopted the metric as a key loyalty and satisfaction metric, even as critics have argued that NPS not only fails to predict growth as its proponents claim, but can divert attention and resources from more important customer loyalty metrics. A new article in the UK’s The Register highlights another way that companies can set themselves up to fail through NPS: using the score to evaluate individual employees.
By Rick Ferguson
The most enduring critique of NPS came from a landmark 2007 research report from a group at Ipsos Loyalty led by Timothy Keiningham and published in the Journal of Marketing. Here’s a money quote summarizing the critique from another paper critical of NPS, this one published by the University of Cambridge just last year:
“Questions have been raised about the veracity of the claims that the NPS is the only score needed. For example, Keiningham et al. (2007) replicated the analyses used in net promoter research and compared the findings of Reichheld (2003) and Satmetrix with the American Customer Satisfaction Index using the same industries employed in Reichheld's study. Their research rejects the claim that the Net Promoter Score (NPS) is the 'single most reliable indicator of a company's ability to grow.' In its macro-level analysis, the study found no real indication that average levels of attitudinal loyalty metrics significantly correlate with the relative change in revenue within the respective industry. Furthermore, single metrics alone cannot predict customer loyalty and consequently are unlikely to deliver actions to managers. Customers' loyalty-based behaviors are multidimensional and therefore a better measurement tool is required.”
This and similar critiques have done little to dent the persistence of NPS as a key boardroom metric. The brilliance of NPS lies in its simplicity; when you have 15 minutes to present the results of your growth strategy to the board, which is the easier story to communicate: A 50-page PowerPoint presentation chock full of a wide variety of difficult-to-summarize data that attempts to document increases in lift, retention, or lifetime value; or a single slide that shows NPS rose by 20 percent in the last quarter? Never mind that the increase in NPS may or may not translate into more loyal buying behaviors, and therefore to growth. As long as NPS is on the rise, heads in the boardroom tend to nod approvingly.
The Register article, meanwhile, highlights a potentially disturbing trend in which companies use NPS to evaluate individual employees, often tying their performance incentives to maintaining a certain NPS score. The problems with this approach are myriad, from the small sample size that skews reporting to the collection of bad data that can obscure real customer service issues. Using the example of an automobile dealer evaluating its sales staff via NPS, the Register article demonstrates how easily, and how quickly, the company’s sales staff will learn to game the system:
“Smart salespeople (the ones who can do probability calculations) will realise that if they have only closed one sale by the week’s end then they should desperately avoid closing another because their chance of a bonus will drop from 70 per cent to 49 per cent if they do. Was that really the intention of the company when it implemented the NPS in this way? Doing the initial calculations for this is painfully simple (how hard is it to multiply 0.7 x 0.7?) but people don’t do them and, because they don’t see the non-linear relationships, they assume linearity with increasingly ludicrous consequences.”
So, perhaps we shouldn’t evaluate a small sales staff by tying each salesperson to an artificially-maintained NPS score. What about organizations that make NPS the centerpiece of their customer loyalty efforts? The biggest fallacy in NPS use lies in the idea that it measures something other than what it actually measures. Just as standardized academic tests are most effective at measuring a student’s ability to perform well on standardized tests, so too is NPS most effective in measuring the stated willingness of customers to recommend you at the moment you asked them the question. To ask it to do more than that is to place too much faith in a single-question survey.
Remember: on an 11-point scale, NPS treats a “1” response identically to a “6” response – and there may be a vast difference in impact, not to mention the absence of useful data, between those two responses. Placing NPS in its proper context as a single snapshot of customer satisfaction, rather than as the “one true predictor of growth,” will allow it to remain a useful tool—one of many in the toolbox, rather than as a single hammer forever in search of a nail.
Rick Ferguson is Editor in Chief of the Wise Marketer Group and a Certified Loyalty Marketing Professional (CLMP).
