Disclaimer: in no case do we pretend to be the ultimate truth. On the contrary, we want to discuss the popular metric. Our observations are recommendatory and based on personal experience 😋
Who doesn't know about NPS? Today it is one of the most popular metrics of customer experience. Top management adores it too.
As it often happens, something popular is being transformed and adjusted to fit the specific company. It's a natural process with its pros and cons. Net Promoter Score is no exception here.
And recently, we have increasingly come across variations of the NPS – reformulated questions, a different rating scale, a different context in which the user should answer the question. In this article, we would like to dot the i's and analyze the main mistakes in using this metric.
Let's start with the basics of NPS. If you are not so interested in this part, use the table of contents and go straight to the errors.
Net Promoter Score is an index of consumer loyalty. A metric that shows a customer's attitude to your company. And helps discover how likely they are to recommend you to their friends or relatives.
Frederick F. Reichheld, who first proposed this term in 2003, conducted many surveys, tested creatives, and found a question the answer to which best-predicted customer behavior:
"How likely is it that you would recommend this company to a friend or colleague?"
The NPS score was also strongly correlated with the growth of companies over time. Reichheld argued that this metric has a right to exist since recommendations and word of mouth are the strongest drivers of revenue growth for most companies.
He was right. According to Nielsen research, 83% of respondents trust the recommendations of family members and friends more than any other form of advertising. Even one dissatisfied loyal customer can cost you several new ones at once! A loyal user is also almost 5x times cheaper than a new one and is likely to return for purchases or clicks.
At the same time, even though NPS measures the attitude towards the company as a whole, problems on any part of the user path can affect the indicator. Imagine that the airline you are flying with today has opened only one check-in desk for 300 passengers. Tired and exhausted, you eventually get on the plane. Smiling flight attendants deliver drinks and food. After 3 hours of a comfortable flight, you land and receive your luggage without delay.
Having experienced such a negative experience in only one section of your Customer Journey, will you be able to positively rate the flight as a whole? Or recommend airlines to your loved ones? Probably not. NPS is an indicator – something is not working as it should. And it affects the reputation of the entire brand.
That's why NPS has solidified as a core metric of customer experience. Today, the Net Promoter Score is measured by the largest companies in the world, including Apple, Microsoft, eBay, PayPal, and Western Union.
How to measure NPS
In its original form, NPS was based on a single question. To answer, the user had to estimate the probability on a scale from 0 to 10.
The respondents were then divided into three groups (0-6 points, 7-8, and 9-10). The average group was discarded, and the percentage of dissatisfied respondents was deducted from the percentage of respondents who scored the highest.
The main errors in measuring NPS
The essence of any methodology is that it exists within a certain framework. These frameworks appeared for a reason. The methodology was tested hundreds, thousands of times, everything unnecessary was discarded, and only what is tested and worked left. We follow the same approach when working with NPS.
Any changes to the questions or the counting system create new entities – such surveys can hardly be called NPS. It means the conclusions drawn based on the estimates may be fundamentally wrong and lead the company in a different direction.
Therefore, we will analyze the main postulates of the Net Promoter Score and tell you what changes in the system we consider critical.
Question
It is no coincidence that on the official website of the NPS, the question "How likely is it that you would recommend this company to a friend or colleague?" is called the ultimate.
Of course, users can be asked any question with a scale from 0 to 10 for evaluation, but it's far from the fact that the results will be valid. The answer of the respondent directly depends on which question you ask. We strongly recommend using the classic question when measuring NPS.
Remember that NPS is based on the evaluation of the probability of a recommendation, so questions like: "With what probability will you make a purchase in store N again?" are irrelevant for measurement by the Net Promoter Score system.
Non-classical rating scale
The NPS is based on a scale from 0 to 10. The answers to the shortened version of the scale (for example, from 1 to 5) can't be calculated using the NPS system.
The same goes for graphic elements. For example, as in this variant:
How can the user navigate which of the emojis to click? And what if you need to choose between 6 and 7 – the border between Critics and Neutrals? Which is stronger in perception – grades 6 and 7 or sad emojis? In our opinion, it is easier to determine the answer to such a question with the help of numbers.
Another critical detail concerning the original scale – be careful, it is 11-point and starts from zero. Again, we can say that the difference is not significant, but the absence of zero on the scale can affect the final result by up to 9%.
The location of the ratings is not in a row
The 11-point scale interface is hard to fit in one line, but the game is worth the candle. The fact is that, in our experience, not all respondents tend to focus on the answer options in surveys.
Not so long ago, our clients watched the user during usability testing of their site. During completion of the task, our form appeared, which offered to evaluate the content on the page. The user quickly clicked the neutral emoji and continued to search for information on the site.
To the question: "Why did you put this mark?" he replied: "Unconsciously." Now we are trying to approach the assessments without comments more consciously. What conclusion can be drawn? We don't know which part of the users act impulsively, but obviously, there are such respondents.
It can also be difficult to perceive two rows of a scale. And then not the most attentive users will make your NPS not valid. We are talking about small samples. If several thousand people participate in the survey, random answers will drown in the total number.
No additional open-ended question
When you measure NPS, you most likely have a goal or desire to improve it. But if users just rate you, you'll never know why you got 2 out of 10 and how to fix it. The example from the previous paragraph also shows this well.
You can endlessly generate hypotheses and check with the updated NPS. But by itself, the NPS will not tell you exactly what and in which sections of the Customer Journey are wrong. Ask respondents an additional open-ended question, for example, "Please comment on your mark." And from that moment on, the metric will live a new life.
We strongly recommend experimenting with open-ended questions. The classic annoying calls "Please rate the quality of service" or "Help us get better" often generate negativity.
Sincerely tell the respondents why you ask them to put all these endless ratings and write reviews.
The wrong moment for the survey
The client needs to be allowed to experience the company's processes before finding out the probability of a recommendation. Remember situations when the company seemed flawless from first sight, but then it made some mistakes: slow support response time, delays in returning money, the bad quality of goods, reputational scandals.
That's why NPS is often sent in e-mail newsletters within a week or even a month after purchase. On average, a good time for a Net Promoter Score survey is ten days after purchase/start of use/delivery and 30 days after installing the application.
On the other hand, if you are selling a service or an impression, on the contrary, you should ask a question right away. As time passes, the client may forget what he thinks about the company.
Focus on Critics
The reasons for criticism are crucial, but understanding why loyal customers love you will bring the company no less benefit. Therefore, excessive focus on Critics is not always obvious.
People who give top marks will give you many cool insights as well. In addition, they are more willing to make contact and agree to participate in more serious research.
NPS measuring the CX of individual CJM stages
It is one of the most common mistakes that we have faced. Imagine scrolling the main page of the site. And then a pop-up with an NPS survey appears in front of your eyes. Why should you recommend a company if you are interacting with it for the first time and only for 20 seconds?
Another example. You bought a product on the website, and you are immediately asked a classic NPS question on the "Thank you" page. Can the assessment at this moment be objective? After placing an order, there is a huge series of processes that can make your experience both positive and negative in a couple of hours.
If we take a simple example of e-commerce, then a lazy call center, non-working (or defective) goods, an inconvenient return process, and much more can affect CX. So how, in this case, can you trust the assessment that you want to measure at the very beginning of the customer journey?
NPS cannot measure individual stages of CJM. You can measure CSI, CES – the average rating of site satisfaction, but NPS is not suitable for these purposes. It is better to use the Net Promoter Score only when you want to understand the general attitude toward your company.
To do this, as mentioned above, the user must marinate in your processes for some time. Then the mark will be as close as possible to his actual feelings.
NPS that does not pass the test of time
Another big problem comes from the previous one: the evaluation of the same user is usually not re-measured over time.
There is nothing permanent – even if you counted the NPS correctly, this is not a guarantee that the promoter will not change his point of view after a couple of months. In our experience, such metamorphoses most often occur when a person gets into a scenario that leads him in a circle. As a result, the one we consider a promoter becomes a neutral or, worse, a critic.
Understanding why people move from promoters to other groups is one of the most valuable insights. There is a reason that product and UX researchers have a Diary Studies method. Its main task is to track how the attitude towards the product changes over time and understand the reasons for these changes.
It works very well when we evaluate complex products, for example, a streaming platform. The user often gets a quick wow effect at the beginning of his CJM, but gradually notices more and more shortcomings.
Something similar, in our opinion, should be done with NPS. To begin with, make sure that you are assembling it correctly. Re-measure it over time and pay attention to those users who have moved from one segment to another. Moreover, critics who have suddenly become promoters should also be interesting.
This way you will better understand why your company (and most importantly – by what processes) forms different experiences for people.
NPS, coupled with open-ended questions, can be the coolest source of insights for your company. But in fact, no one forbids you to test various questions and measurement systems. Just don't call it NPS, and don't try to calculate the metric according to the standard methodology.
The responsibility for the results in case of experiments will be only yours. But keep in mind that the good old NPS with its classic question and rating scale will always be ready to help you get useful data from users.
