Product Teams

6 Takeaways from 6 Months of NPS: A Retrospective

Published Dec 9, 2016

Surprise; gone are the days of collecting 90% of your customer’s lifetime value on Day 1. Consumers today not only have greater choices but as markets continue to become more and more crowded, their alternative product choices are endless. As a result of this, the all important relationship that you build with your users is also a relationship that is easy to break ties with.

The equation is pretty straightforward. Customer pays for your product. Customer has poor experience. Customer churns.

And while monitoring product usage is one piece of the puzzle, it doesn’t provide us insight into how the customer is feeling.

We just wrapped up our first 6 months of NPS and thought we would share a couple of learnings.

For those unfamiliar with NPS, it’s one of the many ways to gather customer feedback. Using a 10 scale system developed by Bain and Company back in 2003, it asks users how likely they would be to recommend you to a friend or colleague, and then provides them an opportunity to provide qualitative feedback.


How NPS is calculated:


Our 6 takeaways from 6 months of NPS:

1. An NPS Number by Itself Is Meaningless

You just received a 9/10 from a super satisfied user but they didn’t leave any feedback as to why they’re happy. When left alone, this high NPS score is simply a vanity metric; it’s not actionable.

Consider your overall NPS score by itself and you’re in the same boat. You have an industry leading NPS score but you’re flying blind as to the reason behind it. Was it the new feature you released last month? Or maybe it was the improvements you made to your onboarding flow? If you’re like most product teams, you’ll be left scratching your head wondering what the main reason for a score was.

2. A Simple Change in Follow-up Can Drastically Impact the Qualitative Feedback Response Rate

In the beginning, we asked users to “Please tell us why you gave this rating. We appreciate your feedback.”

The wording and tone that you use to solicit feedback from your users should be the same wording and tone that you would use if chatting face-to-face.  When brainstorming ways to improve our qualitative response rate, we decided to A/B test a different follow-up method to see the impact.

When it comes to the initial NPS question, you should stay in-line with the universally accepted “how likely are you to recommend X service to your friend or colleague?” However, when it comes to the follow-up question, NPS allows for a bit of flexibility.

We’ve since transitioned to “What’s the main reason for your score?” and seen drastically higher qualitative response rates.

3. Give Your Users Time to Get Into Your Product Before Displaying an NPS

“You’re annoying me with questions while I’m trying to log in.”

“The NPS banner came up at a really bad time (trying to log in) while trying to gather some data/info about something and I just picked something random….”

“You hijacked the page with the survey and finding how to click out took too long.”

This is actual feedback from Pendo users. When designing and deploying NPS to your users, consider giving them time to get settled in the product before asking for feedback.

4. Have a Playbook, and Use It

Our playbook as it stands today is far more in depth than what it was 6 months ago when we started this project. Start small in order to keep it manageable. It can be as simple as:

a. Acknowledge User Feedback via Email to Anyone Who Submits It

Tip: Manually email each user who provides qualitative feedback. The personal touch thanking them for their feedback goes a long way.

b. Find a Way to Internally Share Your Nps Results

Every single person can benefit from this sort of feedback from the customer. Tip: We use Zapier to push our NPS results into a Slack channel with a bunch of extra data to help other teams gain context to the users’ response.


c. Take Action

We’ve talked in previous posts about how we love to dogfood our own product and NPS analysis is no different. Let’s say a user expressed frustration using a certain feature. Use your analytics solution to uncover if other users are also struggling with an area of the product. Could they benefit from an in-app tooltip explaining how to use the feature? Maybe they need a helping hand walking through a process?

5. Even customers who give you 10’s can be valuable sources of product feedback

We have a goal to reach out to every customer who provides feedback. Good or bad, we reach out. As a result, we’ve had some incredible feedback come in from our promoters, even a user who took the time to put together a ~10 min video walking us through the various parts of the product he uses on a daily basis and improvements we could make. This is feedback that we would have never received had we not reached out to that user.

Like we explained earlier, an NPS rating of 10 is meaningless if you don’t know the reason behind it. Fast forward 9 months later and the same user submits an NPS rating of 4. “But they gave us a 10 last time, what gives?”

6. The Entire Company Owns NPS

A single individual can own the playbook, but the entire company owns the score. NPS is a reflection of every single department from Sales to Engineering and everyone in between. To keep the momentum around NPS alive, we are constantly discussing changes in our own score and ways we can improve it. We’ve found it’s a great way to keep everyone completely focused on the customer and their experience with our company.