Net Promoter and Customer Effort
In This Issue
People often ask, "What's the right metric to use on a customer survey?"
The answer, of course, depends on what you're trying to measure. Often the survey has more than one goal, and this will require measuring more than one metric. Unfortunately, the people promoting the Net Promoter methodology have been promoting the idea that you only need to measure one thing (and, of course, that one thing is their metric).
As a case in point, we have a client currently asking both a recommendation question (similar to Net Promoter) and a customer effort question. Customer Effort is a metric designed to measure the roadblocks a customer experiences in trying to get service, and it's a good way to gauge how smoothly a particular transaction went. Net Promoter, in contrast, measures a customer's overall relationship with the brand and company.
In this survey we noticed a curious thing: a meaningful percentage of customers who both said they would recommend the company, but who also said they had to go through a lot of effort to get what they wanted on the customer service call.
This should be surprising to anyone using Net Promoter to measure a particular customer experience--the theory being that customers who just had a bad experience will be less likely to recommend the company.
That theory may have some truth on average, but when it comes to individual customers there's clearly something else going on.
So we listened to a number of the interview recordings to better understand what the customers were saying. And the message was loud and clear: These customers had a bad customer service experience, but were loyal to the company for completely unrelated reasons.
The recommendation question was doing exactly what it was supposed to do: measure the customer's overall relationship with the company. And the customer effort question was also doing exactly what it was supposed to do: find the ways the company made it hard for customers to get the service they expected.
The lesson is simple, but often needs to be repeated. Ask the question about what you want to know. Don't expect a survey question designed to tell you one thing to measure something else.
Net Promoter and Customer Effort are two different questions which measure two different things.
One of the oldest complaints about customer service is having to wait on hold to talk to a person. It's still a problem from time to time in many companies, and we published some research on hold times as part of the mid-2013 NCSS Banking report (see page 3 of the PDF report).
We had a recent opportunity with a client to explore how well customers estimate their wait on hold. Anecdotally, we all know the customer who said he waited ten minutes but only actually spend 30 seconds in queue. For this client, they were able to supply us the actual time in queue for each customer who completed a survey, which we compared to the customer's estimate of the wait for an agent.
The results were interesting and surprising. It turns out that an individual customer's estimate of the time spent waiting bears almost no relationship to the actual queue time for that customer. There were plenty of instances of dramatic over- and under-estimates of the wait time. I'm talking about people who claimed they had to wait ten minutes but actually spent less than a minute in queue--or, conversely, people who said it was under a minute when it was actually several.
However, on average, customers' estimates of the wait time were astonishingly accurate. For example, taking all the people who said their wait time was "about two minutes", and averaging their actual queue time, it was surprisingly close to 120 seconds.
We also found that both actual and perceived wait time correlated to IVR and call satisfaction, but the perceived wait time was a stronger relationship. I suspect this may have to do with the customer's emotional state: the more annoyed he is with the call, the less satisfied, and the longer he thinks he had to wait to speak to someone.
Finally, there's a significant minority of customers (I'm guessing around 20%) who apparently are including the time spent navigating the IVR in their estimates of the wait to speak to someone. So even if the actual queue time was short, a long and complicated IVR makes the whole experience feel longer.
So the lessons we learned are:
- Queue time still matters in customer service. It feels a little old-school in this age of social media and natural language IVR, but make sure you're answering the phones promptly.
- The actual queue time and what the customer thought the queue time was are two different things. You're probably measuring the former but not the latter, but it's the customer's perception which counts.
- Making customers go through extra steps to reach a person makes them think it took longer to reach someone, and makes customers less satisfied with the service.