The Customer Service Survey

Awkward Survey Questions

by Peter Leppik on Thu, 2007-05-31 01:00

I heard an interesting anecdote not long ago about a customer survey:

BigBank does an ongoing customer service survey, which takes the form of a follow-up interview with customers a day or two after they call BigBank's toll-free phone number. The surveys were performed by a third-party outbound call center (lowest bidder) which didn't provide any call recordings or other data other than monthly survey data.

One of the questions on the survey is "Would you recommend BigBank to your friends or colleagues," a fairly trendy satisfaction-constellation question these days. So far so good.

A couple years ago, someone in Sales got the brilliant idea to add a follow-up question: if a BigBank customer answered "yes", then the interviewer would ask for a specific name and phone number of a friend or colleague the customer would recommend BigBank to.

BigBank, of course, would use those names as sales leads, and call the people with the pitch that "Your friend Joe Smith gave us your name."

As soon as this question was added to the survey, BigBank's "net promoter" score (the metric generated from the "recommendation" question) plummeted. Surely the quality of customer service hadn't declined, so why the big drop?

After some months of investigation, BigBank discovered that the interviewers found the follow-up question unbelievably awkward--and I don't blame them. So awkward, in fact, that they absolutely did not want to ask the question. But they had to ask the question, and if a customer answered "yes" to the recommendation question, the interviewers would get in trouble if they didn't put something down for a new sales lead.

So when a customer said "yes" to the recommendation question, the interviewer would simply enter "no" in the form for the survey--intentionally miscoding the answer to avoid having to ask the awkward follow-up. Since the interviews weren't recorded, the odds of getting caught were small.

BigBank removed the follow-up question, flogged the vendor which performed the interviews about their QA practices, and the net promoter score promptly rebounded.

There's a lesson in here: respect the customer, respect the interviewer, and respect the social context of the survey.

Sorry...This form is closed to new submissions.