Vocalabs Newsletter: Quality Times


In This Issue:

Awkward Questions

By Peter Leppik

Writing an interview script isn't difficult, but there are some pitfalls if you're not careful. A few months I heard about a seemingly small change to a survey which had a big impact on the results.

BigBank (not the company's real name) did an ongoing customer service survey which took the form of a follow-up interview with customers a day or two after they call BigBank's toll-free phone number. The surveys were performed by a third-party outbound call center which didn't provide any call recordings or other data beyond monthly survey scores.

One of the questions on the survey was "Would you recommend BigBank to your friends or colleagues," a fairly trendy satisfaction-related question these days. So far so good. A couple years ago, someone in Sales got a brilliant idea: if a BigBank customer answered "yes" to the recommendation question, then the interviewer would ask for the name and phone number of someone the customer would recommend BigBank to.

BigBank, of course, would use those names as sales leads, and call the people with the pitch that "Your friend Joe Smith gave us your name."

As soon as this question was added to the survey, BigBank's "net promoter" score (the metric generated from the "recommendation" question) plummeted. Surely the quality of customer service hadn't declined that dramatically, so why the big drop?

After some months of investigation, BigBank discovered that the interviewers found the follow-up question unbelievably awkward. So awkward, in fact, that they absolutely did not want to ask the question. But if a customer answered "yes" to the recommendation question, the interviewers would get in trouble if they didn't put something down for a new sales lead.

So when a customer said "yes" to the recommendation question, the interviewer would simply enter "no" in the form for the survey--intentionally miscoding the answer to avoid having to ask the awkward follow-up. Since the interviews weren't recorded, the odds of getting caught were small.

BigBank removed the follow-up question, flogged the vendor which performed the interviews over its QA practices, and the net promoter score promptly rebounded.

There's a lesson in here: customer surveys don't take place in a vacuum. There's a social context to the conversation between the interviewer and the customer, and a script which doesn't respect that relationship won't work.

October SectorPulse Results

By Peter Leppik

The most recent round of SectorPulse surveys comparing customer service quality in the mobile phone and financial services industries concluded at the end of September. This is our sixteenth quarterly report on mobile phone companies, and our tenth report on financial services companies.

Mobile Phones

The biggest question in the mobile phone industry was whether the introduction of Apple's hotly anticipated iPhone would have any impact on customer service levels, as it went on sale just two days before the end of the prior survey period.

We don't have any data which directly relates to the iPhone, but there was a clear industry-wide trend towards lower-quality service and more customer dissatisfaction as compared to the June quarter. This could be iPhone related, as we would expect such a product introduction to cause increased subscriber churn and increased call volume across the board.

In the September quarter, Verizon Wireless came out on top, though Verizon, AT&T, and T-Mobile all lost ground in our standard benchmarks. Verizon earned a "B" for Caller Satisfaction and a "B" for Call Completion. T-Mobile posted a "B" for Caller Satisfaction and a "D" for Call Completion, while AT&T earned all "C's."

Our survey sample for Sprint was too small to issue letter grades this quarter, but the company's raw scores put it in the "D" range on both benchmarks.

Financial Services

We publish quarterly data in the Financial Services industry and use rolling six-month statistics. We currently track Bank of America, Citibank, PayPal, Wachovia, Washington Mutual, Wells Fargo, and Wachovia; though our survey sample for Wachovia has never been sufficient to issue letter grades to the company.

Most of the grades this quarter, as usual, were "C's" and "D's", indicating below-median scores. Bank of America took first place with a "C" for Caller Satisfaction and a "B" for Call Completion. Even though we did not issue Wachovia letter grades, the company's survey scores were in the "A" range for Caller Satisfaction, but in the "C" range for Call Completion.

Newsletter Archives