Vocalabs Newsletter: Quality Times


In This Issue

Mistaking Breadth for Depth

Effective customer feedback programs need to consider the depth and breadth of the data they collect:

  • Depth is the amount of information the survey collects from each individual participant. Depth enables root cause analysis and making the feedback actionable.
  • Breadth is the number of customers who participate in the feedback program. Breadth allows statistically meaningful results and accurate metrics.

Deeper surveys generally cost more per survey but there's a lot more useful information in each survey. The most effective feedback programs provide a balance of both depth and breadth. At the extremes, focusing entirely on either depth or breadth often represents wasted effort:

  • Narrow and deep feedback gives a very detailed look at the opinions of each participant, but the budget might not allow a statistically meaningful sample. For example, a focus group where you physically bring participants to a study center and interview them at great length will give fantastic insights and generate many ideas for improvement. But this can cost hundreds to thousands of dollars per participant, making it impossible to include enough people to gather meaningful statistics. A statistically meaningful sample allows you to know whether the insights you get are common to many customers or represent the quirks of a few individuals who happened to participate.
  • Broad but shallow feedback is the opposite. E-mail and IVR surveys are nearly free so it's possible to collect feedback from thousands or even millions of customers. However, many consumers today have little patience for these surveys, so response rates tend to be poor and it's not realistic to expect to get answers to more than a handful of questions. You may get very precise statistical results for a few specific questions, but root causes will be mostly speculation and inference.

There's a common fallacy that you don't need depth if you have enough breadth, and vice-versa. This is mistaken: asking the same questions of more people doesn't give any information about the questions you didn't ask; and getting more detailed information from one person tells you nothing about the opinions of people you didn't talk to.

Balancing depth and breadth takes more thought than just doing a bunch of surveys, and it may require collecting feedback in more than one way. But it's a far more cost-effective way to spend the feedback dollars than going to either extreme.

Wide Plus Deep

Automated customer service surveys (via e-mail, web, or IVR) are cheap, but they often suffer from poor response rates and can usually only have a handful of questions before customers start abandoning the survey. That severely limits the quality and depth of the feedback.

Phone interviews are almost exactly the opposite: customers are much more willing to participate when there's a live person on the other end, and they are willing to answer a lot more questions as long as the interview is well-structured. Interviewers can also probe and go in depth in a way a computer can't. But it's inherently more expensive to conduct a live interview.

A hybrid approach offers the best of both worlds, combining vast ocean of shallow customer feedback from an e-mail or IVR survey with depth and insight through  smaller number of highly targeted interviews.

In most companies most customers get reasonable service most of the time. If a customer was satisfied with the experience there's relatively little to be learned about how to improve. On the other hand, if the customer was not satisfied you can get a lot of useful information through an in-depth interview. Running both an automated survey and an interview process in parallel focuses the survey dollars and effort where it is most effective.

The automated survey generates high-level statistics across the organization and identifies underperforming segments. Live interviews are targeted at those underperforming areas and generate actionable feedback about how to improve. Customers are selected for one survey or the other based on whether the statistics suggest a higher probability of dissatisfaction.

The end result is an extremely cost-effective package: tons of high-level statistics across the entire organization plus highly detailed feedback where the company needs it most, in a single coherent process to support both strategic and tactical decisions about how to deliver the customer experience.

Newsletter Archives