The Customer Service Survey

Mistaking Breadth for Depth

by Peter Leppik on Wed, 2012-02-22 16:54

Effective customer feedback programs need to consider the depth and breadth of the data they collect:

  • Depth is the amount of data the survey collects on each individual participant. Depth is important for understanding what's driving the customer's opinions and making the feedback actionable.
  • Breadth is the number of customers who participate in the feedback program. Breadth is important for generating statistically meaningful results and calculating accurate metrics.

Deeper surveys generally cost more than shallower ones, because getting the additional feedback from each participant usually requires a live interview instead of an automated survey. A program's budget will determine how much depth and breadth you can afford.

The most effective feedback programs provide a balance of both depth and breadth. At the extremes, focusing entirely on either depth or breadth often represents wasted effort:

  • Narrow and deep feedback gives a very detailed look at the opinions of each participant, but the budget normally won't allow a statistically meaningful sample. For example, a focus group or usability test where you physically bring participants to a study center and interview them at great length will give fantastic insights and generate many ideas for improvement. But this can cost hundreds to thousands of dollars per participant, making it financially impossible to include the hundreds of participants necessary to gather meaningful statistics. So it's impossible to know with certainty whether the insights you get are problems common to all customers, or represent the quirks of the individuals who happened to participate.
  • Broad but shallow feedback is the opposite. Automated surveys are nearly free so it's possible to collect feedback from thousands or even millions of customers. However, many consumers today have little patience for these surveys, so its not realistic to expect to get answers to more than a handful of questions (and response rates tend to be poor). Getting thousands of people to answer the same questions is fantastic for precisely calculating the statistical results for those specific questions, but everything else is pretty much speculation.

There's a common fallacy that you don't need depth if you have enough breadth, and vice-versa. This is mistaken: getting more detailed information from one person tells you nothing about the opinions of people you didn't talk to; and asking the same questions of more people doesn't give any information about the questions you didn't ask.

Balancing depth and breadth takes more thought than just doing a bunch of surveys, and it may require collecting feedback in more than one way. But it's the only way to make sure you're getting reliable and actionable results from a customer feedback program.

Sorry...This form is closed to new submissions.