The Customer Service Survey

Goodhart's Law

by Peter Leppik on Fri, 2016-01-15 15:35

In the field of macroeconomics, Goodhart's Law states that "When a measure becomes a target, it ceases to be a good measure."

As an economic theory this is the rough equivalent of Murphy's Law, though with a kernel of deep truth at the core. Macroeconomic measurements distil an enormously complex system down into a handful of simple numbers that require considerable effort to measure. For example, in the mid-20th century in the United States, we had a problem with inflation. Low inflation is desirable because it tends to correlate with economic stability and predictability and encourages the middle class to save and invest for the future. But when policymakers initially tried to slow inflation through wage and price controls rather than addressing the underlying problems in the economy, the result was an unbalanced economy and (eventually) the stagflation of the 1970's. Of course this is a grossly oversimplified summary of 20th century economic history, but the point is that by trying to force inflation to hit a target, the inflation rate stopped being a good proxy for economic stability and predictability.

Goodhart's Law in Customer Experience

Goodhart's Law applies in the world of Customer Experience, too.

Most of the core metrics in any CX effort (for example, survey scores like Net Promoter or Customer Satisfaction; or internal metrics like Delivery Time) are used because they are strongly correlated with customers' future behavior, positive word-of-mouth, and long-term growth of the company.

But if you try to turn a CX metric into a target, it may no longer be useful as a measure of the customer experience. That's because the things you really want to change (customers' future purchases, positive word-of-mouth, long-term growth, etc.) are the result of many complex interactions inside the company and between the company and its customers. And its often easier to hit a goal by gaming the system than it is to fix the underlying problems.

For example, in the case of ABRA Auto Body I blogged about a couple days ago, the company almost certainly did not set out to create a survey which would yield inflated, meaningless scores. Instead, they most likely determined that high survey scores were often strongly correlated to repeat business and new customers through recommendations.

But rather than explore the root causes of high (or low) customer satisfaction and address those, the company probably decided to simply give managers an incentive to hit a certain survey score and let them figure out how to do it.

The result is that it's much easier for a manager to print off a bunch of fliers instructing customers on how to answer the survey, than it is for them to think about how the customer journey might be improved. (It's possible that ABRA doesn't even give managers the authority or budget to change the things that might matter, in which case the manager may have no choice but to try to game the survey.)

The lesson should be obvious: If you want your CX metrics to be useful measurements of your customer experience, then you need to be very wary of how incentives invite manipulation.

Sorry...This form is closed to new submissions.