Vocalabs Newsletter: Quality Times

Issue
86

It's Time to Ditch that Long Survey

In This Issue


It's Time to Ditch that Long Survey

Is your customer survey too long?

In my opinion, if your transactional survey is longer than five minutes (for a phone interview) or one screen (for an online survey), then yes it is too long.

But even if you're not as ruthless about survey length as I am, there's a lot of surveys out there which everybody agrees are too long. I'm talking about the five-page 25-question online surveys we all know and loathe, or the 10-minute phone surveys which just seem to go on forever.

These surveys are amazingly common, yet I have never found anyone, not a single person, who disagrees with the notion that such a long survey is much too long. And I've spoken with a lot of people who currently field long surveys, and even people who designed them in the first place. Some are even our clients. And every single person I've spoken to thinks the long survey should be shorter.

So if everyone can agree that a survey is too long, why doesn't it get changed? I've heard a lot of excuses reasons, and I'm here to tell you why these don't hold water:

  • Reason #1: It takes time to build consensus to change the survey. Guess what? Everybody agrees the survey is too long. You have consensus. Go change the survey.
  • Reason #2: We've identified 57 different business drivers we need to ask about. You can't focus on 57 different things. If you're trying to focus on that many things, you're not actually focusing on anything. Chances are there are only 3-5 of the questions that anyone ever pays any attention to, and you already know what they are. Get rid of everything else. Or if you truly cannot get away from all those secondary drivers, ask each customer about a random set of three or five. This will at least let you track the overall trends and your customers will thank you.
  • Reason #3: We need to maintain continuity with our historical data set. There's really no point maintaining a worthless data set. And anything beyond the handful of questions you're really using is not serving any function and has no value.
  • Reason #4: We have a lot of process built up around the existing survey. Change is hard, we all get it. But maintaining all that process costs time and money. There's no point wasting the resources to keep delivering reports about things nobody cares about.
  • Reason #5: We are legally required to ask all these questions, in exactly this way. Okay, this is probably the one real reason not to fix a survey which is too long. Those working in the healthcare sector probably know I'm thinking of the HCAHPS survey, a legally-mandated monstrosity comprising 32 questions (which some vendors absurdly supplement with a set of their own proprietary questions). So if this is you, sorry, you're stuck. Go write a letter to your senator.

Well-designed short surveys almost always have a higher response rate, yield more useful data, allow more focus on business goals, leave a more positive impression on customers, and are easier to manage. They're usually cheaper, too.

So what's your excuse?


Mandatory Survey Questions are Evil

You're merrily filling out a survey for some company when you come to a question you just can't answer. Maybe the question is asking about something you don't have any experience with, or maybe the question doesn't make sense (not every survey is well-written). So you skip the question and click the "Next" or "Done" button.

Only to be sent back to the same page with angry red highlighting on the skipped question and a stern warning: "You must answer this question to continue."

At this point you can (a) abandon the survey, or (b) make up a fake answer. Honestly answering the question isn't an option.

This is why mandatory survey questions are evil:

  • Mandatory survey questions don't respect the fact that the customer is doing you a favor. The first rule of conducting customer surveys should always be that the customer's feedback is a gift, and should be treated as such. If a customer doesn't want to answer a question, you need to deal with that fact and not act like you're entitled to an answer.
  • There is no situation which actually requires using mandatory survey questions. A good survey designer and a well-designed process can easily find better ways to do the things mandatory questions are there for. Skip logic? Design a path for "no answer" responses. Blank surveys? Filter them out in analysis instead of rejecting what feedback there might be. Giving incentives? If someone turns in a blank survey send them an "Oops, you forgot to fill out the survey" response.
  • Skipped survey questions are a problem of survey design, not the customer's intent. Mandatory questions treat skipped questions as though the problem is the customer doesn't want to do the survey. But you already know the customer wants to do the survey because he's doing the survey. If you have skipped questions, the problem is the survey is poorly designed. Chances are it's too long, or you're asking questions which don't make sense to the customer. In the real world, well-designed surveys have very few skipped questions: we expect just a couple percent of people to skip any given closed-ended survey question.
  • Mandatory survey questions tell the customer you don't care about his feedback. Forcing the customer to answer a survey question he doesn't want to (or can't) answer says that you would rather get no feedback at all from the customer than a survey with a single box unticked. That hardly seems like the message you want to give to someone who is, after all, taking the time to help you improve.

If you have a survey with any required questions, just make them all optional. You will get better feedback, your response rate will go up, and you will sleep better at night knowing that you aren't aligned with the forces of darkness.

Newsletter Archives