The Customer Service Survey

Random Randomness

by Peter Leppik on Fri, 2012-11-30 17:21

On survey questions were there's a list of choices and you ask the participants to choose the option they prefer, people are somewhat more likely to pick the first or last choice on the list. So, for example, if I ask a question like:

Choose Your Favorite Color:

  • Red
  • Yellow
  • Green
  • Blue

the question would be biased towards Red and Blue being favorite colors.

It's sometimes good practice to randomize the list of choices in order to compensate for this bias. So one person might get Yellow, Blue, Green, Red; someone else might get Blue, Green, Red, Yellow; and so forth.

Of course you don't want to do this all the time. In particular, if the choices have a "natural" order, it's best to present them in that order to make it easier to find the choice you're looking for. It turns out that in practice, most survey questions do have a natural order to the responses--we all love our Likert scales!--and so you don't often see a survey where the choices are randomized.

In particular, don't do this:

Really. Just don't do it.

(From Daily WTF)

Sorry...This form is closed to new submissions.