The Customer Service Survey

Getting It Wrong

by Peter Leppik on Wed, 2014-05-07 15:30

A little over a year ago, Forrester Research issued a report called 2013 Mobile Workforce Adoption Trends. In this report they did a survey of a few thousand IT workers worldwide and asked a bunch of questions about what kinds of gadgets they wanted. Based on those survey answers they tried to make some predictions about what sorts of gadgets people would be buying.

One of the much-hyped predictions was that worldwide about 200 million technology workers wanted a Microsoft Surface tablet.

Since then, Microsoft went on to sell a whopping 1-2 million tablets in the holiday selling season (the seasonal peak for tablet sales), capturing just a few percent of the market.

At first blush, one would be tempted to conclude that Forrester blew it.

Upon further reflection, it becomes clearer that Forrester blew it.

So what happened? With the strong disclaimer that I have not read the actual Forrester report (I'm not going to spend money to buy the full report), here are a few mistakes I think Forrester made:

  1. Forrester was motivated to generate attention-grabbing headlines. It worked, too. Ignoring the fact that there have been Windows-based tablets since 2002 and none of them set the world on fire, Forrester's seeming discovery of a vast unmet demand for Windows tablets generated a huge amount of publicity. Forrester might also have been trying to win business from Microsoft, creating a gigantic conflict of interest.
  2. Forrester oversold the conclusions. The survey (as far as I can tell) only asked IT workers what sort of tablet they would prefer to use, at a time when the Microsoft Surface had only recently entered the market and almost nobody had actually used one. That right there makes the extrapolation to "what people want to buy" highly suspect, since answers will be based more on marketing and brand name than the actual product. Furthermore, since this was a "global" survey, there was probably a substantial fraction of the population outside the U.S., Canada, and the E.U. who are unlikely to buy (or be issued) a tablet of any sort in the near future.
  3. Forrester let the hype cycle get carried away. I found many many articles quoting the "200 million Microsoft Surface Tablets" headline without any indication that Forrester did anything to tamp this down. Forrester's actual data basically said that about a third of IT workers surveyed said they would prefer a Microsoft-based tablet rather than Android, Apple, or some other brand, and if you believe there are about 600 million information workers worldwide (which Forrester apparently does), that's 200 million people. When that morphed into "Forrester predicts sales of 200 million Surface tablets," they did nothing to bring that back to reality.

All this is assuming that Forrester actually did the survey right, and they got a random sample, asked properly designed questions, and so forth.

At the end of the day, anyone who built a business plan and spent money on the assumption that Microsoft would sell 200 million Surface tablets any time in the next decade has probably realized by now that they made a huge mistake.

As the old saw goes, making predictions is hard, especially about the future.

Sorry...This form is closed to new submissions.