“We have to plan for the future,” a systems executive told me recently. Well yes, that’s probably far more useful than planning for the past or present. It is, however, much harder.

Two weeks ago we explored the subject of worthless statistics and the odd preference so many people have for bad information over unbiased ignorance. (See “Pie charts and bar charts may bring comfort, but wisdom is another matter,” May 5, page 74.)

This week we talk about forecasting, an activity that fulfills the need many people have to know and plan for the future. Throughout history that need has created professions and industries. In the earliest days, the tools of the forecaster’s trade were tea leaves, entrails, the Zodiac, and crystal balls. More recently, tools like Doppler radar and computer simulations have been added to the arsenal.

Forecasting hasn’t improved much, but forecasting tools sure have. (A recent article in the local newspaper compared the performance of the National Weather Service, television meteorologists, and a guy who predicts the weather using a combination of Radio Shack gear and homemade tools. The backyard hobbyist won by a wide margin.)

And that brings us to the most pernicious forecasting tool yet devised: the market survey. How many articles have you read in this and other reputable journals presenting some industry forecast or other in which the future is presented as a fact: “Within five years, the market for bubble memory will grow to over $5 billion”?

Oops. Bubble memory flopped. It must have been personal digital assistants I was thinking of.

The accuracy of both market and weather forecasting begins to plummet beyond about one day into the future for the same reason: The systems we’re trying to predict are chaotic. And although the mathematics of chaos theory are daunting, one of its basic conclusions is pretty straightforward: When systems (actually, nonlinear systems) reach a certain level of complexity you can’t predict their future state very well.

Now what’s involved in predicting a future market? Oh, just the behavior of large numbers of individual humans. If someone would just invent Isaac Asimov’s psychohistory we could get somewhere.

Market surveys seem so scientific. But what are they really asking? “Do you expect to implement Windows NT Advanced Server in your enterprise within the next three years?”

Now who’s going to answer with an unequivocal “no?” We’re talking about a Microsoft product and a three-year planning horizon. And what does “implement” mean? Replace Novell’s NetWare? Or install a test server? Forecasts of NT Advanced Server sales have no value because there are unpredictable factors involved: Will Microsoft ship the next release on time? Will it have any crippling bugs? Will Bill Gates burn out, causing all of Microsoft’s customers to lose confidence? Will the sun turn nova and wipe out my capital-procurement budget?

Not that I want to single out NT Advanced Server. Whether it’s network computers or wireless data communication, any innovative technology just entering the market faces too many imponderables for forecasts to mean much. Why? Because market success depends as much on unanticipated details as on the basic ideas. Is the advertising campaign exceptionally stupid? Has innovation extended the life of a moribund technology? Did the designer misread the market?

So don’t let forecasts paraded as facts serve as a substitute for your own insight. Except, of course, for the enlightened predictions of your friendly IS Survival Guide soothsayer.

Satire alert

A few weeks ago, I expressed some concern about the Software Publishers Association’s and the Business Software Alliance’s admission that they inflate their software piracy estimates. (See “The Romans had some words for it,” April 7.) Some alert InfoWorld readers told me that the admission was a joke, originally published in the San Jose Mercury News and later picked up as fact elsewhere. By the time I read it in Edupage, the humor had been removed and it looked legitimate.

So here’s the situation: The numbers are still suspect for all the reasons expressed in the satire, but the SPA and BSA haven’t admitted it after all.