HomeIndustry Commentary

Avoiding Type I and Type 2 errors (first appeared in InfoWorld)

Like Tweet Pin it Share Share Email

Technical people, such as programmers, engineers, and scientists, have gained a reputation among nontechnical folk as poor communicators. Most of the problem arises not from poor communications skills but from an excess of them. Tech-folk — the real ones, not the jargon-meisters who substitute neologisms designed to impress the rubes for actual knowledge — assign precise meanings to precise terms to avoid the ambiguity that marketing professionals (for example) not only embrace, but sometimes insist on.

Sometimes precision requires complex mathematics, because English, evolved to handle prosaic tasks like describing the weather, explaining how to harvest crops, and insulting that ugly guy from the next village, isn’t quite up to the task of explicating the nature of a 10-dimensional universe. Many physicists communicate poorly with the innumerate.

Other times precision simply requires a willingness to make distinctions. Take, for example, the words “theory” and “hypothesis.” Most people use theory to mean “an impractical idea,” and hypothesis to mean “a brilliant insight” (“hypothesis” has more syllables so it must be more important). Scientists, in contrast, know that theories have been subjected to extensive testing, and can be used to address real-world problems. It’s hypotheses that are simply interesting notions worthy of discussion and (maybe) testing.

This is a distinction worthy of a manager’s attention, since a lot of our responsibilities boil down to being a broker of ideas, figuring out which ones to sponsor or apply and which to reject or ignore. Last week’s column dealt with how you can assess relatively untested ideas — business hypotheses, if you like. This week we’ll cover the harder question of how to deal with some of the well-worn thoughts that, while popular, may still be poor choices for your department and which may even be downright wrong, no matter how widely used.

Your first step in assessing an idea that’s in wide use (assuming it’s applicable to one of your priority issues) is to show some respect. Keep your ego out of it. Most of us have an ego-driven tendency toward what scientists would call Type 1 and Type 2 errors.

We make Type 1 errors — rejecting good ideas — through our unwillingness to admit that someone could think of something we can’t instantly understand. Remember, lots of smart people have applied these ideas, so they’re unlikely to be an example of mass stupidity. If the idea may apply to your situation, make sure you understand it — don’t reject it through Argument from Personal Incredulity (a term borrowed from the evolutionary scientist Richard Dawkins and discussed at length last week).

Our egos also lead us to the opposite problem, by the way. We commit Type 2 errors — accepting bad ideas — through our desire to be the one to find and sponsor something new and useful.

Next step: Make sure the idea has been tested and not simply used a lot. Businesses survive lots of screwy notions. Using and surviving an idea doesn’t mean it led to valuable results. Look for business outcomes, not warm fuzzies. (In the world of science, psychotherapy has received extensive criticism on the same grounds.)

Your last step is to look at the idea’s original scope. Well-tested scientific theories are rarely invalidated. Instead, as with Newtonian physics (which doesn’t work in quantum or relativistic situations), scientists discover boundaries outside which they don’t apply. Well-tested business ideas also may fail when applied outside their scope. As an example, Total Quality Management (TQM) is unsurpassed at perfecting manufacturing processes, where quality consists of adherence to measurable specifications. TQM’s successes outside the factory, however, have been spotty.

One more thought: Have enough self-confidence to respect your own expertise. Doing something because the experts say so is as miserable an excuse as “I was just obeying orders.”

Don’t worry — if you need an expert to back up the course of action you’ve chosen you can always find a tame consultant willing to recommend it … for a small fee, of course.