Why does the world of business persist in trying to make concepts work after they’ve repeatedly failed?

As pointed out last week, this seems to be a pattern. Whether the repeatedly failed idea is incentive pay, recruiting, performance appraisals, outsourcing, or waterfall software development, plausibility appears to trump the evidence, over and over again.

We need to know why. Otherwise we’ll just do it all over again with the next idea. There isn’t just one reason the world of business throws lots of good money after bad, either — there are quite a few. To get you started:

The Edison Ratio

Last week’s column mentioned this: One reason businesses persist in trying to make failed concepts work is that they should. As Edison pointed out, genius is one percent inspiration and ninety-nine percent perspiration, so the odds that failures are the result of flawed execution are high.

Also, few ideas are either purely good or purely bad, and in fact, you’d do yourself a service by eliminating “good” and “bad” from your business vocabulary altogether. Some ideas are better than others; that’s about all you can say about such things. It’s a very rare idea that’s so good it will survive hapless execution. Almost as rare are ideas so bad that they’re doomed to fail even when executed with immense skill.

The close but no stogie syndrome

The first automobiles weren’t practical forms of transportation — they were for hobbyists and tinkerers, but were in all respects inferior to the horse when you wanted to get around, even when the driver had paved roads available.

The first personal computers weren’t useful for very much, either — they were also for hobbyists and tinkerers. They needed lots of improvements before they were ready for prime time.

The take-home lesson: Many ideas that fail can, with the right refinements, become spectacular successes.

It worked once, in limited circumstances

“What do you mean, incentive pay has never worked? Companies have paid commissions since time began, and it seems to motivate the sales force pretty well.”

Yup. Commissions do motivate sales representatives — so much so that the profession has gained something of an unsavory reputation for doing anything … anything to close a deal.

Commissions work to motivate sales professionals so long as all that matters is making a sale. As companies start to care about customer retention and long-term customer relationships, commission structures either become more complex or start to actively interfere with the company’s strategic goals.

Lots of ideas work in specific circumstances. Mathematicians haven’t invented numbers small enough to describe how many work in all circumstances. Which is why very often, the right answer isn’t to either stay with an idea or to give it up. It’s to figure out where it fits and where it doesn’t.

Flock mentality

Ever hear a flock of parrots wake up in the jungle? The way it works is that one wakes up and squawks. That wakes up some more parrots, who repeat the squawk, waking up others until they’re all squawking.

That doesn’t make the original squawk a blinding insight that must be true. You’re just hearing a bunch of parrots, repeating what each other are saying.

Sometimes, in business, all that’s happened is that someone said (or wrote) something with a lot of confidence — enough so that someone else decided to repeat it. Pretty soon, enough companies are outsourcing IT to India and manufacturing to China (with whoever is promoting the idea publishing lots of books and articles explaining how great it’s going to be) that all the rest figure they’re missing the boat and start making their own plans, too.

This is especially likely when the idea being promoted creates a pleasing narrative — a story that reinforces a decision-maker’s biases.

IT outsourcing, for example, fits right into a commonplace executive bias that IT is a pain in the neck to oversee; outsource it and it becomes Someone Else’s Problem. That this doesn’t hold up to any scrutiny at all doesn’t matter, because very few people scrutinize ideas that fit into narratives that please them.

The best part of this is that when whatever-it-is doesn’t work the problem has to be with the execution. All the CEO has to do is fire someone (sorry, “hold someone accountable”) and the board of directors will be happy as can be.

Now comes the hard part: Everything you just read is about you. And me. It’s about them, too, but they don’t matter. You and I are completely vulnerable to all of the above. Which leads to this uncomfortable question:

What are you going to do about it?

Should executives receive incentive pay? Should anyone?

Last week’s KJR explored the logic behind incentive pay, and concluded it doesn’t hold up to scrutiny (I nearly said “close scrutiny,” which contrasts with the distant scrutiny that’s so popular these days).

Incentive pay shares a characteristic with at least four other well-established business practices — recruiting, performance appraisals, outsourcing, and software development — namely, the widespread certainty that when they don’t work, the problem is with the details of execution, not the fundamental concept:

Recruiting: As my friend Nick Corcodilos of Ask The Headhunter fame has been pointing out for years, the industry-standard recruiting practice of posting a position description, screening resumes based on skill-to-task matching, and so on fills fewer than one out of every ten open positions. And yet, most companies continue to pretend it works, even though, based on the numbers, it’s obviously broken.

Performance appraisals: Okay, I don’t have any documented numbers to back this opinion. Based on what I’ve seen, over the past few decades the performance appraisal process (really, practice) has become increasingly bulky and time-consuming for both managers and those they manage.

The payoff for the additional time and energy diverted to this activity? So far as I can tell, managers hate it, few employees find it valuable, and there’s no evidence that better employee performance correlates with more extensive and intensive performance appraisals.

Outsourcing: As documented in Outsourcing debunked (me, 2011), the only constant in the outsourcing industry is its failure rate. Commonly, three years into most outsourcing deals the contracting company finds itself either renegotiating their contract or terminating it altogether. Overall the numbers seem to show that between 30% and 70% of all outsources fail, depending on whose numbers you’re reading and the type of outsource they’re writing about.

But the numbers you read almost certainly underestimate the failure rate. Here in the Minneapolis/St. Paul metro, for example, it’s widely known that Best Buy is quietly unwinding its high-profile outsource to Accenture, but nobody in either Best Buy or Accenture publicly admits the whole venture was a failure.

Software development: For at least two decades, waterfall development wasn’t a way to develop software. It was the way, even though the way that preceded it (or at least one of the ways) — frequent informal conversations between business managers and programmers, with programmers showing business managers their results as soon as there was something to show and the business managers providing feedback that led to quick course-corrections — demonstrably worked.

I say demonstrably because the near-universal pre-waterfall outcome was stable, tailored-to-the-business, feature-rich applications, unlike waterfall, with its notorious 70% across-the-board failure rate.

Isn’t it interesting that when you read the Agile Manifesto, it sounds a whole lot like nostalgia for the pre-waterfall days?

At least with software development, as an industry we finally acknowledged that waterfall doesn’t work, although we needed a couple of decades dominated by dismal failure to accept this.

Too bad the Agile Manifesto hasn’t kept up with the times. It’s about software delivery to customers at a time when (1) there are no internal customers, and (2) the point isn’t software delivery, it’s successful, designed, planned business change.

Incentive pay, recruiting, performance appraisals, outsourcing, and software development. Five very different concepts. In all five cases, the business community has spent decades assuming the problem was with execution, not in fundamentally flawed concepts.

Here’s the irony: If an assumption was to be made, flawed execution was the right one. It’s the Edison Ratio in action.

Edison, you’ll recall, explained genius as being one percent inspiration and ninety-nine percent perspiration. Given this 99:1 ratio, logic dictates that when something goes wrong, it’s 99 times more likely to have been with the sweat than with the idea … with the execution, not the concept.

So the issue isn’t that business leaders, faced with failures, focused their attention on execution. Quite the opposite, this was admirable. It means they recognized that there’s no substitute for sweating the details.

No, the issue is how long it should take to figure out that the problem is the core concept after all.

In this, business leaders would do well to accept the advice of the source of so much wisdom, W.C. Fields: “If at first you don’t succeed, try, try again. Then give up. There’s no use in being a damn fool about it.”