“Markets don’t fail. They always allocate goods and services perfectly,” a correspondent explained in response to last week’s column about market failure and how its sources might apply to IT shops that act as independent businesses selling to “internal customers.”

My correspondent’s position was, however, unassailable: Markets do always allocate goods and services perfectly, so long as your definition of “perfect” is “whatever allocations the marketplace delivers.”

With non-circular definitions, though, market failures are very real. Take my favorite example, covered in this space almost a decade ago:

The Dollar Auction

I auction off a dollar. There’s just one change to the usual high-bidder wins auction rules: The runner-up has to pay me his last bid too. So if the winning bid is a nickel and the runner up stopped at 4 cents, the winner would net 95 cents, I’d get 5+4=9 cents, and the runner up would be out 4 cents.

Except the runner up wouldn’t stop. He’d certainly bid 6 cents instead of losing the auction. And so on, and so on, until the high bid is a buck and the next highest is 99 cents. The second highest bidder now has to either bid $1.01 for my dollar — losing a penny on the deal — or stop, losing 99 cents. As losing a penny is better than losing 99 cents, there’s no end to the escalation.

Which, as I wrote in 2007, looks a lot like a politically high-stakes project that’s going off the rails. The sponsor can either throw good money after bad or cut her losses. But as each click of the throw-more-money-at-it ratchet looks to have a better ROI than losing everything invested thus far, and also avoids the political embarrassment of backing a loser, the train wreck continues into the indefinite future.

Customer Incongruence

The term “customer” involves three very different roles: Decision-maker (true customer), consumer, and wallet. When the same person fills all three roles, call it customer congruence, and market forces do what they’re supposed to do.

Customer incongruence (my term) happens when different people occupy the different customer roles, as when the family goes to McDonald’s for a Happy Meal. Be honest. It’s the kids who made the buying decision. Consumers? That’s the whole family. Mom or Dad are merely the wallet.

Healthcare provides another example of customer incongruence. The patient is the consumer, the insurance company is the wallet once co-pays and deductibles have been left behind. But who makes the buying decisions for your health care? For most of us it’s our doctor, who tells us what drugs to take and what surgeries to undergo.

That’s right: The seller of healthcare services holds the most important customer role: decision-maker. Which often leads to such inconveniences as buying a very expensive pharmaceutical solution when a relatively inexpensive alternative would be just as efficacious.

Which is not to suggest we should all prescribe our own treatments. Even with WebMD, few of us know enough. So while neither physicians nor IBM’s Watson are perfect diagnosticians, the alternative — self-diagnosing and self-prescribing patients — would result in a lot of unnecessarily dead people.

It’s a customer incongruity, which explains, at least in part, why the U.S. healthcare system is such a mess.

But IT organizations that act as a sellers to internal customers create customer incongruities that are just as challenging. The parallel with healthcare providers is, I hope, clear: Business executives and managers know where it hurts, but selecting or building IT solutions is complex enough to require professionals who know the field. They have the needed expertise.

As a result, to a very real extent, the IT organization acts as both seller and buyer of the company’s portfolio of information technologies.

The [partial] solution involves both formal governance and informal relationship management: Governance in the form of an IT Steering Committee or the equivalent — business managers who acquire enough expertise to oversee decisions about the company’s IT investments; relationship management in the form of the same trust-building you engage in with your doctor (and vice versa), and for the same reasons.

The better, admittedly partial solution is to not consider the rest of the business to be IT’s customer in the first place. It doesn’t do much for dollar-auction situations — what’s needed there is an executive culture that makes risk-taking safer by accepting that risk means some efforts must fail to pan out.

But customer incongruities go away when IT has no customers — when IT and everyone in the business collaborate to figure out and implement desired business changes.

I don’t know if it’s good economic theory. My experience, though, tells me it works quite well.

George Will is catching up to me.

Two weeks ago he caught on to something I covered 21 years prior — that the average American is more affluent by most measures than the wealthiest of historical figures.

Then, last week he wrote about Baumol’s disease a mere 16 years after yours truly wrote about it in this space.

Do you think Mr. Will is mining the KJR archives? Are you? Please do. There’s good stuff in there, if I do say so myself.

In any event, Baumol’s disease is why sometimes you can’t increase productivity no matter what you try. It’s because, to use the classic example, the number of musicians Beethoven needed to play one of his string quartets back when he was composing is exactly the same as the number needed today, when he’s decomposing.

Baumol’s disease is an example of “market failure” — situations where a marketplace fails to efficiently allocate goods and services.

Which has what to do with managing IT?

I’m glad you asked. The answer is, nothing at all, if you’re among the enlightened IT leaders who have rejected what used to be considered best practice: Running an IT organization like a business — like an independent supplier delivering goods and services to its “internal customers.”

But like it or not, sometimes IT is stuck in the role of internal supplier. If you’re among the stuckees, and don’t have a clear path to unsticking yourself, you have little choice but to treat the rest of the enterprise as the market you trade in.

If that’s your situation, it’s worth a bit of your time and attention learning some of the better-known causes of market failure, figuring out whether they might apply to your internal market, and if they do, what you can do to mitigate the effects.

Market Failure Cause #1: Baumol

Since I brought up Dr. Baumol and his disease, it’s worth thinking through how it might apply to IT. It probably doesn’t have much impact on application development, or at least, not yet — most app dev shops have a number of opportunities for improving productivity.

It probably doesn’t apply to IT operations, either: Few have taken full advantage of all opportunities to automate infrastructure management and administration, and that’s before the cloud comes into play.

But business analysts and relationship managers are a different story. A lot of this work is person-to-person, and there’s a limit to how much you can compress conversations without losing the point.

Market Failure Cause #2: Monopolies

Market economies rely on the effects of competition to achieve efficient allocation of goods and services.

But internal IT isn’t supposed to have competitors. In this it’s like modern-day power companies, or perhaps urban cable service providers.

In the world at large, society deals with monopolies by establishing regulatory bodies such as public utilities commissions, which establish rules, regulations, and rates that give consumers roughly the same prices and quality of service they’d get if it was possible to avoid having monopoly providers.

Traditional IT has used much the same approach, setting up IT steering committees to allocate IT services.

It’s possible to avoid this path to unpopularity through the use of charge-backs, also known as transfer pricing. Whoever has budget can use it to buy IT services from the IT service catalog. Which seems much better, except that all this really does is move responsibility for IT resource allocation upstream to the corporate budgeting process, which achieves the near-impossible feat of being even less popular than IT’s governance processes.

IT can de-monopolize itself, though, by encouraging what’s usually called “shadow IT” — information technology implemented without engaging IT’s services at all. This increases the number of information technology providers in the company, creating true competition.

This doesn’t have to become a free-for-all, either, although this characterization comes up just about always when I recommend promoting shadow IT.

What’s needed is what happens in non-monopolized marketplaces. Take, for example, the construction business. There’s plenty of competition, but it isn’t a free-for-all at all. What keeps it all in check is a regulatory apparatus that establishes the rules all builders must adhere to.

IT already has an equivalent regulatory apparatus — its technical architecture function, which establishes the standards to which all production information technology must adhere.

Baumol’s disease and monopolies are only two of many market failures. Email me with your favorites (or, even better, leave a Comment) and next week we’ll pick up where we’re leaving off.