You can’t make this stuff up. Ray Todd Stevens tells us:

I am working on a recovery project for a company that outsourced the coding for a major application on which the company depends. They did this to an offshore company. Someone at some point realized that the code had no comments, and so insisted on the code being commented. So comments appeared and nobody apparently looked at them other than to ensure that they are there. The project was accepted and the offshore vendor was paid.

Now we are dealing with some bugs, and have found that yes there are comments. The ones that are in English refer to process and procedure documents that were apparently internal to the vendor. My client certainly does not have them. The rest of the comments are not in English. The variables are named after Indian Gods. So far the comments that we have determined the language for are in eight different languages. Most of these refer to things that we can’t even figure out, and don’t appear to relate to the code they are tied to.

The external documentation is about as bad although it is in English.

I guess you get what you pay for.

If you’re looking for anecdotal ammunition to argue against offshore outsourcing, look no further. But that would be wrong. Once more, with feeling: Anecdotes illustrate points, they don’t prove them. The question is, what point does this anecdote illustrate? Surely not that all offshore outsourcers provide comments only under duress, and then in inaccessible languages. An anecdote this tasty has to illustrate something more profound than that.

For example, this: Alignment of goals is more powerful and reliable than specification of responsibilities.

This point plays itself out in all sorts of situations. When you delegate tasks instead of goals, for example, you have to inspect the results with far greater care, because the assignee’s work is finished when it meets the written specifications. Whether it does anything useful is Someone Else’s Problem — not the case when you delegate the goal.

When you contract with a vendor, relying on specifications almost guarantees you’ll pay more for less. Never mind offshore IT services. It’s just as true when dealing with ad agencies: Plenty develop campaigns designed to win them awards and new clients and not to make your cash register go ka-ching.

It’s the same when you don’t rely on a vendor. If you want to make sure IT delivers software that does nothing useful for the business, define your Systems Development Lifecycle so that business analysts work with business departments to determine their requirements and create written specifications that programmers use to design and build applications. If you’d prefer to accomplish something, have business analysts facilitate conversations between business departments and developers, using written specifications to remind everyone of what they decided in the face-to-face conversation. The former is reliance on specifications; the latter gives everyone the opportunity to align their goals.

Then there’s the ever-popular dress code — the subject that wouldn’t die. Companies that try to define and enforce proper dress mostly achieve resentment and a cadre of jailhouse lawyers adept at malicious obedience — at adhering to the letter, but not the spirit of the code. Companies that encourage employees to use their good judgment, dressing professionally and appropriately for their work seem to get better results.

Alignment of goals is more powerful and reliable than specification of responsibilities for a reason well-known to computer scientists: When in doubt, add more processors. When you rely on specifications you rely solely on your own brain to figure things out. When you rely on alignment of goals, you get to take advantage of everyone’s brainpower. More processors.

Which leaves just one small challenge to be addressed: How, exactly, do you align everyone’s goals?

The short answer is, that’s what leadership is about. A slightly longer answer is that it isn’t all that hard, because most people, most of the time, prefer things that way — they want to belong to a group that has a purpose they can buy into and contribute to. For some reason, likely related to a personal need for control, many managers give every appearance of working very hard to prevent this very common and natural human tendency from operating.

There are few situations in IT, or anywhere else in a modern business, where specifications are enough. Usually, good judgment matters. The only way employees can apply good judgment is if their goals and yours line up.

Making that happen is up to you, not them.

The most disturbing business book of the past ten years has a pretty dull title: Measuring and Managing Performance in Organizations (Robert D. Austin, 1996). It isn’t exactly an easy read either. It’s chock-full of economic and behavioral theory, and bibliographic references to more economic and behavioral theory.

Reading the book is work. Read it anyway. It asks the musical question, does instituting a system of performance metrics ever do more good than harm? Its answer: Yes, but not often. Usually, metrics create organizational dysfunction. The problem isn’t easily fixed, because four separate challenges stand between you and a system of measures that helps your organization succeed.

The simplest challenge is bad formulas. The business metrics landscape abounds with them. You can fix them, but doing so requires both ingenuity and mathematical insight. Most organizations have plenty of the former but only limited amounts of the latter. For example: Often, the right metric requires multivariate statistical analysis (multiple regression analysis or analysis of variance). If your staff lacks expertise in, for example, testing data for stationarity and heteroskedasticity, the right metric is unavailable to you.

And this is the easiest problem to fix.

The second challenge is completeness. It’s critical, because you get what you measure. If the creation of value depends on several factors and you can only measure one or two of them, you’ll still get what you measure.

Take an organization that develops and deploys software. Software value depends on fit to function, performance, absence of defects (bugs), and ease of use. One at a time: (1) Performance is (relatively) easy to measure objectively, so you do, and improve it. (2) Users will help you accurately count bugs, so you and they do, and you fix them. (3) Users will also help you understand fit to function, but not in a reliable, quantitative way (“This software is worthless!” is a typical data point). (4) Ease of use, being a matter of aesthetics, is in the eye of the beholder.

Since no two users will agree on either fit to function or ease of use, you’ll appoint an official owner, and adjust the software to his/her satisfaction. Value to the business improves very little compared to how much you’ve invested to improve it.

And that’s a simple measurement situation, which also serves to illustrate a third metrics challenge: Reliance on proxy measures.

The happiness of the appointed owner is a proxy measure for both fit to function and ease of use — it’s indirect and unreliable. Using proxy measures isn’t a wrong thing. Especially in subjective situations it’s the only option available to you. But every time you use a proxy measure you increase your uncertainty. Think of it as the difference between interviewing a driver about a car’s braking ability and measuring stopping distance with a tape measure.

Which leaves the final and most intractable challenge to making metrics useful for last — the pesky human beings who both collect and report the raw data. They expect to be measured by the results, and they’re right — the numbers “objectively” determine raises, bonuses and promotions. This creates a powerful incentive to shade, spin, distort, game, or simply falsify the Garbage In half of the system. Not being fools, employees at all levels, up to and including the CEO, will do exactly that. Don’t believe me? Review the recent histories of MicroStrategy, WorldCom, Tyco, and Enron.

It’s what gets in the way of the cockpit or dashboard analogy used by many (including yours truly not long ago) to illustrate the value of organizational metrics. As Austin explains:

“[The] analogy would be more accurate if it included a multitude of tiny gremlins controlling wing flaps, fuel flow, and so on of a plane being buffeted by winds and generally struggling against nature, but with the gremlins always controlling information flow back to the cockpit instruments, for fear that the pilot might find gremlin replacements. It would not be surprising if airplanes guided this way occasionally flew into mountainsides when they seemed to be progressing smoothly toward their destinations.”

The situation isn’t hopeless. Instituting useful metrics isn’t impossible, nor is it a worthless endeavor. It is, however, hard work, and calls for considerable finesse.

You’ll also need a culture of honest inquiry, which means relinquishing a cherished assumption on the part of most metrics advocates: That you can use metrics as a tool to manage and motivate the performance of individuals and workgroups.

Doing that lets the gremlins in.