Here are some simple rules of thumb to use when interpreting any published statistic:

• If it appears in a refereed scientific journal, you can rely on the data collection and computation — the statistical significance.

• If the results are confirmed and accepted by the scientific community you can also rely on the researcher’s interpretation — the importance.

• If the statistic appears anywhere in the business press, you can apply the principle of cui bono to determine who paid for the study and decided in advance what the results should look like.

Business has become obsessed by metrics, which would be okay if most business managers (1) were good at mathematics; (2) had the time to think carefully; and (3) started with goals instead of starting with the desire to end up with a number. And so it came to pass that Computerworld published an article recently titled, “Proving SOA Worth Is a Big Challenge for IT: Tools emerging to manage, measure benefits of the complex architecture.”

Early on, the article says that managing SOA is more difficult than the equivalent client/server or Internet projects (the comparison with object-oriented techniques was, bafflingly, omitted). Does it strike anyone else as peculiar that anything built on a core technology called the “Simple Object Access Protocol” (SOAP) should be described as complex and difficult? Maybe the “Simple” in SOAP refers to the objects, not the protocol — SOAP, in this interpretation, would only be appropriate when accessing simple objects.

But I digress. About those tools for assessing the value of the technology: One anonymous technology executive cited in the article is planning to use technology from AmberPoint to “… prove the value of the 100-plus Web services it has in place today.”

I sure hope his bonus doesn’t depend on it, because it isn’t going to happen. You can deploy monitoring tools to tabulate the number of times a Web service is invoked, and workflow tools to detect and register them, but unless you equate activity and value, that’s as far as you’re going to get. And anyway, his description was more about increasing the level of service re-use than about proving value.

Good plan: Increase re-use first, report the value of re-use second.

You have to feel sorry for the guy. He explained his situation like this: “… only 20% of the services were being registered by the company’s 1,500 developers … At the same time, even fewer consumers were logging information about how they would be using the services.”

Maybe this all makes sense. Maybe it’s okay that developers don’t follow documentation standards, or that the documentation standards don’t require registration of Web services that go into production. Maybe it makes sense to try to increase service re-use through technology instead of methodology.

And maybe it really is tough to document SOA’s value.

Or maybe I’m about to save you a lot of pain and effort. Here’s how to go about it:

Step 1: Estimate the cost of the next ten projects in the queue using your current architecture.

Step 2: Estimate the cost of the same projects using SOA.

Step 3: Subtract the results of Step 2 from the results of Step 1.

Step 4: Compare to the up-front cost of implementing SOA.

Enough sarcasm, fun as it is. After all, the poor souls cited in this article were dealing with a tough political challenge — justifying investments in core technologies about which business executives neither know anything nor care to.

When it comes to IT, most business executives care about only four parameters: Cost, delivered value, successful project completion, and system uptime. If they’re really sophisticated, they’ll count system slowdowns against you when assessing uptime.

SOA won’t improve uptime. That leaves some combination of decreased cost, increased value, and improved successful project completion. If that’s what you promise, you’d better “encourage” your developers to register the services they program. Oh … and require use of methodologies that enforce service re-use.

Here’s advice that’s even more important: Don’t explain, under any circumstances, that SOA is more complex than the technologies it replaces. That might lead to a hard-to-answer question — how is it that a more-complex technology will let you deliver results more quickly.

I’m sure there’s an answer. You just won’t be allowed to provide it, because the question will be purely rhetorical.