All Posts in Metrics

|In: Metrics|Last Updated:

“Could you please stop saying there’s no such thing as an IT project?” a reader politely asked. “When I have switches/routers/servers that are out of support from their vendors and need to be replaced with no business changes I have to call these IT projects.”

I get this a lot, and understand why folks on the IT infrastructure side of things might find the phrase irritating.

And I agree that projects related to IT infrastructure, properly executed, result in no visible business change.

But (you did know “but” was hanging in the air, didn’t you?) … but, these projects actually do result in significant business change.

It’s risk prevention. These projects reduce the likelihood of bad things happening to the business — bad things like not being able to license and run software that’s essential to operating the business; to purchase and use hardware that’s compatible with strategic applications, and so on.

It’s important for business executives to recognize this category of business change project, if for no other reason that none of us want a recurrence of what happened to IT’s reputation following our successful prevention of Y2K fiascoes. Remember? Everyone outside IT decided nothing important or interesting had happened, and that’s if they didn’t conclude we were just making the whole thing up.

Successful prevention is, we discovered, indistinguishable from the absence of risk. So we need to put a spotlight on the business risks we’re preventing so everyone recognizes our successes when we have them.

Not to mention the need for everyone to be willing to fund them.

Which leads to a quick segue to IT architecture, which, depending on the exact framework and source, divides the IT stack into information systems architecture, subdivided into applications and data; and technology architecture, subdivided into platforms, infrastructure, and facilities.

Switches and routers, along with everything else related to networking are infrastructure. With the exception of performance engineering, infrastructure changes ought to be invisible to everyone other than the IT infrastructure team responsible for their care and feeding.

Servers, though, belong to the platform sub-layer, along with operating systems, virtualization technology, development environments, database management systems … all of the stuff needed to build, integrate, and run the applications that are so highly visible to the rest of the business.

The teams responsible for platform updates know from painful experience that while in theory layered architectures insulate business users from platform changes, in fact it often turns out that:

  • Code written for one version of a development environment won’t run in the new version.
  • The vendors of licensed COTS applications haven’t finished adapting their software to make it compatible with the latest OS or DBMS version.
  • Especially in the case of cloud migrations, which frequently lead to platform, infrastructure, and facilities changes, performance engineering becomes a major challenge. And as everyone who has ever worked in IT infrastructure management knows, poor application performance is terribly, terribly visible to the business.

Et cetera.

Not that these platform update challenges are always problems. They can also be opportunities, for clearing out the applications underbrush. Part of the protocol for platform updates is making sure all application “owners” (really, stewards) aren’t just informed of the change but are actively involved in the regression testing and remediation needed to make sure the platform change doesn’t break anything.

The opportunity: If nobody as the steward for a particular application, retiring it shouldn’t be a problem.

On a related topic, regular readers will recall the only IT infrastructure metric that matters is the Invisibility Index. Its logic: Nobody notices the IT infrastructure unless and until something goes wrong.

Invisibility = success. Being noticed = failure.

Something else regular readers will recognize is that Total Cost of Ownership (TCO) is a dreadful metric, violating at least three of the 7 C’s of good metrics. TCO isn’t consistent, complete, or on a continuum: It doesn’t always go one way when things improve and the other when they deteriorate; it measures costs but not benefits; and it has no defined scale, so there’s no way to determine whether a given product’s TCO is good or bad.

But perhaps we should introduce a related metric. Call it TCI — the Total Cost of Invisibility. It’s how much of its operating budget a business needs to devote so those responsible for the IT infrastructure can continue to keep it invisible.

They’ll keep it invisible by running what aren’t IT projects. But are quite technical nonetheless.

Comments

|In: Metrics|Last Updated:

Blame Lord Kelvin, who once said, “When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”

It’s a philosophy that works well in physics and engineering — fields where what matters can, for the most part, be unambiguously observed, counted, or otherwise be quantified.

The further you get from physics and engineering, the more you might wish Lord Kelvin had added, “Of course, just because you can attach a number to something, that doesn’t mean you understand anything about it.”

So if you accidentally touch a bare copper wire, it’s fair to consider how loud you yell “Ouch!” to be an inferior metric to how many volts and amperes you were exposed to.

But on the other side of the metrics divide, imagine you’re researching headaches and want to rank them in order of decreasing agony.

You think cluster headaches are the worst (they get my vote), followed by migraines, sinus, tension, and faking it to get sympathy. But really, how can you tell?

There’s the well-known pain scale. It does a pretty good job of assigning a number by assessing how debilitating the pain is.

But debilitation is an index, not a direct measure. It passes most of the seven Cs, but not entirely. In particular its calibration is imperfect at best — some people seem to tolerate the same pain better than others, although there’s really no way of knowing whether they actually tolerate pain better or whether the same stimulus doesn’t result in as painful an experience as someone else might feel.

Which insights we need to pivot into something that has to do with helping you run your part of the enterprise better.

Consider it done.

Start with the difference between leadership and management. If people report to you, you lead (or should). If you’re responsible for producing results, you manage. With infrequent exceptions, leaders are also managers and vice versa.

Metrics are natural tools for managing. What they do for managers is help them assess whether the results they’re responsible for producing are what they’re supposed to be. The results in question are about the process (or practice) characteristics that matter:

> Fixed cost — the cost of turning the lights on before any work gets done.

> Incremental cost — the cost of processing one more item.

> Cycle time — how much time elapses processing one item from start to finish.

> Throughput — how much work the function churns out in a unit of time … its capacity, in other words.

> Quality — adherence to specifications and the absence of defects in work products.

> Excellence — flexibility, the ability to tailor to individual needs, and to deliver high-value product characteristics.

When it comes to managing whatever process or practice it is you manage, pick the three most important of these six dimensions of potential optimization, establish metrics and measurement systems to report them, and use the results to (1) make sure things are running like they’re supposed to; (2) let you know if you’re improving the situation or not; and (3) let employees know if they’re improving the situation or not.

You only get to pick three because except when a process is a mess — at which point you can probably improve all six optimization dimensions — improvements result in trade-offs. For example, if you want to improve quality, one popular tactic is simplifying process outputs and disallowing tailoring and customization. More quality means less excellence and vice versa.

If it turns out you aren’t getting what you’re supposed to get, that means your process has bottlenecks. You’ll want to establish some temporary metrics to keep track of the bottlenecks until you’ve fixed them.

I say temporary because once you’ve cleared out one bottleneck you’ll move on to clearing out the next one. Letting metrics accumulate can be more confusing than illuminating. Also, as pointed out last week, metrics are expensive. Letting them accumulate means increasingly complex reporting systems that are costly to maintain and keep current.

Given the value metrics provide for effective management, lots of organizations try to use them as a leadership tool as well. The result is the dreaded employee satisfaction survey.

In Leading IT I established eight tasks of leadership: Setting direction, delegation, decision-making, staffing, motivation, team dynamics, establishing culture, and communicating. A system of leadership metrics should assess how well these are accomplished by a company’s collective leadership.

Which gets us to this week’s KJR Challenge: Define metrics for these that can survive the seven Cs.

Comments