All Posts in Organizational Effectiveness

|In: Industry Commentary|Last Updated:

One of the most enjoyable experiences of my career was an upgrade.

It wasn’t the upgrade itself that was so much fun. It was the financial justification. Usually, getting capital approved for IS projects is about as much fun as being one of Sigfried and Roy’s cats – you get to jump through a bunch of flaming hoops while someone you’d enjoy clawing to death cracks a whip in your general direction.

This time, since we needed to upgrade our interactive voice response (IVR) system, we calculated the impact of unplugging the existing one. Since it was a 24-line system that was saturated during peak hours, this was simple: We just toted up the cost of adding another twenty-four customer service representatives to the call center and compared it to our original sub-$100,000 investment. Upgrade approved.

Occasionally, it’s possible to calculate the return on information technology through a clear and unambiguous calculation like this one. Usually, as pointed out in the last two columns, the value we provide is harder to quantify. Except for the rare process we can fully automate (an IVR is an example), the value we deliver comes in the form of capabilities used by the rest of the organization … capabilities that let the company achieve its strategic goals or that enable more effective marketing efforts, core processes, internal and external communications, and individual employees.

Capabilities are enablers: Each is necessary; none by itself is sufficient. How do you measure the value of a capability? How much does the fuel pump contribute to winning the Indianapolis 500?

To begin, you need to understand the value of achieving the business goal itself. Take a core business process. Process redesigns reduce unit costs and cycle times while improving the quality of whatever product the process creates.

To implement the new process a company probably needs new technology, new skills, a shift in culture, and perhaps a reorganization (even if it doesn’t, it will probably reorganize anyway because that’s what you do when you’re a big company undergoing change). What contribution does IT make to the tangible and intangible benefits?

The new process is completely different from the old one, so you can’t simply measure how much technology lowered costs, sped things up and improved quality. The best you can do is to determine the optimal process design possible without any technology changes … a good idea when redesigning a process in any event … and compare it to the one you plan to implement. You can plausibly define the difference as the value created by IT. (Caveat: This method isn’t mathematically rigorous; skeptics can challenge it on several grounds. Since nothing better is available, though, call it an approximation and be happy.)

While in principle, you can use this technique to measure the value of your legacy systems, too – by determining how the business would run if you unplugged them – in practice there’s little point. They deliver value – they’re in use, after all – so just measure unit costs and overhead, driving them as low as you can.

Another pothole in the road to IT measurement: Not every business goal is defined quantitatively. When a company redefines its strategy, for example, the usual reason is that the old one has run out of steam. The value of change is survival, not a numeric target. If the company needs new information technology to achieve its strategic goals, what value does it contribute? There’s no answer. Your contribution, while important, can’t be quantified. Don’t try.

And then there’s the hardest job of all: Measuring the value of a capability you deliver that’s unused or under-used. Lots of companies have thrown out gigabytes of customer information over the past decade, for example, because they didn’t have an immediate use for it. It had no value then. Now, with new strategies based on having extensive knowledge of customers, it would be a gold mine … but it’s gone, because the future value of the capability wasn’t understood at the time.

That’s just one reason measurement has its limits, but the ability to envision the future does not.

Comments

|In: Office Politics|Last Updated:

Here’s an alarming statistic that I read recently: 81 percent of everyone surveyed thinks their IS organization is average or below average. If “below average” translates to “below the mean,” only 20 percent of us are in the top 50 percent.

Since human perception is a pretty dull scalpel, “average or below average” may not be quite as precisely defined as “worse than or equal to exactly half the total number.” Let’s try a different interpretation. Figure anything within one standard deviation of the mean counts as average. In round numbers about two-thirds of any sample falls inside one standard deviation. The remaining third splits in half, so one sixth of any sample is above average. The remainder – five sixths, or just over 83 percent, are average or below.

Mystery solved! The 81 percent who figure their IS departments are average or worse are almost exactly the number who ought to think so according to the inviolable laws of statistical sampling.

The authors of the paper reporting this statistic made no further comment, so we don’t know if its absurdity escaped them or not. That three college professors who specialize in business metrics resorted to this kind of number, though, speaks poorly of the state of the art in IS measurement. I certainly didn’t do anything like that in my new book, Bob Lewis’s IS Survival Guide from MacMillan Computer Publishing (nor would I ever stoop to shamelessly plugging it in this column).

As we found last week, we have plenty of measures to choose from, all internal ones that tell us how good our processes are compared to internal baselines or external benchmarks. What we lack are external measures that assess the value we create for the enterprise. The measures we have tell us, to borrow a phrase, whether we’re doing things right, but not whether we’re doing the right things.

We do have one external measure at our disposal. The cost of technology is depressingly easy to measure, and our detractors gleefully proclaim it during budget season. But the value we create? That’s a lot tougher.

The purpose of any measurement system is improvement (ignoring its important use in political self-defense). The point of calculating the value we deliver to the enterprise is helping us increase it. How do we create useful measures of value? It’s tough. At the highest level, the formula for calculating value is Bang per Buck. We know how to measure the buck part, which leaves the bang as the part we need to measure. Start by listing the major categories of benefit we provide:

  • Capabilities needed so the company can achieve its strategic goals.
  • Capabilities needed for effective marketing efforts.
  • Fully automated (and therefore high-efficiency) processes.
  • Capabilities needed by redesigned processes.
  • Capabilities for improving communications with customers and suppliers.
  • Capabilities for improving internal communications.
  • Capabilities that allow individual employees to be more effective in their jobs.

See a trend? Except for the rare situation that allows for complete process automation, the value we deliver is capabilities. They’re enablers – necessary but not sufficient conditions for success. To measure the value we deliver, we need to understand how to measure the value of a capability when that capability may or may not be used effectively.

How will we go about that? In principle, we need to list every contributor to success in each of these categories, then assign a weighting factor to each of them that reflects its relative importance or contribution.

Great theory. Can we turn it into practice?

Oh, gee, we’re about out of space. Too bad … you’ll have to tune in next week to read the next installment.

Comments