‘Tis the season for bad metrics

Like Tweet Pin it Share Share Email

The world is awash in bad metrics.

Start with a timely example from your favorite daily newspaper or shouting-heads news commentary: “The top 1 percent paid 35 percent of all income taxes” (usually terminated by an exclamation point.)

No, tax policy isn’t relevant to leading and managing an IT organization, but devising and interpreting metrics is essential to the job, so what the heck. Tax policy is, if nothing else, more fun to tear into than IT SLAs. So let’s dig in, starting here: This statistic has no meaning.

None, because there’s no referent. If the top 1 percent earned 100 percent of all income its tax burden is way too low; if it earned 5 percent it’s probably too high. According to the Tax Foundation, the number is 18.7 percent, which might lead you to conclude the top 1 percent is being unfairly burdened by current tax policy.
Perhaps they are. But we’re far from finished, because:

The 18.7 percent number ignores income sheltering. It also ignores the income retained by corporations in which the 1 percent hold large amounts of stock.

Then there’s the question of why we’re only including income tax. Surely, what matters is the total tax burden – federal income tax, state income tax, payroll taxes, sales taxes, property taxes, and, in some states, personal property taxes.

Interestingly enough, comparisons of income to total tax burden are hard to come by.

For that matter, should we be thinking in terms of total income, or should we be thinking in terms of disposable income after an allowance for a person’s basic needs are subtracted out? The top income brackets have a far greater share of disposable income than 18.7 percent no matter where you draw the blurry line that separates necessities from nice-to-haves.

My point in all this isn’t to suggest the top 1 percent, top 0.1 percent, or top anything percent are paying too much or too little in taxes, let alone whether the country’s total tax rates are too high, too low, or are, Goldilocks-like, just right.

My point is to suggest we all need to start crying FOUL! every time a politician or member of the commentariat parades yet another meaningless number in front of us to prove how awful it all is.

These fine folks probably aren’t deliberately trying to deceive us. I suspect it’s worse: I don’t think they know how to devise and interpret useful metrics at all. They’re just passing their ignorance along to the rest of us.

Oh, by the way: This whole conversation is probably the wrong conversation. A better one would start with the questions: (1) What kind of society do we want to have? (2) What roles should government play in providing it? And (3) what’s the best way to pay for it?

We wouldn’t all agree, but at least we’d know what we’re disagreeing about.

Lest you think this has no relevance to IT, take a look at (to choose an easy target) just about any IT benchmarks ever published. Your company’s CEO probably looks at, if nothing else, the ever-popular percent of revenue spent on information technology, so you’d better look at it too.

It’s a dopey metric for so many reasons they’re hard to count, yet its popularity persists. Many reasons? Sure, like:

  • Is a smaller number better or worse? Most CEOs will assume less is better. Spending less means a smaller number for operating expenses for the same amount of revenue, and that’s good, isn’t it?

Sure, except when it means fewer investments in information technology that would increase revenue or reduce other operating expenses.

  • Are the numbers comparable? Different companies have different levels of IT spending outside the IT department. Some companies have outsourced whole business processes, with the relevant IT owned, managed, maintained and enhanced by the BPO vendor – they’re still paying for the IT, only it doesn’t show up on the books.
  • Has your company invested more in information technology in the past? The more IT you’ve built, the more you spend on operations and applications maintenance to keep the joint running. To avoid exceeding the percent-of-revenue benchmark that means you have to spend less on new development.

Just like the tax question, the percent-of-revenue benchmark also leads to the wrong conversation. The right conversation? It starts with very much the same questions as the tax conversation: (1) What kind of business do we want to run? (2) What roles should IT play in providing it? And (3) what’s the best way to pay for it?

Business managers probably call the IT chargeback a tax, too.

Comments (6)

  • I once worked at a large company that boasted that it spent 12% of revenue on R&D. Those of us on the inside knew this was a farce because this number included the huge amount of money and time we spent on maintenance fixing all the things that were not properly researched and developed in the first place.

  • Perfect timing – read your post on one of my pet hates and then saw this quote on a well known news site:

    “Eating seven or more portions of fruit and vegetables every day, rather than the five currently recommended, could significantly cut the risk of death, researchers say.”


  • Hi Bob,

    1. May I pass along the first part of your article to one of my senators, Dianne Feinstein?

    2. As an old building maintenance guy, I’d say that IT expenses work the same way as plant maintenance expenses – they are too technical for someone outside the field to understand, unless they are willing to take the time to study and gain a sufficient understanding of the field such that people in the field tell him that he does understand. Otherwise, take the word of the supervisor, so long as things keep working, workers can do their jobs, and the law is not being broken.

    If this imagined CEO does gain the necessary knowledge, then I would suggest the CEO take a page from Robert Reich and compare the percentage to IT now to the percentages to IT when the organization was doing its best and try to maintain that percentage.

  • 99% of the words in this article are arranged in such a way that is not useful to me in educating my business leadership of the inadequacy and cloudy meaning of most metrics. The target was 80%. Please address….

  • My favorite was the Kansas City Star headline – “Half of all US Cities Below Average!” I don’t remember what they were measuring, but I do agree.

Comments are closed.