All Posts in Leadership

|In: Leadership|Last Updated:

The most disturbing business book of the past ten years has a pretty dull title: Measuring and Managing Performance in Organizations (Robert D. Austin, 1996). It isn’t exactly an easy read either. It’s chock-full of economic and behavioral theory, and bibliographic references to more economic and behavioral theory.

Reading the book is work. Read it anyway. It asks the musical question, does instituting a system of performance metrics ever do more good than harm? Its answer: Yes, but not often. Usually, metrics create organizational dysfunction. The problem isn’t easily fixed, because four separate challenges stand between you and a system of measures that helps your organization succeed.

The simplest challenge is bad formulas. The business metrics landscape abounds with them. You can fix them, but doing so requires both ingenuity and mathematical insight. Most organizations have plenty of the former but only limited amounts of the latter. For example: Often, the right metric requires multivariate statistical analysis (multiple regression analysis or analysis of variance). If your staff lacks expertise in, for example, testing data for stationarity and heteroskedasticity, the right metric is unavailable to you.

And this is the easiest problem to fix.

The second challenge is completeness. It’s critical, because you get what you measure. If the creation of value depends on several factors and you can only measure one or two of them, you’ll still get what you measure.

Take an organization that develops and deploys software. Software value depends on fit to function, performance, absence of defects (bugs), and ease of use. One at a time: (1) Performance is (relatively) easy to measure objectively, so you do, and improve it. (2) Users will help you accurately count bugs, so you and they do, and you fix them. (3) Users will also help you understand fit to function, but not in a reliable, quantitative way (“This software is worthless!” is a typical data point). (4) Ease of use, being a matter of aesthetics, is in the eye of the beholder.

Since no two users will agree on either fit to function or ease of use, you’ll appoint an official owner, and adjust the software to his/her satisfaction. Value to the business improves very little compared to how much you’ve invested to improve it.

And that’s a simple measurement situation, which also serves to illustrate a third metrics challenge: Reliance on proxy measures.

The happiness of the appointed owner is a proxy measure for both fit to function and ease of use — it’s indirect and unreliable. Using proxy measures isn’t a wrong thing. Especially in subjective situations it’s the only option available to you. But every time you use a proxy measure you increase your uncertainty. Think of it as the difference between interviewing a driver about a car’s braking ability and measuring stopping distance with a tape measure.

Which leaves the final and most intractable challenge to making metrics useful for last — the pesky human beings who both collect and report the raw data. They expect to be measured by the results, and they’re right — the numbers “objectively” determine raises, bonuses and promotions. This creates a powerful incentive to shade, spin, distort, game, or simply falsify the Garbage In half of the system. Not being fools, employees at all levels, up to and including the CEO, will do exactly that. Don’t believe me? Review the recent histories of MicroStrategy, WorldCom, Tyco, and Enron.

It’s what gets in the way of the cockpit or dashboard analogy used by many (including yours truly not long ago) to illustrate the value of organizational metrics. As Austin explains:

“[The] analogy would be more accurate if it included a multitude of tiny gremlins controlling wing flaps, fuel flow, and so on of a plane being buffeted by winds and generally struggling against nature, but with the gremlins always controlling information flow back to the cockpit instruments, for fear that the pilot might find gremlin replacements. It would not be surprising if airplanes guided this way occasionally flew into mountainsides when they seemed to be progressing smoothly toward their destinations.”

The situation isn’t hopeless. Instituting useful metrics isn’t impossible, nor is it a worthless endeavor. It is, however, hard work, and calls for considerable finesse.

You’ll also need a culture of honest inquiry, which means relinquishing a cherished assumption on the part of most metrics advocates: That you can use metrics as a tool to manage and motivate the performance of individuals and workgroups.

Doing that lets the gremlins in.

Comments

|In: Business Ethics|Last Updated:

I might owe Tom Peters an apology.

Last week I mentioned a Fast Company article in which Peters said he’d faked his data. According to a follow-up in Business Week shared by reader Ed Kimball, it appears Peters (and invisible co-author Robert Waterman) did nothing of the kind.

Peters reviewed and approved the Fast Company article, but now claims Fast Company’s writer, Alan Webber, invented the quote. Webber politely and obliquely disagreed. So either Tom Peters faked faking his data, or Fast Company faked his faking it for him.

Reliable evidence, and relying on evidence, is vital to making smart decisions in business. In Good to Great, Jim Collins quotes Lyle Everingham, CEO of Kroger during its transition from muddling through to 25 years of outstanding performance: “Once we looked at the facts, there was really no question about what we had to do.” A&P, its lackluster competitor, pretended. It created a new store concept, The Golden Key, to test ideas. Its executives didn’t like what its evidence told them so they closed it.

Kroger had a culture of honest inquiry — where executives, managers and employees do their best to use trustworthy evidence to drive decision-making. Creating a culture like this takes work, persistence, and sometimes political dexterity. Here are some specific measures you can take to foster a culture of honest inquiry in your workplace:

  • It starts with wanting to know what’s really going on out there. Enron and WorldCom happened, in part, because their executives were so busy trying to make their companies look good that they obscured what was really going on to themselves. Your dashboards, financial reports, and other forms of organizational listening are to make you smarter. If not, don’t bother.
  • Confidence comes from doubt. Certainty, in contrast, comes from arrogance. If an employee is confident and can explain why, wonderful. If that employee’s certainty pre-empts everyone else’s ability to make their case, the employee is on the wrong side of things.
  • Start every decision by creating a decision process. You don’t have to be in charge to encourage this habit. Just ask the question, “How will we make this decision?” That changes the discussion away from who wins to how to create confidence in the outcome. The results: A better decision, a stronger consensus, and a few more employees who see the benefit of honest inquiry.
  • Don’t create disincentives for honesty. If you ask for honest data, and use it to “hold people accountable,” you won’t get honest data. Why would you? The superior alternative is to employ people who take responsibility without external enforcement, and to create incentives for that kind of behavior. This works much better, and takes less effort.
  • The “view from 50,000 feet” is for illustration, not persuasion. A high-level strategic view is essential for focusing the efforts of the organization. High-level logic, in contrast, is oxymoronic: Detailed evidence and analysis is what determines whether the high-level view makes sense, or just looks good in the PowerPoint.
  • Evidence too far removed from the original source is suspect. Don’t trust summaries of summaries of summaries, especially if they tell you what you want to hear. Even with the best of intentions the game of telephone is in play. And many of those on the side of intellectual relativism don’t have the best of intentions.
  • Be skeptical of those with a financial stake in the decision. But don’t ignore them. A conflict of interest suggests bias, but doesn’t automatically make someone wrong. Be wary and dig into their evidence, especially if their evidence is a summary of a summary of a summary. But if you demonstrate to your satisfaction that they’ve cooked the evidence … go ahead and ignore them from now on. They’ve earned it.
  • Beware of anecdotes and metaphors. They’re useful … for illustrating a point, or for demonstrating that something is possible. For anything else you need statistically valid evidence. Yes, Disraeli said there are three types of lie. He miscounted — argument by anecdote is far more pernicious than argument by statistics, and argument by metaphor is even worse. Yes — you do have to understand statistics well enough to evaluate the evidence. Sorry. That’s part of your toolkit.

If you work in a business culture infected with intellectual relativism, it will take time and patience to build the habit of rationalism. You won’t do so by preaching and lecturing about the general principle.

The way to build a culture of honest inquiry is one decision at a time.

Comments