All Posts in Policies and Procedures

|In: Industry Commentary|Last Updated:

In the beginning was Deming.

Deming introduced measurement to industry in the 1950s. A statistician, he relied on sophisticated multivariate techniques. He encouraged those who didn’t understand sampling theory, analysis of variance and multiple regression to stay away and let the experts handle the data collection and math.

American industry ignored Deming and his theories — I can’t help but suspect America’s enduring cultural distrust of the intellectual — but Japan did not, turning itself from a defeated military power to a fierce economic superpower in the process.

As a culture we are nothing if not adaptable, and so “metrics” happened in (to?) America. Sadly, we aren’t so adaptable that we were willing to actually learn, for example, statistics.

Which might explain why most of the metrics stories I hear are horror stories, not successes.

I’ll be the first to admit to flaws in my sample. I’m sure it is non-stationary, biased, suffers from heteroskedasticity and perhaps the heartbreak of psoriasis as well. I’m nonetheless confident in this conclusion: Call centers are the most mis-measured function in business, and IT Service Desks are the most mis-measured type of call center. Which brings us to this week’s anonymously provided tale of woe:

At the Help Desk where I currently work, they like to measure First Call Resolve (FCR).

What I am finding out from longtime tech support people is they like to game the metrics. Here’s how it works:

First, the system defaults every call to Resolved whether it really is or not.

If a user never calls back to say “That didn’t fix it”, you get a freebie. If, on the other hand, the user does call back to say, “Yep, that fixed it,” and the tech on the line documents the call in the system notes, you just lost an FCR. Why? Because you weren’t the LAST person to talk to the user. Yep, the last person to talk to the user on a call gets the FCR. It’s as if the relief pitcher who comes in with two outs in the ninth, a five-run lead and the bases empty gets credit for the win.

So potentially someone else could make you look bad, just by creating the last entry in a call log. And after five days, a ticket goes from Resolved to Closed and there’s nothing you can do about it.

You always open a ticket on a new call, find out what the issue is, then decide if it’s your queue or not. If not, you transfer it. However, the queue you transfer to will never get an FCR on that call even if they solve it immediately. Why? Two people have talked to the user, that’s why.

Some tech guys get around that by opening up their own ticket and documenting it for an FCR, leaving your ticket out to hang. And that can ding YOU just as badly since it’s been presumed you abandoned it. If you get a call where no one handles it but they actually need to call outside of tech support to, say, Financial Ops, you document it but those calls are not counted as FCR. They are deducted from your score.

Strangely enough, some of us were talking about this the other day — about how to increase FCR legitimately. Some of them had researched suspicious FCR standings (which are posted for individuals) and found that some of the lumps were posting gratuitous FCR calls (in other words, fake calls). You could tell by the shortness of the ticket and the vague incorrect answers that were in the notes.

The best way to find the lumps? Not the FCR metrics — they’re worthless. Instead, just ask the other techs. Most tech people are fairly honest in appraising their fellow co-workers. It’s not that we don’t like them. It’s idiotic management playing games with our stats.

So, like the plaque says, “Be careful what you measure. You’ll get it.”

The culprit here is clear: Simplemindedness.

A Help Desk is neither simple nor solitary. It’s one part of a system that should: Work on what’s most important first; resolve incidents quickly; and prevent recurrences.

Assessing the system’s outputs isn’t all that challenging. Assessing the engineers and technicians who create the outputs is tougher.

They have to resolve what they can, properly refer what they can’t, accurately document their work, and escalate underlying systemic defects. That requires: Intelligence, judgment, technical skill, diligence, and collaboration. It’s a multivariate situation.

Just ask Deming.

Comments

|In: Leadership|Last Updated:

Asked if the study of creation could provide any insights as to the nature of its Creator, the great biologist J. B. S. Haldane replied that clearly the Creator has ” … an inordinate fondness for beetles.”

Coleopterans account for about a quarter of all animal species on this earth. Mammals, in contrast, contribute about a quarter of a percent of all species. Don’t feel bad. By other measures I’m sure we human beings are more important.

Humans dominate all other species (except, perhaps, for some viruses and bacteria, along with ants, spiders and the aforementioned beetles) because of our ability to think: To make tools, to plan, to transfer skills and knowledge to our progeny.

That’s how humans dominate other species. Individual humans dominate other humans through their social skills. These two facts explain most of human history: We win through intelligence. I win through what Daniel Goleman calls “emotional intelligence.”

This is why most businesses fail to learn from their successes, as explained in detail in the last several editions of Keep the Joint Running: Learning from success is good for the company but not necessarily for those who run the company.

Emotional intelligence is a vital quality for effective leaders. The problem arises when they decide to value emotional intelligence more than tangible skills, knowledge and logical decision-making in the organizations they lead. It’s how organizations start down the sad path to what my business partner calls “mediocracy.”

It needn’t be so. As evidence I submit this tale from a regular KJR correspondent:

I recently finished a six month job rotation as manager of our Service Desk (formerly “Helpless Desk” according to our users).

The prior manager measured performance on a strict count of tickets handled, weighted by priority. Since we can’t reward by dollars (union shop), we use perks like window cubes, preferred schedules, and first choice on vacations as incentives.

The top performer for the 13 quarters prior to my arrival was an individual who I really didn’t see doing anything. The quarter ended and he was again the top man. I decided to find his secret to success in an effort to raise everyone else’s game. So I started analyzing his tickets.

His method was simple. Every single call he took was labeled Critical. If he took a ticket originally handled by someone else, he upgraded it. If he got repeat calls from the same user with the same problem, each was a new ticket. He closed out every ticket within two hours, regardless of whether the problem was handled or not. We measured volume, so he delivered volume.

However, when he trained someone new, he always preached completion, follow-up, and thorough documentation — in short, how the job really should be done. He gamed the metric both ways (boost my numbers, lower your numbers). And we rewarded him for it.

I changed the rules and the measures. Those answering the phones started with the question “Is this a problem we have worked on before?” and reopened the old ticket instead of writing a new one. They wrote all tickets and graded their priority. They assigned the tickets and those handling them did not have authority to change the priority. I measured who got things fixed “once and for all” rather than incident volume.

Lo and behold, the top performer fell immediately into the bottom 10%. In the next quarterly survey of our effectiveness, users rated service 20% higher than ever previously. I was offered the position permanently (and declined it).

Our “top” performer left due to “absenteeism problems.”

I left my successor a plaque that read “Be careful what you measure. You’ll get it.” (It’s from one of your columns on metrics — credit where it’s due.)

My successor went even further. He rewards the people who identify patterns of problems and solve them, for individual recognition. He did away with the perk system in preference to a team system. His metrics are number of tickets, number of reopens, time from receipt of call to start of work, and customer satisfaction.

He also tracks the number of calls received to prevent gaming the first number by not writing tickets. The desk is way up in satisfaction, and he definitely is on the right track.

Oh, and he still uses the manual written by the former top performer to train people.

Comments