Dear Car Companies,
I won’t be filling out your satisfaction surveys today.
It isn’t that I don’t want you to know whether or not I’m satisfied, and why. Car Company #1, I’d love to explain that your service department did a fine job taking care of my car, and even washed and waxed it while I waited, and that I didn’t mind all that much having had to wait an extra few minutes because the guy who was supposed to tell me my car was ready got stuck on a phone call.
Car Company #2, I’d be happy to explain that your salesman did his best, but needed more training on your new models … and that the training you did give him about making sure we knew how to handle the car’s nifty electronics resulted in his taking more of our time than we really would have liked.
But I can’t. Your sales representative (CC2) and service representative (CC1) explained the rules to me clearly: Either I give them a perfect score, or you give them a failing grade.
Why would I participate in a sham like this?
Look, there are four “metrics fallacies” — four ways using metrics can make a business worse. You must employ an army of analysts. Haven’t any of them explained these to you?
I usually charge good money for this, but just this once I’ll explain them to you for nothing, if you promise to pay attention. The fallacies:
- Measuring things wrong.
- Measuring the wrong things, whether you measure them right or wrong.
- Failing to measure something important.
- Extending your metrics to individual employees.
You botched #4. But then, a lot of companies botch #4 because a lot of business executives seem to assume that if something goes wrong, it must be someone’s fault.
I guess it’s good they’re measuring all cases, not just the problem ones, because that implies they’re also assuming that if something goes well, someone must deserve the credit.
But they botch it because (I guess) they think their employees are so dim they can’t figure out how to game the metrics to their advantage … like, for example, letting customers know that anything less than a perfect score will land them in a world of hurt.
Here’s a hint: If they are that dim, you’re hiring dim employees, which is a seriously bad idea, especially for the employees you’re putting in front of your customers.
Don’t get me wrong. It isn’t that I think you shouldn’t pay attention to how your employees are doing. Quite the opposite.
The employees you decide to hire — how you choose them, how you train them, how you do your best to keep them, motivate them, and promote the best of them — they’re the single most important determinant of your success. I’m confident of this because I’ve watched outstanding employees succeed in spite of bad processes, substandard tools, and execrable managers, just as I’ve watched disgruntled employees get mediocre results in spite of having the best process designs and tools at their disposal (I didn’t add “great managers” because if they had great managers they wouldn’t have been disgruntled).
What you have to understand is that metrics don’t report root causes. They report symptoms. Unless, that is, you have a predefined list of potential root causes and monitor them all.
But that isn’t what your customer satisfaction survey is doing. The poor schmuck in the service department you’ve asked me to evaluate didn’t do anything wrong. He was stuck, having to choose which of two customers he had to dissatisfy — the one on the phone or me. Why would I give you any ammunition to shoot him with, when the problem, assuming this counts as a problem, was that you had the same person answering the phone and dealing with in-person customers?
And why would I give you ammunition to shoot the salesman with, when the problem was with your sales training program?
Tell you what. Why don’t you send me a new survey? This one would assess my satisfaction with your customer satisfaction assessment process. I’d be happy to fill it out. You could use the results as the ammunition you need to shoot yourself.
Okay, that was mean. It’s just that I’ve written books about this, I’ve given speeches about this, and (now pay attention — this is important) I consult about this, which means that if you had been paying attention, you wouldn’t have made this mistake and my bank balance would be higher.
It’s an outcome that’s known in some circles as a “win/win.”
Sincerely,
Robert Lewis
President, IT Catalysts, Inc.
Sounds like my Honda dealer where they even put posters up telling their customers to tell the follow up survey phone caller that they had an excellent experience.
The worse part is the guilt trip where the employees tell you how import it is that you grade them as excellent.
It’s no wonder I try to void the dealer service department at all costs.
Bob, excellent article as usual. I’ve wrestled with metrics for years, persuading higher-ups about the right and wrong ways. Or failing to do so and watching the predictable results. I instituted a very simple, effective 3-question survey emailed to every customer when a help desk ticket is closed: Did we do what you asked effectively, did we do it in time, and were we nice about it? There is also an optional comment field. In return for making it quick and simple, we get around a 45% return rate. It measures the highest-level symptoms, and if a bad trend occurs, we use manual analysis to find out what’s causing it.
Even so, and in spite of a very prominent message that their response helps our whole organization do better and will not be used to blame the technician, some customers report declining to complete the survey because they don’t want to get someone in trouble, and worse, they don’t want their tech mad at them.
I actually filled out a customer satisfaction survey when I bought my latest car. Neither the salesman nor the sales manager had said anything about giving them all 10’s, and on the survey itself it asked “Did any employee attempt to influence how you complete this survey?” I answered honestly and hopefully helped them to learn a little bit about how they might improve.
Bob,
I completely agree. I have complained to companies whose employees say that anything less than a top score causes them harm.
You should say, however, “anything less than a perfect score”, not “anything better than a perfect score” in the paragraph before the hint.
The worst part about managing metrics is those who work the system get rewarded. And even promoted.
Bob, some wise person, probably a consultant, said (many times, more than once):
“Some folks, if they don’t know, you can’t tell them.”
Place in that category those who design surveys that must be scored 100% or the employee is bashed.
Don’t fuss over that which you cannot change . . . unless you can obtain a consulting gig. And if they are keen enough to hire you, they aren’t doing dumb surveys.
John Blair
“a lot of business executives seem to assume that if something goes wrong, it must be someone’s fault.”
This is so fundamental that we tend to overlook the scope of its impact (and it also touches on a personal pet peeve). Business (and lately politics) has pounced on the word “accountable” as if making someone accountable will correct all problems. But it only underscores the truth of what Bob said about assigning blame for any/every single thing that is wrong.
So you might ask, “What’s so wrong about asking someone to be accountable?” It’s wrong because it’s all negative and never positive. “Accountability” implies blame, but never credit. Have you ever heard of anyone being accountable for a success?
Let’s stop using the word “accountable” and start using the word “responsible.” I’m willing to be responsible for results because it includes the possibility for praise upon success. But I will always try to skip opportunities to be accountable (and I still don’t want to be responsible for survey feedback about my service…).
Well played Mr. Lewis. One of your shortest, most pointed, most needed, and funny. Now do one on political surveys which always seem to support whoever sends it. You could probably find several in your mail or inbox these days.
A few years ago I was with my son when he was filling out a customer service survey for some product he had purchased. As he labored his way through “satisfied”, “very satisfied”, “extremely satisfied” and other categories, I said to him, “You know, some low paid retail clerk might have a component of variable pay hanging off your survey.”
“Thanks, Dad,” he said, and checked off “extremely satisfied” for every category.
The worst survey I ever took was an online survey from a local medical clinic. Any time I did not give the the highest score, up popped a follow up question “Why did we not excel?”
Realistically, no one is going to rate 10 out of 10 in every category. More likely, I will rate them above average. (After all, if I thought them below average I would not do business with them.)
Imagine the self-serving aspect of this approach.
1) They think they deserve A+ on everything; and if I don’t give it to them, then something is wrong.
2) The average customer will most likely just give them the highest rating to stop the stupid pop ups. Or, the customer will just quit the survey. Either way, the get all high marks.