(originally published in Customer Management Insight by ICMI)

Getting a Real Measure on Satisfaction (Part I)

Surveys alone do not reflect true customer satisfaction levels. Behavioral metrics hold the key to managing dissatisfaction.

Have you noticed how you are seemingly bombarded with requests to fill out satisfaction surveys these days? They pop up in your email after a hotel stay; they come in over the phone after you’ve had your car serviced; and they dance across your screen while you’re trying to read a Web page.

I’ve watched this take place over the past few years with a mixed reaction. On the one hand, I’m pleased to see customer satisfaction take a “front-and-center” position on the performance metric buffet — clearly, it’s long overdue. On the other hand, most of us are getting precious little value out of the exercise.

Given the popularity of satisfaction surveys, it seems dangerous to be challenging their wisdom. The intent, in many cases, is terrific. I firmly believe that many service professionals want to know what customers really think and feel about the service they just received — and if you can get to it, the information is pure gold. The failure, though, comes in the execution.

What’s Causing the Problem?

There are three main reasons why the information you get from these surveys is only vaguely reflective of true satisfaction levels:

1. The view of what constitutes a customer is much too narrow;
2. Evaluations are heavily slanted toward input rather than behavior; and
3. Some of you are manipulating the hell out of the data.

Those three points lay the groundwork for a two-part article series. I’ll focus on the first two reasons in this article, and in Part 2, we’ll have some fun with those who are using the data for all the wrong reasons.

Let’s keep in mind our goal here as we move forward — we truly want to know what the customer thinks and feels about our service (and possibly, our product). So if we are taking a random sample of customers and running a survey past them, how could our view be too narrow? Let’s think in terms of a retail operation first. Then we’ll turn our attention over to call centers.

Breakfast at Café Queue

I was reminded of my “narrow view of the customer” theory the other day when I was on the road and stopped at my favorite chain café for a morning coffee and bagel. It was fairly busy when I arrived and, shortly thereafter, the queue grew even longer.

I had my breakfast at a table near the front door and watched. In a 10-minute span, at least six potential customers, before stepping inside, either walked or drove away once they saw how long the line was. (Now there’s an exercise here in lost revenue that shouldn’t be ignored, but it doesn’t quite fit our discussion about satisfaction surveys. But for those of you who, like me, can’t help but run the numbers, read the box below for a quick analysis of the amount of money that’s going up in smoke.)

So what does this scenario have to do with satisfaction surveys? In this café, the customer satisfaction surveys were located next to the register. I didn’t notice how many of the paying customers picked one up, but I can guarantee that none of the walk-aways did. This results in three fatal outcomes:

1. The café’s ratings will be higher than they should be.
2. Management may never realize that their main problem is slow service because the people who could provide that feedback never walk in the door.
3. Because of Point No. 2, management will always underestimate the importance of speedy service in overall satisfaction.

How Does It Apply to Call Centers?

Of course, in a retail operation, it is easy to see all of this unfold in front of you (if you’re looking, that is). What about in a call center operation? If you run a call center, suppose that you randomly select 100 of your callers each day to survey. Assuming that 100 is a statistically significant sample, how could this possibly be a problem? Well, similar to our café example, you would be missing out on the abandons. And you would miss out on those who get busy signals and don’t call back. And you would miss out on those callers who had a previous bad experience with your company and who now do everything they can to avoid contacting you. And you would miss out on those who aren’t able to contact you during the hours that you’re open.

What’s the common trait among the calls that you would miss? They have a complaint. From “you take too long to answer” to “your service is terrible” to “you aren’t open when I have time to call,” the people who never make it into the potential population sample are the ones who are decidedly low on the satisfaction scale.

What Can You Do About It?

When managing dissatisfaction, we need to develop a much more extensive view of measures of dissatisfaction. Sure, a low rating on a customer satisfaction survey pretty clearly denotes a problem. And how about an abandoned call? Isn’t the fact that the caller hung up a clear indication that you took longer to answer than what he or she would have liked? It certainly seems to me that an abandoned call is a complaint. And what about customers who call after hours? Wouldn’t that be a pretty clear indication of someone who doesn’t appreciate your hours of availability?

Our inclusion of these measures begins to address the second reason why customer satisfaction survey feedback does not reflect true satisfaction levels — favoring input over behavior. We typically limit our evaluation of customer satisfaction to surveys only, yet there are many measures of behavior that paint a more clear and accurate picture. Consider the following scenario: As part of a satisfaction survey, a call center asks if customers were satisfied with the length of time it took to answer their call. Ninety-five percent say yes, so the center reports that they are answering in a timely manner — after all, only 5 percent of survey respondents were dissatisfied with the answer time. But 8 percent of callers are abandoning. Wouldn’t it be more appropriate to say that the percent of those dissatisfied with the answer time is closer to 13 percent?

Abandoned rates and after-hours calls are some of the more obvious behavioral indicators of dissatisfaction, but you need not stop there. Consider all of the calls that get transferred in your center. Can you imagine that those callers were hoping to explain their story to one agent, only to get transferred and have to repeat it all over again? Neither can I. Ditto for voicemail messages, if you happen to engage in the practice of giving callers an option to leave them (a practice we don’t typically recommend). What about callers who are placed on hold? The same logic applies — people don’t make a call hoping that they’ll get stuck listening to hold music while an agent tries to find an answer. Escalated complaints? These are yet another behavioral indicator, whether they are via phone, email, fax or postal mail. Transfers, voicemails, holds and escalated complaints are all behavioral metrics that leave a pretty strong clue about dissatisfaction. Measure them, and reduce them, and you are bound to have higher levels of satisfaction.

So are we really advocating considering abandons, busies, after-hour calls, transfers, voicemails, holds and escalated complaints as measures of caller dissatisfaction? To an extent, yes, we are. Clearly, a caller who is put on hold for 33 seconds while an agent verifies information is less likely to have a serious service complaint than one who has just sent a scathing letter to the CEO. But we need to make sure that our measurements are inclusive, and we need to recognize the importance of behavioral measurements. Yes, it is a painful exercise. We don’t like hearing about complaints in the first place, so it’s even more difficult to go out of our way to find them. Yet if we really want to measure true satisfaction and address the most pressing problems, there is no choice but to get the full, accurate picture.

Calculating the Abandoned Rate There are many theories about how to count abandoned calls. Some feel that anytime a caller hangs up before being answered the call should be considered abandoned. Others try to eliminate the “fast-cleardowns” (those who abandon shortly after getting in queue) from the abandoned total. If you decide to deselect these, you’ll have to come up with a threshold that distinguishes a fast-cleardown from an abandoned. I’m of the opinion that if you use a threshold you should set it low — no more than 10 seconds or so. By that time, you’ll have cleared out wrong numbers, and that’s your main goal. How Much Potential Revenue Are You Losing? In our café scenario, within a 10-minute span of time, six potential customers never even stepped foot inside to place an order. In a call center situation, that is called an abandoned call. In a retail operation, it is called… nothing. No one ever noticed it, let alone captured it in a report. Some unsuspecting assistant manager ran a sales report at the end of the shift and probably felt pretty good about the numbers, oblivious to how much better they could have been. But it’s far from nothing. Six customers every 10 minutes equates to 36 lost customers in an hour. If you figure on two strong morning hours, that’s a total of 72 lost customers that morning. Multiply that by $5 on average per customer order, and that’s $360 in extra revenue you could have seen. The margin on this kind of incremental revenue is at least 50 percent, and so you’ve lost out on a minimum of $180 in profit, probably more. What would it have taken to get to that money? Well, there were two people serving the queue. Frankly, they weren’t terribly efficient, so you probably could have resolved the issue with some good old-fashioned performance coaching and/or process reengineering. But even if you had to add a third for a four-hour shift, that wouldn’t have cost more than $50, and probably less. Take that from our $180 of profit, and you are still left with an extra $130 per morning, five mornings a week, for a total of $650 a week or over $33,000 annually — which is probably pretty darn close to what that unsuspecting assistant manager makes a year. And if, by chance, the same thing is happening in all of this chains’ 1,027 company-owned and franchised units, then the resulting $34 million is enough lost revenue to make a CFO cry (by the way, if you happen to be that CFO and you are reading this, rest assured that our consulting fee to help you fix this is considerably less than $34 million).

- Jay Minnucci
President
Service Agility
(215) 679-5250
jaym@serviceagility.com