Article : Benchmarking in Call Centers

The Truth About Benchmarking
It seems that trade magazines, consultants and conferences speakers alike are obsessed with benchmark data. Pressured to justify rosy promises for skyrocketing customer satisfaction and speedy return on investment, everyone is quoting on industry averages and how they are impacted by the strategy du jour.

But how credible and reliable are these benchmarks? Can you use them to compare with and rank your own operation? How would you react if your manager questioned you why your help desk's performance numbers are not quite on par with industry averages quoted in a recent magazine article, which is what happened recently to a client?

One of the more sought-after measurements in call center operation is the cost to operate various contact channels such as telephone, email and Web chat. Help desks attempt to lower their operating costs by channeling customer traffic to less-expensive self-service channels, and providers of self-service software often use cost-per-access metrics to substantiate their claims for return on investment.

We recently surveyed commercial benchmark databases, trade magazine articles, analyst reports and software vendor literature to find out their estimates for the cost per access for four access channels: telephone, email, Web-based self-help, and Web-based chat. The information compiled from 26 different sources is shown in the graph below. Each data point indicates the average cost per accesses quoted by one of the surveyed sources; some sources quote the cost as a range of costs, which are depicted by the vertical lines.

The first striking observation is that the average cost per telephone call varies between $3.50 and $32.74, more than a nine-fold difference! When considering the range of costs (as indicated by the vertical lines), the span of costs further increases to $2.00-$50.00 per call! Similarly wide disparity can be found in the cost per email and Web access, which range between $1.00-$40.00 and $0.05-$5.63, respectively.

What is the reason for this disparity? Who are you to believe? Which cost figure is most appropriate to compare to your own help desk? To answer these questions we need to take a closer look at the cost-per-access metric and how it is measured.

A common way to calculate cost per telephone call is to divide the cost of a help desk agent by the average number of calls the agent handles. However, the definition of "cost" varies dramatically from one organization to another, and depends on their accounting methods for various cost items such as IT capital costs, IT support costs, utilities and overhead.

The size, organizational structure and operational model of the help desk also plays a role in operating costs. There can be significant differences in staff utilization and internal efficiencies between a small help desk that handles a couple of hundred calls per day and a help desk at Microsoft staffed by 80 full time employees and contractors and Web site operating costs of about $4 million per year.

Even the definition of a "call" can be misleading. Because many help desks monitor customer incidents, or "tickets", and not actual telephone calls, it is impossible to tell if the benchmark numbers refer to the cost of an incident or the cost of each individual telephone call. When many incidents require several interactions between the help desk and the customer, the cost per incident will be proportionally higher than the cost per call.

These factors can explain the wide discrepancies in the cost-per-telephone-call figures. But inconsistent benchmark data is not limited to this metric. Almost every help desk metric is subject to lax, sometimes conflicting definitions and inconsistent measurements, making accurate comparison difficult and inconclusive. Another example is average handling time – the time it takes an agent to resolve a customer issue – which is important in determining resource load and utilization. Average handling times quoted by the various sources we surveyed ranged from as low as one minute to as high as several hours. Here again, the prime reason for the wide range stems from the way help desks measure and report their performance data. A help desk that defines resolution time as the total elapsed time from opening the call to a positive confirmation from the customer will report much longer handling times than a help desk that measures only actual telephone time.

Independent of how they are measured, resolution times are greatly impacted by the nature of the problem and the effort required from the help desk agent to resolve it. Resetting a password will typically take la few minutes, while troubleshooting a failure in a complex system may take hours. Simply comparing resolution times between help desks, and disregarding differences in subject matter, skill sets and the support environment will result in very misleading conclusions.

Peer Group Comparison
The approach many benchmark databases propose for comparing help desks is "peer group comparison" – data is organized by industry categories so that one can view select data from purportedly comparable help desks. Most benchmark questionnaires use an abbreviated version of the Standard Industrial Classification (SIC), which classifies organizations by their primary activity, where typical categories might include Advertising/Broadcasting/Graphics, Aerospace, Computer Hardware, Distributor, Insurance, Nonprofit, Research & Development, Utility and Wholesale – a hodgepodge of classifications that have little to do with the operational profile of the help desk. This segmentation, even when used in a consistent and objective fashion, does not provide any information about the type of support provided by the help desk. Many organizations employ separate help desk teams, providing diverse support services ranging from answering employee benefits questions to supporting software developers and field service engineers. Each of these help desk serves a very different audience and has dramatically different needs and operational characteristics, yet in most benchmark databases they will have be classified under a single industry category. Consequently, comparing two help desks classified in the database under "Insurance" may result in comparing a human resources help desk with an IT help desk. Conversely, an insurance company is not very likely to consider a manufacturing organization as a peer, yet if both help desks provide support for standard Windows applications, a comparison is likely justifiable.

Yet another point to consider when using industry benchmark databases is their sampling and measurement techniques. Without dwelling on statistical theories, it is important to recognize that most databases are populated with data solicited from those who were willing to contribute it, usually in return for free access to the results or similar incentives. The measurements were not designed and conducted by an independent observer using random sampling, but rather represent self-reported data of very small self-selected population.

Our conclusion from this analysis is that decision-making using "industry averages" without understanding how they were obtained and what they represent is fraught with risk. Fueled by media and vendor hype, many help desk managers as well as consultants frequently use incorrect and unsubstantiated assumptions and are overly focused on few, often wrong, metrics. To get the most value from benchmark data the help desk manager must first conduct a systematic Knowledge Gap Analysis to determine the relevant and important metrics at the help desk in question. Benchmark data from similar help desks can then serve as a source for additional data, to determine potential improvement areas but should never be selected blindly as an arbitrary target.