Measurement dysfunction

From Joel on Software: a good pice on Measurement (via Lee at Headshift)

“Thank you for calling may I help you?” Then — Click! You’re cut off. That’s annoying. You just waited 10 minutes to get through to a human and you mysteriously got disconnected right away.

Or is it mysterious? According to Mike Daisey Amazon rated their customer service representatives based on the number of calls taken per hour. The best way to get your performance rating up was to hang up on customers thus increasing the number of calls you can take every hour.

An aberration, you say?

When Jeff Weitzen took over Gateway, he instituted a new policy to save money on customer service calls. “Reps who spent more than 13 minutes talking to a customer didn’t get their monthly bonuses,” writes Katrina Brooker (Business 2.0, April 2001). “As a result, workers began doing just about anything to get customers off the phone: pretending the line wasn’t working, hanging up, or often–at great expense–sending them new parts or computers. Not surprisingly, Gateway’s customer satisfaction rates, once the best in the industry, fell below average.”

It seems human nature to want to measure stuff so what’s the solution to gaming? I suggest that each time one of these measures is set, a group of people discuss the potential downsides and put in a place a qualitative way of assessing its impact. Like an open-space discussion for those being measured to talk about its impact on them. As well as reading the seductive numbers, the manager has to show up for the meeting…

4 thoughts on “Measurement dysfunction

  1. Paul Goodison

    How true that is. I have heard stories of agents being rewarded on the number of calls they can take in an hour and consequently putting the phone down as people come through all day, never actually serving anyone. And another call centre mangement initiative to accept calls during busy times only to arrange to call back in quiet times. The figures making this call centre have much better average handling times than others and giving customers a poor experience.

    Personally I would like to see a first time fix approach ensuring that the customer has an issue or problem resolved before hanging up. That would move towards a more qualitative approach while allowing it to be measured.

    Of course management and practioners engaging in a conversation about what they are trying to achieve would probably be a first in most places, let alone discussing the impact of objectives and measures on the customer.

  2. Ton Zijlstra

    Hi John,

    What most measurements fail to take into account is that the measurement influences the measurement. Like Dave Snowden said, humans are experts in gaming the system.

    Oversimplification is in my experience a large contributing factor in this: loads of organisations think one measurement/indicator is enough to rate a process or an entire system.

    This opens it up to people making the measurement the goal of their activities in stead of the actual desire outcome of those activities.

    And who can blame them? If their managers think they are one-dimensional (one measurement says it all) people will act one-dimensional.

    So we need more measurements around a process/functional unit that together feed decisionmaking and as you say introduce more qualitative ones. Verna Allee view on value networks provides a great entry point for that I think. First step: stop connecting single measurements to reward systems.

  3. Chris Macrae

    The world’s largest organisations are ruled by measurements that conflict with people systems (emotional quality of relationships and transparent mapping of how and when and where to connect).

    The number 1 system bias is separate everything with the additive operand. Look at the logic of the ridiculous numbers that ‘add up’ at the back of an annual report. Let’s separate the past from how relationships compound the future. Let’s pretend if we are worth billions to business stakeholders but nothing that society trusts, billions+0=billions; reality demonstrated by Andersen and 50 other disasters since 2000 is billions*0=0

    The number 2 living system bias is charge people as costs (machines can be owned as investments)

    The number 3 bias is in a networking world where every system compounds impacts on others is to rule through RofI not for all shareholders but only speculators; not for all stakeholders but only for speculators; not for globalisation impacts across networks of corporations and governments but for speculators and their lawyers in Delaware.

    As academic of law, Joel Bakan, says in his book and movie this measurement system makes our biggest corporations pathological. Unless we introduce a second twin standard of goverance to map the value multiplication of intangibles and the trust-flow auditing of conflicts, we will be enslaving our kids in a world far more viciously systemised than anything Orwell’s Big Brother imagined.

    London declares itself as the first Intangibles Collaboration City -let’s open source the system maps and transparent networking maps of value

    multiply. Is this an issue we can discuss at Islington’s First Tuesday Roundhouses on Authenticity?


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.