There Are Lies, Damned Lies And Statistics.
Benchmarks are a peculiarly virulent form of statistics which can be used to make a positive change, or misused to make a dodgy sale.
On one hand a benchmark, such as a world record in 100m race, can motivate an athlete to perform better and better each time they train.
It gives something to aim towards, sets an expectation that if others can do it, then why you cannot, and leads to search for better ways of achieving the results.
But all that happens only if the benchmarks have credibility.
When you find out what Lance Armstrong has been doing to achieve his miraculous results you lose faith in the benchmark, and start suspecting other benchmarks too.
You test every benchmark extremely carefully, and use it sparingly lest you get dodged again.
An example is in freight rates benchmarks. Costs of operating a truck, a train or a ship are, by now, indexed to nth degree.
One can quite easily benchmark prices to these operating costs without any consideration of the local market conditions, your peculiar service requirements, and the balance between the supply and demand -which changes from moment to moment.
When a ship is full, it is full. The only way to get your container on board that ship would be get someone else’s container knocked off.
That will require a lot more price incentive or market power than most buyers are willing to muster.
This is another way of saying that supply chain infrastructure suffers from highly inelastic pricing paradigm, and as such price benchmarks are useful only to compare the changes in the market conditions from moment to moment.
When analysts start using these kind of benchmarks for performance reviews, or assess cost savings potential – they are not only missing the point, they are really using dodgy statistics to make a sale.
In our work, we use benchmarks many times but we are extremely careful to use the right benchmark for the right purpose and state its limitations.
Most executives who are not close enough to the benchmarks or the markets can easily miss the fine print to their regret later.