Measuring: You really do get what you measure

This is the last in a series of articles exploring business metrics and their usefulness in the engineering software space. The last few articles in this series have covered metrics that I believe are important in running a business but, of course, in order to analyze data, one needs the data in the first place. This post will cover discovering and maintaining business data so that valid conclusions can be reached. The first piece in this series covered external market data — size, growth rate, etc. — used to determine shares and trajectories. This is perhaps the easiest to come by, since market analysts like Schnitger Corporation spend much of their working time gathering it. But, as always, be prepared to investigate the methodologies and prejudices that go into creating the data models to understand how (if) your definitions match the analysts’. Keep in mind that a market that is defined too broadly is likely to lead a competitor to chase too many tangentials while a market that is too narrowly defined may not disclose all of the factors that could affect business down the road such as new entrants and missed opportunities. Share of wallet (aka share of requirement) is tougher to calculate. It’s a measure of how much of the budget your company or brand is able to capture — and the information can only be gathered from your customers. You can do it or you can have a third party do it, but it is crucial information to have on hand. Perhaps the best way to start is through a customer advisory group, which can offer representative data that doesn’t require surveying too large a group. Light versus heavy usage and some of the other external metrics are important too, and can best be gathered by simple observation. What do your users discuss at user group meetings or on chat boards? What do your support people report back about what they see at the customer site or in training sessions? For data of this type, accuracy to any number of decimal places is pointless; you’re looking for “bigger than a breadbox but smaller than a VW” and trends over time. Most of the metrics we discussed require detailed internal data and for conclusions to be accurate, this data must be accurate. Too many companies tell me that they can’t determine total revenue from a particular customer; there’s really no excuse for this. Assigning costs should be the harder part; use activity-based costing to break out direct and indirect costs per customer. If some costs are impossible to assign to specific customers, fine. Keep them as company costs; but be consistent from period to period. Customer acquisition costs are often found in many accounting buckets; again select a methodology and apply it consistently. The way customers buy and pay for products has changed dramatically over the last two years. Many customers have changed their definition of what they need and what how they pay for it. Every business has tremendous opportunities to mine existing data to better understand both old and new buying behaviors. A few last thoughts: • As engineers, we are obsessed with decimal places. Recognize that many business decisions should be based on directionality (up or down) without knowing the exact details. • Don’t assume a quantitative answer is correct. We often assign too much weight to a number; if intuition tells you that something is wrong, check the assumptions. • Wait at least one period before implementing any changes based on these metrics. If a customer looks unprofitable in period 1, it may be profitable in period 2 because of buying cycles. Wait until you understand the “seasonality” of your business and your customers’ patterns. Finally, don’t use the need to do more analyses as an excuse for putting off decisions. “Analysis paralysis” is a very real disease and its consequences can be fatal.