The essence of my argument was that, in environments where transactions are small in frequency, but large in magnitude ($’s), the traditional approach to forecasting destroys information — rather than creating it.

The traditional approach: statistics

Here’s an example.

Let’s assume that you are the manager of a project environment in a technology company.  Your job is to deploy teams of technical experts to deliver large projects that have been sold by your sales department.

Your challenge is that you need to make resource-allocation decisions based upon both current and future orders.

You ask your sales department for an estimate of future sales and — after much gnashing of teeth and grinding of gears — you are provided with a forecast.  This forecast advises that sales next month will be $653k; the following month: $432k; and so on.

These numbers seem strangely precise, considering that they are estimates.  Furthermore, the forecast doesn’t give you any indication of how confident you should be of these estimates falling within a particular range of values.

Suspicious; you ask the head of sales how these numbers were compiled.  He advises that the following method was used:

1. Salespeople used a set of rules to assign a probability to each opportunity upon which they are working (these rules prompt salespeople to consider factors like number of competitors, seniority of decision-makers, and so on).
2. Salespeople’s data was aggregated so as to provide a risk-adjusted estimate of future orders by calendar month.

Ask yourself, how useful is this forecast?

The forecast tells you that orders in two months’ time will total $297k.  When you enquire into this number you discover that it primarily consists of a $635k order that Sales believe they have a 34% probability of winning.

What decisions can you make on the basis of this estimate?  Can you set-aside 34% of the necessary resources?  Should you recruit new personnel?  And, in either case, can you pay for these resources using the $216k revenue that constitutes the risk-adjusted value of the $635k sale ($635k x 34%)?

Of course not.

If Sales fails to win that order in two months’ time as projected, the project will place no demand whatsoever on your resource pool, and will generate precisely $0 in revenues!

To use statistics in this environment is obviously foolish because orders are won so infrequently.  The problem is compounded by assigning probabilities to opportunities based primarily upon salespeople’s (subjective) opinions.

So, if this approach is so obviously foolish, why do so many organisations adopt it?

I can think of a handful of reasons:

1. Because it does make sense in environments where transactions are high in frequency and low in magnitude
2. Because it appears (at first glance) to be scientific
3. Because senior management (or other departments) demand it
4. Because the CRM came with this capability built-in
5. Because no one is aware of an alternative approach

An alternate approach: heuristics

Let me propose an alternative approach — the approach we use internally.

But first, let’s remind ourselves that, because the dataset upon which we are reporting is not statistically significant, our approach must not make use of statistics.

The alternative approach is remarkably simple.

Sales maintains two sets of assumptions for each opportunity:

1. Optimistic (but not exuberant)
2. Pessimistic (but not paranoid)

So, for each of the critical opportunity variables (opportunity value and commencement month) both an optimistic and a pessimistic value is recorded — and revised weekly.  These values are not calculated, they are arrived at by discussion in sales meetings.  In other words, a value for each variable is negotiated by the salesperson, the sales coordinator and the sales manager.

Then, each week, two scenarios are created for distribution to senior management (and other departments) — the optimistic and the pessimistic scenario.  Each scenario details specific projects (and the full value of each) — as opposed to risk-adjusted aggregates. Ideally, each scenario will also detail resourcing and cashflow implications.

Each department then uses these two scenarios to plan as best they can.

What happened to precision?

At first glance the second (heuristic) approach lacks the precision of the traditional approach to forecasting (statistical).

However, upon closer examination it becomes obvious that the precision of the traditional approach is an illusion.  Contrary to popular belief, the mathematical manipulation of salespeople’s opinions does not convert subjective data into objective information.

The heuristic approach acknowledges the existence of uncertainty, rather than disguising it.  Accordingly, it is significantly more useful to those who intend to use the reports provided by sales to make actual business decisions.

The relatively primitive lifestyles of these islanders were interrupted by Japanese aircraft dropping large supplies of clothing, medicine, canned food and tents to support the Japanese war effort.

Some of these supplies were shared with islanders, in exchange for their assistance.

After the war, when planes and their valuable cargos disappeared, some islanders took to imitating the rituals they’d observed the Japanese performing. They carved headphones from wood and wore them while sitting in fabricated control towers.  And they waved landing signals while standing on abandoned runways.

I’ve noticed the emergence of a similar cargo cult in organisations in recent years — particularly those organisations that sell major products and services.

Sales departments have observed a rapid evolution in the performance of their organisations’ operations departments. They’ve seen outputs increase by orders of magnitude. And they’ve seen quality and on-time performance improve by similar degrees.

Of course, sales departments have also observed that these performance improvements have been accompanied by an increase in the usage of mathematics in operations — and, particularly, an increase in the role of statistics.

Sales departments have taken to imitating these rituals — in the hope that they will have a similar effect on the performance of their departments. Increasingly, we’re seeing mathematical models, sophisticated databases and business intelligence tools being applied in the pursuit of greater (or more consistent) sales results.

What sales departments fail to recognise, however, is that operations uses such tools to measure (and predict) the behaviour of processes that are inherently measureable. Sales processes, for the most part, are not.

Let’s consider the practice of sales forecasting: a perfect example of this management hocus-pocus.

In most major-account sales environments, salespeople are expected to regularly enter data into their firms’ CRM’s, reporting on the status of sales opportunities and the activities performed in pursuit of these opportunities. This data is then manipulated by management and used to estimate month-by-month revenue numbers. These revenue numbers are then provided to operations so that they can be used to determine production quotas and raw-material requirements; as well as to update cashflow models.

At first glance this practice sounds sensible enough. However, we don’t have to dig far beneath the surface to realise that the mathematics is not quite as elegant as it appears!

The first thing that we should recognise is that, in most organisations, major sales are landed quite infrequently. This is due, in part, to the fact that salespeople dedicate only a small amount of their time to business-development (selling) activities. Most of their time is consumed by customer-service and clerical tasks, and fulfilment-related activities. (Believe it or not, a typical salesperson performs just two business-development appointments a week.)

Aside from infrequent major sales, another symptom of salespeople’s limited business-development activity is a shortage of smaller sales. (Salespeople’s limited capacity and their performance-based pay cause them to focus on larger transactions.)

Because salespeople dedicate only a small amount of their capacity to selling, it follows that the organisation has only a small number of opportunities under management at any one point in time. Management must scrutinise this small dataset and somehow predict the month-by-month emergence of major (and infrequent) orders.

In most organisations, management should recognise the impossibility of this undertaking and resolve that it has as much chance of predicting sales revenue in three months time as it does the weather!

But management is not so easily deterred. It is true that there are only a small number of opportunities under management. However, management has been insisting that salespeople enter an increasing volume of data relating to each opportunity into the CRM. The assumption is that more data means more accurate predictions.

Let’s consider the nature and source of this data and the conditions under which it is collected.

Aside from minimal amounts of objective data, salespeople are expected to enter numerous best-guesses relating to everything from customer predispositions to competitor intentions. Even much of the data that appears to be objective at first glance reveals itself as otherwise under close scrutiny. For example, even though the opportunity-management process is divided into a series of milestones, the achievement of few of these milestones is actually signalled by unambiguous customer behaviour.

What’s even scarier is that few organisations have an objective definition of sales opportunity. It’s a common phenomenon for major opportunities to appear in the CRM just days before they are won when, in reality, opportunities of this size would have required months’ worth of work.

The prevailing assumption in sales departments seems to be that if we take a large number of subjective data and multiply them together, the process of multiplication will somehow furnish us with an objective result.

But it gets worse. The inaccuracy of the data in the CRM is not just limited by salespeople’s personal biases. In most cases the problem is compounded by the fact that salespeople enter data into the CRM weekly (as opposed to daily) and by the fact that management provides opportunities and incentives (of various kinds) to salespeople to game the system by flavouring the data that they enter.

Now the lunacy of this pointless and time-consuming exercise does not escape operations. In most organisations, finance and production have concluded that their best-guesses are more accurate than the numbers delivered by the sales department’s elaborate models. If these forecasts are put to use, it’s more likely that they will be used as ammunition in the ongoing rivalry between operations and sales!

If the sales department wishes to remedy this problem, there are two obvious courses of action. It can either accept that forecasts in which operations cannot have the necessary degree of confidence are of less value than no forecasts at all — and stop producing them. Or it can figure out how to reengineer the sales department so that its output becomes predictable.

At a minimum, the latter course of action would require that salespeople spend the majority of their time selling — and that they pursue a sensible mix of major and minor sales. It would require an objective definition of sales opportunity, and milestones that are achieved only when potential customers exhibit unambiguous behaviours (customer opinions are worthless). And it would require that the responsibility for scheduling salespeople’s activities and compiling reports be removed from salespeople.

Of course, if management were to examine operations carefully, it would discover that operation’s mathematical models and scientific methods begin with the careful collection of objective data. Operations people know that garbage in equals garbage out.

Even though some Pacific Island cargo cults persist to this day, Islanders’ rituals are as unproductive now as they were in the nineteen-fifties.

It’s time that sales departments recognise that the root cause of inaccurate sales forecasts has nothing at all to do with inferior mathematical models and insufficient technology.