Archive for June 2008
Improving project forecasts
Many projects are plagued by cost overruns and benefit shortfalls. So much so that a quick search on Google News almost invariably returns a recent news item reporting a high-profile cost overrun. In a 2006 paper entitled, From Nobel Prize to Project Management: Getting Risks Right, Bent Flyvbjerg discusses the use of reference class forecasting to reduce inaccuracies in project forecasting. This technique, which is based on theories of decision-making in uncertain (or risky) environments,1 forecasts the outcome of a planned action based on actual outcomes in a collection of actions similar to the one being forecast. In this post I present a brief overview of reference class forecasting and its application to estimating projects. The discussion is based on Flyvbjerg’s paper.
According to Flyvbjerg, the reasons for inaccuracies in project forecasts fall into one or more of the following categories:
- Technical – These are reasons pertaining to unreliable data or the use of inappropriate forecasting models.
- Psychological – This pertains to the inability of most people to judge future events in an objective way. Typically it manifests itself as undue optimism, unsubstantiated by facts; behaviour that is sometimes referred to as optimism bias. This is the reason for statements like, “No problem, we’ll get this to you in a day.” – when the actual time is more like a week.
- Political – This refers to the tendency of people to misrepresent things for their own gain – e.g. one might understate costs and / or overstate benefits in order to get a project funded. Such behaviour is sometimes called strategic misrepresentation (commonly known as lying!) .
Technical explanations are often used to explain inaccurate forecasts. However, Flyvbjerg rules these out as valid explanations for the following reasons. Firstly, inaccuracies attributable to data errors (technical errors) should be normally distributed with average zero, but actual inaccuracies were shown to be non-normal in a variety of cases. Secondly, if inaccuracies in data and models were the problem, one would expect this to get better as models and data collection techniques get better. However, this clearly isn’t the case, as projects continue to suffer from huge forecasting errors.
Based on the above Flyvbjerg concludes that technical explanations do not account for forecast inaccuracies as comprehensively as psychological and political explanations do. Both the latter involve human bias. Such bias is inevitable when one takes an inside view, which focuses on the internals of a project – i.e. the means (or processes) through which a project will be implemented. Instead, Flyvbjerg suggests taking an outside view – one which focuses on outcomes of similar (already completed) projects rather than on the current project. This is precisely what reference class forecasting does, as I explain below.
Reference class forecasting is a systematic way of taking an outside view of planned activities, thereby eliminating human bias. In the context of projects this amounts to creating a probability distribution of estimates based on data for completed projects that are similar to the one of interest, and then comparing the said project with the distribution in order to get a most likely outcome. Basically, reference class forecasting consists of the following steps:
- Collecting data for a number of similar past projects – these projects form the reference class. The reference class must encompass a sufficient number of projects to produce a meaningful statistical distribution, but individual projects must be similar to the project of interest.
- Establishing a probability distribution based on (reliable!) data for the reference class. The challenge here is to get good data for a sufficient number of reference class projects.
- Predicting most likely outcomes for the project of interest based on comparisons with the reference class distribution.
In the paper, Flyvbjerg describes an application of reference class forecasting to large scale transport infrastructure projects. The processes and procedures used are published in a guidance document entitled Procedures for Dealing with Optimism Bias in Transport Planning, so I won’t go into details here. The trick, of course, is to get reliable data for similar projects. Not an easy task.
To conclude, project forecasts are often off the mark by a wide margin. Reference class forecasting is an objective technique that eliminates human bias from the estimating process. However, because of the cost and effort involved in building the reference distribution, it may only be practical to use it on megaprojects.
1Daniel Kahnemann received the Nobel Prize in Economics in 2002 for his work on how people make decisions in uncertain situations. His work, which is called Prospect Theory, forms the basis of Reference Class Forecasting.
Lead, don’t take the easy way out
Over the last few weeks, parliamentary proceedings in Australia have been dominated by debates (if one can call them that) on the price of petrol. In the process, the public has been treated to the unedifying spectacle of a government and an opposition squabbling over a GST cut on excise which, if passed, will reduce the price of petrol by the princely sum of 4 cents per litre. A cut that will sooner than later be swallowed by ever rising oil prices.
Rather, than lead – in this case by telling the truth about hard choices that face us – politicians continue to take the easy way out by looking after their own short-term interests (i.e. the next election). Hence the fixation on cutting petrol prices, even if by only an insignificant amount. The truth is we need to look at long-term solutions such as improving public transport and fuel efficiency while also looking at alternate energy sources. All hard yet necessary options which, if implemented, might well irritate the electorate. Incidentally, regarding the first point, anecdotal evidence suggests that soaring petrol prices have already pushed more people into public transport, thereby putting further strain on an already creaky system. Addressing that, for a start, would be more productive than arguing over a 4c price reduction.
In the words of Ross Gittins, a Sydney Morning Herald columnist – our pollies are too gutless to give us the bad oil . And there lies a lesson in how not to lead, because Gittins is absolutely right: our politicians aren’t leading, they’re taking the easy way out.
Communication impedance mismatches on projects
Introduction
Most folks have been in situations where something they’ve said or written has been comprehensively misunderstood by the recipient of the message. When this happens it’s as if the other party is on a different wavelength, thus completely missing what’s conveyed. In this post I propose the term communication impedance mismatch to refer to this phenomenon. Below I explain why this bit of jargon, which has its origins in electrical engineering, is an appropriate one to use in this context. I also look at some reasons why communication impedance mismatches are so common on projects. In this connection, readers may also want to have a look at my earlier post on obstacles to project communication.
What’s an impedance mismatch?
So, what is an impedance mismatch? A good place to start is with some definitions from Wikipedia. Electrical impedance is essentially a measure of opposition to the flow of alternating current in a circuit. The impedance of a circuit component depends, among other things, on the frequency1 of the alternating signal. Now, for a fixed frequency, it is possible to adjust the circuit impedance so that power transfer through the circuit is maximised. This is called impedance matching. Basically, if impedances aren’t matched, power transfer through the circuit isn’t optimal.
Impedance matching is the principle behind radio tuning (and hence a connection to communication). In brief, radio tuning works as follows: impedance varies with signal frequency (or wavelength); so, for a fixed impedance, signals of a specific frequency – the tuned frequency – will be “let through” while the others will be “blocked”2. Although I’ve been using frequency as the variable here, I could just as well have used wavelength as the two are related. So, the wavelength metaphor that I used earlier is really quite apt- if the other party is on a different wavelength they will not get the message.
Anyway, this technical term from physics and electrical engineering has a history of being appropriated by other fields (see this post, for example). As an example from software development, the object-oriented programming crowd use the term to refer to the mismatch between data representations in a relational and object-oriented worlds. The term has a nice “jargony” feel about it. And seeing that it’s been appropriated by others before, I have no hesitation in appropriating it for the communication lexicon.
Why are communication impedance mismatches common on projects?
OK, so why am I so fussed about communication impedance mismatches on projects? Here why: at least one study claims that poor communications are the most frequent cause of project failure. It is therefore worth looking at why communication impedance mismatches are so common on projects. Here are some reasons that come to mind:
-
Team members don’t know each other well: A project is, by definition, a time-bound undertaking with a clear start and finish. Hence, in many cases, the people involved in a project would not have worked with each other before. Even worse, they may not even know each other. Such a situation is fraught with the potential for communication impedance mismatches. To alleviate this, it is sometimes recommended that team members spend time getting to know each other before the project begins. This is often done via team building activities, which I confess I’m not a great fan of. I much prefer letting people find their own niche within a team, rather than forcing a false sense of togetherness through contrived activities. Either way, a project manager has to be conscious of the potential for misunderstandings caused by team members simply not knowing each other well enough.
-
Projects are high-stress environments: Projects can be high stress environments, especially when things aren’t going well. Paradoxically, it is when things aren’t going well that good communication is needed. However, in times of stress, one generally finds that communication impedance mismatches rule. Minor misunderstandings can be blown all out of proportion. At such times, a good project manager acts as an impedance matching device, getting all involved parties to communicate with each other on a common wavelength.
-
Communication gap between the customer and supplier: The objectives of customers and suppliers on projects are typically different, and often even contradictory (for example, the customer wants it done cheap whereas the supplier wants to make as much money as reasonably possible). This fundamental tension between the two parties often leads to communication impedance mismatches. These can be resolved by a project manager who understands both points of view, and looks for negotiated, or collaborative, solutions that take into account both parties’ objectives.
These are just some of the reasons for the ubiquity of communication impedance mismatches in project environments. There are a host of others, which I’m sure you may have come across in your work. I’d welcome additions to the list through your comments.
In Conclusion
Communication impedance mismatches occurs whenever communications – written, verbal or otherwise – are misunderstood. They often occur in a project environment because of the temporary and time-bound nature of projects, and also because projects are comprised of parties with conflicting interests. A project manager has to be aware of the potential for communication impedance mismatches, so that he or she can act to reduce them before they cause unnecessary strife.
1The standard mains frequency is 60 Hz (or cycles/second) in the US and 50 Hz elsewhere..
2I couldn’t find any good, non-technical online references, but see this short explanation or this longer one in Yahoo Answers for more..
Eight to Late 