Archive for the ‘Bias’ Category
On the limitations of scoring methods for risk analysis
Introduction
A couple of months ago I wrote an article highlighting some of the pitfalls of using risk matrices. Risk matrices are an example of scoring methods , techniques which use ordinal scales to assess risks. In these methods, risks are ranked by some predefined criteria such as impact or expected loss, and the ranking is then used as the basis for decisions on how the risks should be addressed. Scoring methods are popular because they are easy to use. However, as Douglas Hubbard points out in his critique of current risk management practices, many commonly used scoring techniques are flawed. This post – based on Hubbard’s critique and research papers quoted therein – is a brief look at some of the flaws of risk scoring techniques.
Commonly used risk scoring techniques and problems associated with them
Scoring techniques fall under two major categories:
- Weighted scores: These use several ordered scales which are weighted according to perceived importance. For example: one might be asked to rate financial risk, technical risk and organisational risk on a scale of 1 to 5 for each, and then weight then by factors of 0.6, 0.3 and 0.1 respectively (possibly because the CFO – who happens to be the project sponsor – is more concerned about financial risk than any other risks ). The point is, the scores and weights assigned can be highly subjective – more on that below.
- Risk matrices: These rank risks along two dimensions – probability and impact – and assign them a qualitative ranking of high, medium or low depending on where they fall. Cox’s theorem shows such categorisations are internally inconsistent because the category boundaries are arbitrarily chosen.
Hubbard makes the point that, although both the above methods are endorsed by many standards and methodologies (including those used in project management), they should be used with caution because they are flawed. To quote from his book:
Together these ordinal/scoring methods are the benchmark for the analysis of risks and/or decisions in at least some component of most large organizations. Thousands of people have been certified in methods based in part on computing risk scores like this. The major management consulting firms have influenced virtually all of these standards. Since what these standards all have in common is the used of various scoring schemes instead of actual quantitative risk analysis methods, I will call them collectively the “scoring methods.” And all of them, without exception, are borderline or worthless. In practices, they may make many decisions far worse than they would have been using merely unaided judgements.
What is the basis for this claim? Hubbard points to the following:
- Scoring methods do not make any allowance for flawed perceptions of analysts who assign scores – i.e. they do not consider the effect of cognitive bias. I won’t dwell on this as I have previously written about the effect of cognitive biases in project risk management -see this post and this one, for example.
- Qualitative descriptions assigned to each score are understood differently by different people. Further, there is rarely any objective guidance as to how an analyst is to distinguish between a high or medium risk. Such advice may not even help: research by Budescu, Broomell and Po shows that there can be huge variances in understanding of qualitative descriptions, even when people are given specific guidelines what the descriptions or terms mean.
- Scoring methods add their own errors. Below are brief descriptions of some of these:
- In his paper on the risk matrix theorem, Cox mentions that “Typical risk matrices can correctly and unambiguously compare only a small fraction (e.g., less than 10%) of randomly selected pairs of hazards. They can assign identical ratings to quantitatively very different risks.” He calls this behaviour “range compression” – and it applies to any scoring technique that uses ranges.
- Assigned scores tend to cluster around the mid-low high range. Analysis by Hubbard shows that, on a 5 point scale, 75% of all responses are 3 or 4. This implies that changing a score from 3 to 4 or vice-versa can have a disproportionate effect on classification of risks.
- Scores implicitly assume that the magnitude of the quantity being assumed is directly proportional to the scale. For example, a score of 2 implies that the criterion being measured is twice as large as it would be for a score of 1. However, in reality, criteria are rarely linear as implied by such a scale.
- Scoring techniques often presume that the factors being scored are independent of each other – i.e. there are no correlations between factors. This assumption is rarely tested or justified in any way.
Many project management standards advocate the use of scoring techniques. To be fair, in many situations they are adequate as long as they are used with an understanding of their limitations. Seen in this light, Hubbard’s book is an admonition to standards and textbook writers to be more critical of the methods they advocate, and a warning to practitioners that an uncritical adherence to standards and best practices is not the best way to manage project risks .
Scoring done right
Just to be clear, Hubbard’s criticism is directed against scoring methods that use arbitrary, qualitative scales which are not justified by independent analysis. There are other techniques which, though superficially similar to these flawed scoring methods, are actually quite robust because they are:
- Based on observations.
- Use real measures (as opposed to arbitrary ones – such as “alignment with business objectives” on a scale of 1 to 5, without defining what “alignment” means.)
- Validated after the fact (and hence refined with use).
As an example of a sound scoring technique, Hubbard quotes this paper by Dawes, which presents evidence that linear scoring models are superior to intuition in clinical judgements. Strangely, although the weights themselves can be obtained through intuition, the scoring model outperforms clinical intuition. This happens because human intuition is good at identifying important factors, but not so hot at evaluating the net effect of several, possibly competing factors. Hence simple linear scoring models can outperform intuition. The key here is that the models are validated by checking the predictions against reality.
Another class of techniques use axioms based on logic to reduce inconsistencies in decisions. An example of such a technique is multi-attribute utility theory. Since they are based on logic, these methods can also be considered to have a solid foundation unlike those discussed in the previous section.
Conclusions
Many commonly used scoring methods in risk analysis are based on flaky theoretical foundations – or worse, none at all. To compound the problem, they are often used without any validation. A particularly ubiquitous example is the well-known and loved risk matrix. In his paper on risk matrices, Tony Cox shows how risk matrices can sometimes lead to decisions that are worse than those made on the basis of a coin toss. The fact that this is a possibility – even if only a small one – should worry anyone who uses risk matrices (or other flawed scoring techniques) without an understanding of their limitations.
Cognitive biases as project meta-risks – part 2
Introduction
Risk management is fundamentally about making decisions in the face of uncertainty. These decisions are based on perceptions of future events, supplemented by analyses of data relating to those events. As such, these decisions are subject to cognitive biases – human tendencies to base judgements on flawed perceptions of events and/or data. In an earlier post, I argued that cognitive biases are meta-risks, i.e. risks of risk analysis. An awareness of how these biases operate can pave the way towards reducing their effects on risk-related decisions. In this post I therefore look into the nature of cognitive biases. In particular:
- The role of intuition and rational thought in the expression of cognitive biases.
- The psychological process of attribute substitution which underlies judgement-related cognitive biases
I then take a brief look at ways in which the effect of bias in decision-making can be reduced.
The role of intuition and rational thought in the expression of cognitive biases
Research in psychology has established that human cognition works through two distinct processes: System 1 which corresponds to intuitive thought and System 2 which corresponds to rational thought. In his Nobel Prize lecture, Daniel Kahneman had this to say about the two systems:
The operations of System 1 are fast, automatic, effortless, associative, and often emotionally charged; they are also governed by habit, and are therefore difficult to control or modify. The operations of System 2 are slower, serial, effortful, and deliberately controlled; they are also relatively flexible and potentially rule-governed.
The surprise is that judgements always involve System 2 processes. In Kahneman’s words:
…the perceptual system and the intuitive operations of System 1 generate impressions of the attributes of objects of perception and thought. These impressions are not voluntary and need not be verbally explicit. In contrast, judgments are always explicit and intentional, whether or not they are overtly expressed. Thus, System 2 is involved in all judgments, whether they originate in impressions or in deliberate reasoning.
So, all judgements, whether intuitive or rational, are monitored by System 2. Kahneman suggests that this monitoring can be very cursory thus allowing System 1 impressions to be expressed directly, whether they are right or not. Seen in this light, cognitive biases are unedited (or at best lightly edited) expressions of often incorrect impressions.
Attribute substitution: a common mechanism for judgement-related biases
In a paper entitled Representativeness Revisited, Kahneman and Fredrick suggest that the psychological process of attribute substitution is the mechanism that underlies many cognitive biases. Attribute substitution is the tendency of people to answer a difficult decision-making question by interpreting it as a simpler (but related) one. In their paper, Kahneman and Fredrick describe attribute substitution as occurring when:
…an individual assesses a specified target attribute of a judgment object by substituting a related heuristic attribute that comes more readily to mind…
An example might help decode this somewhat academic description. I pick one from Kahneman’s Edge master class where he related the following:
When I was living in Canada, we asked people how much money they would be willing to pay to clean lakes from acid rain in the Halliburton region of Ontario, which is a small region of Ontario. We asked other people how much they would be willing to pay to clean lakes in all of Ontario.
People are willing to pay the same amount for the two quantities because they are paying to participate in the activity of cleaning a lake, or of cleaning lakes. How many lakes there are to clean is not their problem. This is a mechanism I think people should be familiar with. The idea that when you’re asked a question, you don’t answer that question, you answer another question that comes more readily to mind. That question is typically simpler; it’s associated, it’s not random; and then you map the answer to that other question onto whatever scale there is—it could be a scale of centimeters, or it could be a scale of pain, or it could be a scale of dollars, but you can recognize what is going on by looking at the variation in these variables. I could give you a lot of examples because one of the major tricks of the trade is understanding this attribute substitution business. How people answer questions.
Attribute substitution boils down to making judgements based on specific, known instances of events or issues under consideration. For example, people often overrate their own abilities because they base their self-assessments on specific instances where they did well, ignoring situations in which their performance was below par. Taking another example from the Edge class,
COMMENT: So for example in the Save the Children—types of programs, they focus you on the individual.
KAHNEMAN: Absolutely. There is even research showing that when you show pictures of ten children, it is less effective than when you show the picture of a single child. When you describe their stories, the single instance is more emotional than the several instances and it translates into the size of contributions. People are almost completely insensitive to amount in system one. Once you involve system two and systematic thinking, then they’ll act differently. But emotionally we are geared to respond to images and to instances…
Kahnemann sums it up in a line in his Nobel lecture: The essence of attribute substitution is that respondents offer a reasonable answer to a question that they have not been asked.
Several decision-making biases in risk analysis operate via attribute substitution – some of these include availability, representativeness, overconfidence and selective perception (see this post for specific examples drawn from high-profile failed projects). Armed with this understanding of how these meta-risks operate, lets look at how their effect can be minimised.
System two to the rescue, but…
The discussion of the previous section suggests that people often base judgements on specific instances that come to mind, ignoring the range of all possible instances. They do this because specific instances – usually concrete instances that have been experienced – come to mind more easily than the abstract “universe of possibilities.”
Those who make erroneous judgements will correct them only if they become aware of factors that they did not take into account when making the judgement, or when they realise that their conclusions are not logical. This can only happen through deliberation: rational analysis, which is possible only through a deliberate invocation of System 2 thinking.
Some of the ways in which System 2 can be helped along are:
- By reframing the question or issue in terms that forces analysts to consider the range of possible instances rather than specific instances. A common manifestation of the latter is when risk managers base their plans on the assumption that average conditions will occur – an assumption that Professor Sam Savage calls the flaw of averages (see Dr. Savage’s very entertaining and informative book for more on the flaw of averages and related statistical fallacies).
- By requiring analysts to come up with pros and cons for any decision they make. This forces them to consider possibilities they may not have taken into account when making the original decision.
- By basing decisions on relevant empirical or historical data instead of relying on intuitive impressions.
- By making the analysts aware of their propensity to be overconfident (or under-confident) by evaluating their probability calibration. One way to do this is by asking them to answer a series of trivia questions with confidence estimates for each of their answers (i.e. their self-estimated probability of being right). Their confidence estimates are then compared to the fraction of questions correctly answered. A well calibrated individual’s confidence estimates should be close to the percentage of correct answers. There is some evidence to suggest that analysts can be trained improve their calibration through cycles of testing and feedback. Calibration training is discussed in Douglas Hubbard’s book, The Failure of Risk Management. However, as discussed here, improved calibration by through feedback and repeated tests may not carry over to judgements in real-life situations.
Each of the above options forces analysts to consider instances other than the ones that readily come to mind. That said, they aren’t a sure-cure for the problem: System 2 thinking does not guarantee correctness. Kahneman discusses several reasons why this is so. First, it has been found that education and training in decision-related disciplines (like statistics) does not eliminate incorrect intuitions; it only reduces them in favourable circumstances (such as when the question is reframed to make statistical cues obvious). Second, he notes that sytem 2 thinking is easily derailed: research has shown that the efficiency of system 2 is impaired by time pressure and multi-tasking. (Managers who put their teams under time and multi-tasking pressures should take note!). Third, highly accessible values, which form the basis for initial intuitive judgements serve as anchors for subsequent system 2-based corrections. These corrections are generally insufficient – i.e. too small. And finally, System 2 thinking is of no use if it is based on incorrect assumptions: as a colleague once said, “Logic doesn’t get you anywhere if your premise is wrong.”
Conclusion
Cognitive biases are meta-risks that are responsible for many incorrect judgements in project (or any other) risk analysis . An apposite example is the financial crisis of 2008, which can be traced back to several biases such as groupthink, selective perception and over-optimism (among many others). An understanding of how these meta-risks operate suggest ways in which their effects can be reduced, though not eliminated altogether. In the end, the message is simple and obvious: for judgements that matter, there’s no substitute for due diligence – careful observation and thought, seasoned with an awareness of one’s own fallibility.
Cognitive biases as project meta-risks
Introduction and background
A comment by John Rusk on this post got me thinking about the effects of cognitive biases on the perception and analysis of project risks. A cognitive bias is a human tendency to base a judgement or decision on a flawed perception or understanding of data or events. A recent paper suggests that cognitive biases may have played a role in some high profile project failures. The author of the paper, Barry Shore, contends that the failures were caused by poor decisions which could be traced back to specific biases. A direct implication is that cognitive biases can have a significant negative effect on how project risks are perceived and acted upon. If true, this has consequences for the practice of risk management in projects (and other areas, for that matter). This essay discusses the role of cognitive biases in risk analysis, with a focus on project environments.
Following the pioneering work of Daniel Kahneman and Amos Tversky, there has been a lot of applied research on the role of cognitive biases in various areas of social sciences (see Kahneman’s Nobel Prize lecture for a very readable account of his work on cognitive biases). A lot of this research highlights the fallibility of intuitive decision making. But even judgements ostensibly based on data are subject to cognitive biases. An example of this is when data is misinterpreted to suit the decision-maker’s preconceptions (the so-called confirmation bias). Project risk management is largely about making decisions regarding uncertain events that might impact a project. It involves, among other things, estimating the likelihood of these events occurring and the resulting impact on the project. These estimates and the decisions based on them can be erroneous for a host of reasons. Cognitive biases are an often overlooked, yet universal, cause of error.
Cognitive biases as project meta-risks
So, what role do cognitive biases play in project risk analysis? Many researchers have considered specific cognitive biases as project risks: for example, in this paper, Flyvbjerg describes how the risks posed by optimism bias can be addressed using reference class forecasting (see my post on improving project forecasts for more on this). However, as suggested in the introduction, one can go further. The first point to note is that biases are part and parcel of the mental make up of humans, so any aspect of risk management that involves human judgment is subject to bias. As such, then, cognitive biases may be thought of as meta-risks: risks that affect risk analyses. Second, because they are a part of the mental baggage of all humans, overcoming them involves an understanding of the thought processes that govern decision-making, rather than externally-directed analyses (as in the case of risks). The analyst has to understand how his or her perception of risks may be affected by these meta-risks.
The publicly available research and professional literature on meta-risks in business and organisational contexts is sparse. One relevant reference is a paper by Jack Gray on meta-risks in financial portfolio management. The first few lines of the paper state,
“Meta-risks are qualitative, implicit risks that pass beyond the scope of explicit risks. Most are born out the complex interaction between the behaviour pattern of individuals and those of organizational structures” (italics mine).
Although he doesn’t use the phrase, Gray seems to be referring to cognitive biases – at least in part. This is confirmed by a reading of the paper. It describes, among other things, hubris (which roughly corresponds to the illusion of control) and discounting evidence that conflicts with one’s views (which corresponds to confirmation bias) as meta-risks. From this (admittedly small) sampling of the literature, it seems that the notion of cognitive biases as meta-risks has some precedent.
Next, let’s look at how biases can manifest themselves as meta-risks in a project environment. To keep the discussion manageable, I’ll focus on a small set of biases:
Anchoring: This refers to the tendency of humans to rely on a single piece of information when making a decision. I have seen this manifest itself in task duration estimation – where “estimates plucked out of thin air” by management serve as an anchor for subsequent estimation by the project team. See this post for more on anchoring in project situations. Anchoring is a meta-risk because the over-reliance on a single piece of information about a risk can have an adverse effect on decisions relating to that risk.
Availability: This refers to the tendency of people to base decisions on information that can be easily recalled, neglecting potentially more important information. As an example, a project manager might give undue weight to his or her most recent professional experiences when analysing project risks. Here availability is a meta-risk because it is a barrier to an objective consideration of risks that are not immediately apparent to the analyst.
Representativeness: This refers to the tendency to make judgements based on seemingly representative, known samples . For example, a project team member might base a task estimate based on another (seemingly) similar task, ignoring important differences between the two. Another manifestation of representativeness is when probabilities of events are estimated based on those of comparable, known events. An example of this is the gambler’s fallacy. This is clearly a meta-risk, especially where “expert judgement” is used as a technique to assess risk (Why? Because such judgements are invariably based on comparable tasks that the expert has encountered before.).
Selective perception: This refers to the tendency of individuals to give undue importance to data that supports their own views. Selective perception is a bias that we’re all subject to; we hear what we want to hear, see what we choose to see, and remain deaf and blind to the rest. This is a meta-risk because it results in a skewed (or incomplete) perception of risks.
Loss Aversion: This refers to the tendency of people to give preference to avoiding losses (even small losses) over making gains. In risk analysis this might manifest itself as overcautiousness. Loss aversion is a meta-risk because it might, for instance, result in the assignment of an unreasonably large probability of occurrence to a risk.
A particularly common manifestation of loss aversion in project environments is the sunk cost bias. In situations where significant investments have been made in projects, risk analysts might be biased towards downplaying risks.
Information bias: This is the tendency of some analysts to seek as much data as they can lay their hands on prior to making a decision. The danger here is of being swamped by too much irrelevant information. Data by itself does not improve the quality of decisions (see this post by Tim van Gelder for more on the dangers of data-centrism). Over-reliance on data – especially when there is no way to determine the quality and relevance of data as is often the case – can hinder risk analyses. Information bias is a meta-risk for two reasons already alluded to above; first, the data may not capture important qualitative factors and second, the data may not be relevant to the actual risk.
I could work my way through a few more of the biases listed here, but I think I’ve already made my point: projects encompass a spectrum of organisational and technical situations, so just about any cognitive bias is a potential meta-risk.
Conclusion
Cognitive biases are meta-risks because they can affect decisions pertaining to risks – i.e. they are risks of risk analysis. Shore’s research suggests that the risks posed by these meta-risks are very real; they can cause project failure So, at a practical level, project managers need to understand how cognitive biases could affect their own risk-related judgements (or any other judgements for that matter). The previous section provides illustrations of how selected cognitive biases can affect risk analyses; there are, of course, many more. Listing examples is illustrative, and helps make the point that cognitive biases are meta-risks. However, it is more useful and interesting to understand how biases operate and what we can do to overcome them. As I have mentioned above, overcoming biases requires an understanding of the thought processes through which humans make decisions in the face of uncertainty. Of particular interest is the role of intuition and rational thought in forming judgements, and the common mechanisms that underlie judgement-related cognitive biases. A knowledge and awareness of these mechanisms might help project managers in consciously countering the operation of cognitive biases in their own decision making. I’m currently making some notes on these topics, with the intent of publishing them in a forthcoming essay – please stay tuned.
—
Note
Part II of this post published here.
Eight to Late 