Archive for the ‘Project Management’ Category
Cox’s risk matrix theorem and its implications for project risk management
Introduction
One of the standard ways of characterising risk on projects is to use matrices which categorise risks by impact and probability of occurrence. These matrices provide a qualitative risk ranking in categories such as high, medium and low (or colour: red, yellow and green). Such rankings are often used to prioritise and allocate resources to manage risks. There is a widespread belief that the qualitative ranking provided by matrices reflects an underlying quantitative ranking. In a paper entitled, What’s wrong with risk matrices?, Tony Cox shows that the qualitative risk ranking provided by a risk matrix will agree with the quantitative risk ranking only if the matrix is constructed according to certain general principles. This post is devoted to an exposition of these principles and their consequences.
Since the content of this post may seem overly academic to some of my readers, I think it is worth clarifying why I believe an understanding of Cox’s principles is important for project managers. First, 3×3 and 4×4 risk matrices are widely used in managing project risk. Typically these matrices are constructed in an intuitive (but arbitrary) manner. Cox shows – using very general assumptions – that there is only one sensible colouring scheme (or form) of these matrices. This conclusion was surprising to me, and I think that many readers may also find it so. Second, and possibly more important, is that the arguments presented in the paper show that it is impossible to maintain perfect congruence between qualitative (matrix) and quantitative rankings. As I discuss later, this is essentially due to the impossibility of representing quantitative rankings accurately on a rectangular grid. Developing an understanding of these points will enable project managers to use risk matrices in a more logically sound manner.
Background and preliminaries
Let’s begin with some terminology that’s well known to most project managers:
Probability: This is the likelihood that a risk will occur. It is quantified as a number between 0 (will definitely not occur) and 1 (will definitely occur).
Impact (termed “consequence” in the paper): This is the severity of the risk should it occur. It can also be quantified as a number between 0 (lowest severity) and 1(highest severity).
Note that the above scales for probability and impact are arbitrary – other common choices are percentages or a scale of 0 to 10.
Risk: In many project risk management frameworks, risk is characterised by the formula: Risk = probability x impact. This formula looks reasonable, but is typically specified a priori, without any justification.
A risk can be plotted on a two dimensional graph depicting impact (on the x-axis) and probability (on the y-axis). This is typically where the problems start: for most risks, neither the probability nor the impact can be accurately quantified. The standard solution is to use a qualitative scale, where instead of numbers one uses descriptive text – for example, the probability, impact and risk can take on one of three values: high, medium and low (as shown in Figure 1 below). In doing this, analysts make the implicit assumption that the categorisation provided by the qualitative assessment ranks the risks in correct quantitative order. Problem is, this isn’t true.
Figure 1: A 3x3 Risk Matrix
Let’s look at the simple case of two risks A and B ranked on a 2×2 risk matrix shown in Figure 2 below. Let’s assume that the probability and impact of each of the two risks are independent and uniformly distributed between 0 and 1. Clearly, if the two risks have the same qualitative ranking (high, say), there is no way to rank them correctly unless one has quantitative knowledge of probability and impact – which is usually not the case. In the absence of this information, there’s a 50% chance (all other factors being equal) of ranking them correctly – i.e. one is effectively “flipping a coin” to choose which one has the higher (or lower) rank. This situation highlights a shortcoming of risk matrices: poor resolution. It is not possible to rank risks that have the same qualitative ranking.
Figure 2: A 2x2 Risk Matrix
“That’s obvious,” I hear you say – and you’re right. But there’s more: if one of the ratings is medium and the other one is not (i.e. the other one is high or low), then there is a non-zero chance of making an incorrect ranking because some points in the cell with the higher qualitative rating have a lower quantitative value of risk than some points in the cell with the lower qualitative ranking. Look at that statement again: it implies that risk matrices can incorrectly assign higher qualitative rankings to quantitatively smaller risks – i.e. there is the possibility of making ranking errors. This point is seriously counter-intuitive (to me anyway) and merits a proof, which Cox provides and I discuss below. Before doing so, I should also point out that the discussion of this paragraph assumes that the probabilities and impacts of the two risks are independent and uniformly distributed. Cox also points out that the chance of making the wrong ranking can be even higher if the joint distribution of the two are correlated. In particular, if the correlation is negative (i.e. probability decreases as impact increases), a random ranking is actually better than that provided by the risk matrix. In this situation the information provided by risk matrices is “worse than useless” (a random choice is better!). Negative correlations between probability and impact are actually quite common – many situations involve a mix of high probability-low impact and low probability-high impact risks. See the paper for more on this.
Weak consistency and its implications
With the issues of poor resolution and ranking errors established, Cox asks the question: What can be salvaged? The underlying problem is that the joint distribution of probability and impact is unknown. The standard approach to improving the utility of risk matrices is to attempt to characterise this distribution. This can be done using artificial intelligence tools – and Cox provides references to papers that use some of these techniques to characterise distributions. These techniques typically need plentiful data as they attempt to infer characteristics of the joint distribution from data points. Cox, instead, proposes an approach that is based on general properties of risk matrices – i.e. an approach that prescribes a set of rules that ensure consistency. This has the advantage of being general, and not depending on the availability of data points to characterise the probability distribution.
So what might a consistency criterion look like? Cox suggests that, at the very least, a risk matrix should be able to distinguish reliably between very high and very low risks. He formalises this requirement in his definition of weak consistency, which I quote from the paper:
A risk matrix with more than one “colour” (level of risk priority) for its cells satisfies weak consistency with a quantitative risk interpretation if points in its top risk category (red) represent higher quantitative risks than points in its bottom category (green)
The notion of weak consistency formalises the intuitive expectation that a risk matrix must, at the very least, distinguish between the lowest and highest (quantitative) risks. If it can’t, it is indeed “worse than useless”. Note that weak consistency doesn’t say anything about distinguishing between medium and lowest/highest risks – merely between the lowest and highest.
Having defined weak consistency, Cox derives some of its surprising consequences, which I describe next.
Cox’s First Lemma: If a risk matrix satisfies weak consistency, then no red cell (highest risk category) can share an edge with a green cell (lowest risk category).
Proof: To see how this is plausible, consider the different ways in which a red cell can adjoin a green one. Basically there are only two ways in which this can happen, which I’ve illustrated in Figure 3. Now assume that the quantitative risk of the midpoint of the common edge is a number n (n between 0 and 1). Then if x and y and are the impact and probability, we have
xy=n or y=n/x
So, the locus of all points having the same risk (often called the iso-risk contour) as the midpoint is a rectangular hyperbola with negative slope (i.e. y decreases as x increases). The negative slope (see Figure 3) implies that the points above the iso-risk contour in the green cell have a higher quantitative risk than points below the contour in the red cell. This contradicts weak consistency. Hence – by reductio ad absurdum – it isn’t possible to have a green cell and a red cell with a common edge.
Figure 3: Figure for Lemma 1
Cox’s Second Lemma: if a risk matrix satisfies weak consistency and has at least two colours (green in lower left and red in upper right, if axes are oriented to depict increasing probability and impact), then no red cell can occur in the bottom row or left column of the matrix.
Proof: Assume it is possible to have a red cell in the bottom row or left column. Now consider an iso-risk contour for a sufficiently small risk (i.e. a contour that passes through the lower left-most green cell). By the properties of rectangular hyperbolas, this contour must pass through all cells in the bottom row and the left-most column, as shown in Figure 4. Thus, by an argument similar to the one of the previous lemma, all points below the iso-risk contour in either of the red cells have a smaller quantitative risk than point above it in the green cell. This violates weak consistency, and hence the assumption is incorrect.
Figure 4: Figure for Lemma 2
An implication that follows directly from the above lemmas is that any risk matrix that satisfies weak consistency must have at least three colours!
Surprised? I certainly was when I first read this.
Between-ness and its implications
If a risk matrix provides a qualitative representation of the actual qualitative risks, then small changes in the probability or impact should not cause discontinuous jumps in risk categorisation from lowest to highest category without going through the intermediate category. (Recall, from the previous section, that a weakly consistent matrix must have at least three colours).
This expectation is formalised in the axiom of between-ness:
A risk matrix satisfies the axiom of between-ness if every positively sloped line segment that lies in a green cell at its lower end and a red cell at its upper end must pass through at least one intermediate cell (i.e. one that is neither red nor green).
By definition, no 2×2 cell can satisfy between-ness. Further, amongst 3×3 matrices, only one colour scheme satisfies both weak consistency and between-ness. This is the matrix shown in Figure 1: green in the left and bottom columns , red in upper right-most cell and yellow in all other cells. This, to me, is a truly amazing consequence of a couple of simple, intuitive axioms.
Consistent colouring and its implications
The basic idea behind consistent colouring is that risks that have the identical quantitative values should have the same qualitative ratings. This is impossible to achieve in a discrete risk matrix because iso-risk contours cannot coincide with cell boundaries (Why? Because iso-risk contours have negative slopes whereas cell boundaries have zero or infinite slope – i.e. they are horizontal or vertical lines). So, Cox suggests the following: enforce consistent colouring for extreme categories only – red and green – allowing violations for intermediate categories. What this means is that cells that contain iso-risk contours which pass through other red cells (“red contours”) must be red and cells that contain iso-risk contours which pass through other green cells (“green contours”) must be green. Hence the following definition of consistent colouring:
- A cell is red if it contains points with quantitative risks at least as high as those in other red cells, and does not contain points with quantitative risks as small as those on any green cell.
- A cell is green if it contains points with risks at least as small as those in other green cells, and does not contain points with quantitative risks as high as those in any red cell.
- A cell has an intermediate colour only if it a) lies between a red cell and a green cell or b) it contains points with quantitative risks higher than those in some red cells and also points with quantitative risks lower than those in some green cells.
An iso-risk contour is green if it passes through one or more green cells but no red cells and a red contour is one which passes through one or more red cells but no green cells. Consistent colouring then implies that cells with red contours and no green contours are red; and cells with green contours and no red contours are green (and, obviously, cells with contours of both colours are intermediate)
Implications of the three axioms – Cox’s Risk Matrix Theorem
So, after a longish journey, we have three axioms: weak consistency, between-ness and consistent colouring. With that done, Cox rolls out his theorem – which I dub Cox’s Risk Matrix Theorem (not to be confused with Cox’s Theorem from statistics!), which can be stated as follows:
In a risk matrix satisfying weak consistency, between-ness and consistent colouring:
a) All cells in the leftmost column and in the bottom row are green.
b) All cells in the second column from the left and the second row from the bottom are non-red.
The proof is a bit long, so I’ll omit it, making a couple of plausibility arguments instead:
- The lower leftmost cell is green (by definition), and consistent colouring implies that all contours that lie below the one passing through the upper right corner of this cell must also be green because a) they pass through the lower leftmost cell which is green and b) none of the other cells they pass through are red (by Cox’s second lemma). The other cells on the lowest or leftmost edge of the matrix can only be intermediate or green. That they cannot be intermediate is a consequence of between-ness.
- That the second row and second column must be non-red is also easy to see: assume any of these cells to be red. We then have a red cell adjoining a green cell, which violates between-ness.
I’ll leave it at that, referring the interested reader to the paper for a complete proof.
Cox’s theorem has an immediate corollary which is particularly interesting for project managers who use 3×3 and 4×4 risk matrices:
A tricoloured 3×3 or 4×4 matrix that satisfies weak consistency, between-ness and consistent colouring can have only the following (single!) colour scheme:
a) Leftmost column and bottom row coloured green.
b) Top right cell (for 3×3) or four top right cells (for 4×4) coloured red.
c) All other cells coloured yellow.
Proof: Cox’s theorem implies that the leftmost column and bottom row are green. The top right cell must be red (since it is a tricoloured matrix). Consistent colouring implies that the two cells adjoining this cell (in a 4×4 matrix) and the one diagonally adjacent must also be red (this cannot be so for a 3×3 matrix because these cells would adjoin a green cell which violates Cox’s first lemma). All other cells must be yellow by between-ness.
This result is quite amazing. From three very intuitive axioms Cox derives essentially the only possible colouring scheme for 3×3 and 4×4 risk matrices.
Conclusion
This brings me to the end of this post on the Cox’s axiomatic approach to building logically consistent risk matrices. I highly recommend reading the original paper for more. Although it presents some fairly involved arguments, it is very well written. The arguments are presented with clarity and logical surefootedness, and the assumptions underlying each argument are clearly laid out. The three principles (or axioms) proposed are intuitively appealing – even obvious – but their consequences are quite unexpected (witness the unique colouring scheme for 3×3 and 4×4 matrices). Further, the arguments leading up to the lemmas and theorems bring up points that are worth bearing in mind when using risk matrices in practical situations.
In closing I should mention that the paper also discusses some other limitations of risk matrices that flow from these principles: in particular, spurious risk resolution and inappropriate resource allocation based on qualitative risk categorisation. For reasons of space, and the very high likelihood that I’ve already tested my readers’ patience to near (if not beyond) breaking point, I’ll defer a discussion of these to a future post.
Note added on 20 December, 2009:
See this post for a visual representation of the above discussion of Cox’s risk matrix theorem and the comments that follow.
Managing participant motivation in knowledge management projects
Introduction
One consequence of the increasing awareness of knowledge as an organisational asset is that many organisations have launched projects aimed at managing knowledge. Unfortunately, a large number of these efforts focus entirely on technical solutions, neglecting the need for employee participation. The latter is important; as stated in this paper, published a decade ago, “Knowledge transfer is about connection not collection, and that connection ultimately depends on choice made by individuals…” This suggests that participant motivation is a key success factor for knowledge management initiatives. A recent paper entitled, Considering Participant Motivation in Knowledge Management Projects, by Allen Whittom and Marie-Christine Roy looks at theories of motivation from the context of knowledge management projects. This post is a summary and annotated review of the paper.
Many researchers claim that the failure rate of knowledge management projects is high, but there seems to be some confusion as to just how high the figure is (see this paper, for example). In the introduction to their paper, Whittom and Roy claim that the failure rate may be higher than 80% – but they offer no proof. Still, with many independent researchers quoting figures ranging from 50 to 80%, one can take it as established that it is a matter that merits investigation. Accordingly, many researchers have looked at causes of failure of knowledge management projects (see this paper or this one). Some specifically identify lack of participant motivation as a cause of failure (see this paper). Whittom and Roy claim that despite the work done thus far, knowledge management research does not provide any suggestions as to how motivation is to be managed in such projects. Their aim, therefore, is to:
- Present concepts from theories of motivation that are relevant to knowledge management projects.
- Propose ways in which project managers can foster participant motivation in a way that is consistent with business objectives.
These points are covered in the next two sections. The final section presents some concluding remarks.
Theoretical Overview
Motivation and Knowledge Transfer
The authors define motivation as the underlying reason(s) for a person’s actions. Motivation is usually classified as extrinsic or intrinsic depending on whether its source is external or internal to the individual. People who are extrinsically motivated are driven by rewards such as bonuses or promotions. Typically such individuals work for rewards. On the other hand intrinsically motivated individuals are self-driven, and need little supervision. Their enthusiasm, however, depends on whether or not their personal goals are congruent with the task at hand. This is important: their aims and objectives may not always be aligned with business goals. Further, intrinsically motivated individuals perform creative or complex tasks better than others, but this type of motivation varies greatly from one person to another and cannot be controlled by management. See my post on motivation in project management for a comprehensive discussion on extrinsic and intrinsic motivation.
The authors then discuss the link between motivation and the willingness to share knowledge. Knowledge falls into two categories: tacit and explicit. Tacit knowledge is hard to codify and communicate (e.g. a skill, such as riding a bicycle) whereas explicit knowledge can be formalised and transmitted (e.g. how to open a bank account). Tacit knowledge is in “people’s heads” and is consequently harder to capture. More often than not, though, it turns out to be more valuable than explicit knowledge. In their paper entitled, Motivation, Knowledge Transfer and Organisational Forms, Osterloh and Frey state that, “…Intrinsic motivation is crucial when tacit knowledge in and between teams must be transferred…” Following this work, Gartner researchers Morello and Caldwell proposed a model in which intrinsic motivation drives the creation and sharing of tacit knowledge which in turn drives the dissemination and use of tacit knowledge in the organisation (I couldn’t find a publicly available copy of their work – but there is an illustration of the model in Figure 1 Whittom and Roy’s paper).
The message from motivation research is clear: intrinsic motivation is critical to the success of knowledge management projects.
Rewards and Recognition
Rewards and recognition are “levers of motivation”: they can be used to enhance and direct employee motivation towards achieving organisational goals. Reward systems are aimed at aligning individual efforts with organisational objectives. Recognition systems, on the other hand, are designed to express public appreciation for high standards of achievement or competence. These may be set according to criteria that diverge from preset objectives (as an example – a public thanks for a job well done can be made irrespective of whether the job is in line with company objectives)
Rewards can be extrinsic (not related to the task) or intrinsic (related to the task) and material or non-material. Extrinsic rewards are typically material – i.e. they involve giving the recipient something tangible. Financial incentives are the most common form of extrinsic rewards because they are easily administered through the pay system. Extrinsic rewards can also be non-financial (gift certificates or a meal at a nice restaurant, for example). For the same investment, non-financial rewards are found to have a more lasting effect than financial ones. This makes sense: people are more likely to remember a memorable meal than a few hundred dollar raise; the latter is sometimes forgotten as soon as it comes into effect. A downside of financial rewards is that they are easily forgotten and may actually decrease intrinsic motivation (see this paper by David Beswick). Another is that they may encourage sub-standard work, particularly in cases where benchmarks are based on volume rather than quality of output.
Extrinsic rewards can also be non-material – promotions and training opportunities, for example (see this paper by Wolfgang Semar for more on non-material, extrinsic rewards).
Intrinsic rewards generally pertain to the satisfaction derived from performing a task. The moral satisfaction arising from a job done well is also a form of intrinsic reward. It should be clear that these rewards work only for intrinsically motivated individuals. Intrinsic rewards are invariably non-material and they cannot be controlled by management. However, awareness of factors influencing intrinsic motivation can help managers create the right environment for intrinsically motivated individuals. Kenneth Thomas, in his book entitled, Intrinsic Motivation at Work – Building Energy and Commitment, identifies four psychological factors that can influence intrinsic motivation. They are:
- Feelings of accomplishment: These can be enhanced by devising interesting work tasks and aligning them with employee interests.
- Feelings of autonomy: These can be enhanced by empowering employees with responsibility and authority to do their work.
- Feelings of competence: These can be enhanced by offering employees opportunities to demonstrate and enhance their expertise.
- Feelings of progress: These can be enhanced by fostering a collaborative atmosphere in which project successes are celebrated.
These factors are (to an extent) under management control. If nothing else, it is worth being aware of them so that one can avoid doing things that might reduce intrinsic motivation.
Motivation crowding and psychological contracts
The authors then examine the effects of rewards on intrinsic motivation in the context of knowledge management projects (Recall that intrinsic motivation was seen to be a key success factor in knowledge management projects). They use motivation crowding theory to frame their discussion. Crowding theory suggests that intrinsic motivation can be enhanced (“crowded-in”) or undermined (or “crowded-out”) by external rewards.
To understand motivation crowding, one has to look at how extrinsic (or external) rewards work. Basically there are two ways in which an extrinsic reward can be perceived. To quote from the paper,
External interventions, such as rewards, may influence this perception either through information or control. If people see a reward as being related to their competence (information), intrinsic motivation for the task will be encouraged or maintained. On the other hand, if they see a reward as a way to control their performance or autonomy, intrinsic motivation would be decreased.
Extrinsic rewards can have a positive or negative effect on information and control. This is best understood through an example: consider a company that announces cash incentives for the top three contributors to a knowledge database. This reward has a positive control aspect (i.e. encourages participation) but a negative information aspect (i.e. no check on quality of contributions). Consequently, the reward encourages high volume of contributions with no regard to quality. This situation typically undermines or “crowds-out” intrinsic motivation. Note that motivation “crowding out” is sometimes referred to as motivation eviction in the literature.
Crowding-out is also seen in recurring tasks. For example, if a monetary incentive is offered for a task, there will be an expectation that the incentive be offered the next time around. On the other hand, non-monetary interventions such as increased employee involvement and autonomy in project decision making can “crowd-in” or enhance intrinsic motivation.
These effects are intuitively quite obvious, but it’s interesting to see them from a social science / economics point of view. If you’d like to find out more, I highly recommend the paper, Motivation crowding theory: A survey of empirical evidence, by Bruno Frey and Reto Jegen.
The take home lesson from the above is that intrinsic motivation can sometimes be negatively affected by external rewards. Manager, beware.
Whittom and Roy also discuss the notion of psychological contracts between the employer and employee. These contracts, distinct from formal employment contracts, refer to the unstated (but implied) informal, mutual obligations pertaining to respect, autonomy, work ethic, fairness etc. An employee’s intrinsic motivation can be greatly reduced if he or she perceives that the contract has been breached. For example, if an employee’s regarding improvements to a knowledge database are ignored, she might feel undervalued. In her eyes, management (and hence the organisation) has lost credibility, and the psychological contract has been violated. In psychological contract theory, personal relationships are seen to be an important driver of intrinsic motivation: people are more likely to enjoy working in teams in which they have good relations with team members.
Discussion
Practices to foster intrinsic motivation
One conclusion from the aforementioned theories is that intrinsic motivation is essential for the transfer of tacit knowledge. Accordingly, the authors suggest the following practices to maintain and enhance intrinsic motivation of employees involved in knowledge management projects:
- Avoid the use of monetary rewards. Instead, use non-monetary rewards that recognize competence. Monetary rewards may also encourage the transfer of unimportant knowledge.
- Involve employees in the formulation of project objectives.
- Encourage team work and team bonding. A good team dynamic encourages the sharing of tacit knowledge. The technique of dialogue mapping facilitates the sharing and capture of knowledge in a team environment.
- Emphasise how the employee might benefit from the project – this is the old WIIFM factor. This needs to be done in a way that shows how the benefit is integrated into the organisation’s culture – i.e. the benefit must be a realistic and believable one, else the employee will see right through it.
- Good communication between management and employees. This one is a “usual suspect” that comes up in virtually all best practice recommendations. Unfortunately it is seldom done right.
Contextual recommendations based on knowledge and motivation types
Theories of motivation indicate that, as far as motivation for knowledge sharing is concerned, one size does not fit all. The particular strategy used depends on the nature of the knowledge that is being captured (tacit or explicit), participants’ motivational drivers (intrinsic or extrinsic) and organizational resources. Based on this, the authors discuss the following contexts
- Tacit knowledge management / intrinsic motivation: This is an ideal situation. Here the manager’s role is to support participants in achieving project objectives rather than to influence their behaviour through rewards. Extrinsic rewards should be avoided because participants are intrinsically motivated.
- Tacit knowledge management / extrinsic motivation: From the preceding discussion of motivation theories, it is clear that this is not a good situation. However, all is not lost. A manager can develop knowledge management strategies based on structured training, discussion groups etc. to help codify and transfer tacit knowledge. These strategies should highlight the project benefits (for the employee and the organisation). Further, extrinsic rewards can be offered, but their “crowding-out” effect over time should be kept in mind.
- Explicit knowledge / intrinsic motivation: Here the knowledge management aspect is easier because the knowledge is explicit. Typically, once the objectives are identified, it is clear how knowledge should be captured and organized. Obviously, structured training and tools such as Wikis and databases can help facilitate knowledge transfer. Further, these will be more effective than case (2) above, because the participants are intrinsically motivated.. Recommendations, as far as rewards are concerned are the same as in the first case.
- Explicit knowledge / extrinsic motivation: For knowledge management the same considerations apply as in case (3). However, these strategies will be less effective because employees are extrinsically motivated. For rewards management, the considerations of case (2) apply.
As discussed above, the motivation strategy should be determined by whether the team members are intrinsically or extrinsically motivated. Unfortunately, though, the strategy often dictated by the culture of the organization – the manager may have little say in determining it. The authors do not discuss what a manager might do in such a situation.
Conclusion
The paper presents no new data or analysis of existing data. As such it must be evaluated on the basis of new concepts and theoretical constructs that it presents. From this perspective there’s little that’s new in this paper. That said, project managers leading knowledge management projects might find the paper a worthwhile read because of its coverage of motivation theories (crowding theory and psychological contracts, in particular).
Let me end with an extrapolation of the above discussion to software projects. The holy grail of knowledge management initiatives is to capture tacit knowledge. By definition, this knowledge is difficult to codify. One sees something similar in requirements gathering for application software. The analyst needs to capture all the explicit and tacit process knowledge that’s in users’ heads. The former is easy to capture; the latter isn’t. As a result requirements usually do not capture tacit process knowledge. This is one aspect of what Brooks referred to as the essential problem of software design – figuring out what the software really needs to do (see this post for more on Brooks’ argument). Well designed software embodies both kinds of knowledge, so software projects are knowledge management projects in a sense. As far as motivation is concerned, therefore, the theories and conclusions sketched above should apply to software projects. An intrinsically motivated development team will improve the chances of success greatly; a trite statement perhaps, but one that may resonate with those who have had the privilege of working with such teams.
The legacy of legacy software
Introduction
On a recent ramble through Google Scholar, I stumbled on a fascinating paper by Michael Mahoney entitled, What Makes the History of Software Hard. History can offer interesting perspectives on the practice of a profession. So it is with this paper. In this post I review the paper, with an emphasis on the insights it provides into the practice of software development.
Mahoney’s thesis is that,
The history of software is the history of how various communities of practitioners have put their portion of the world into the computer. That has meant translating their experience and understanding of the world into computational models, which in turn has meant creating new ways of thinking about the world computationally and devising new tools for expressing that thinking in the form of working programs….
In other words, software– particularly application software – embodies real world practices. As a consequence,
…the models and tools that constitute software reflect the histories of the communities that created them and cannot be understood without knowledge of those histories, which extend beyond computers and computing to encompass the full range of human activities…
This, according Mahoney, is what makes the history of software hard.
The standard history of computing
The standard (textbook) history of computing is hardware-focused: a history of computers rather than computing. The textbook version follows a familiar tune starting with the abacus and working its way up via analog computers, ENIAC, mainframes, micros, PCs and so forth. Further, the standard narrative suggests that each of these were invented in order to satisfy a pre-existing demand, which makes their appearance almost inevitable. In Mahoney’s words,
…Just as it places all earlier calculating devices on one or more lines leading toward the electronic digital computer, as if they were somehow all headed in its direction, so too it pulls together the various contexts in which the devices were built, as if they constituted a growing demand for the invention of the computer and as if its appearance was a response to that demand.
Mahoney says that this is misleading for because,
…If people have been waiting for the computer to appears as the desired solution to their problems, it is not surprising that they then make use of it when it appears, or indeed that they know how to use it…
Further, it
…sets up a narrative of revolutionary impact, in which the computer is brought to bear on one area after another, in each case with radically transformative effect….”
The second point – revolutionary impact – is interesting because we still suffer its fallout: just about every issue of any trade journal has an article hyping the Next Big Computing Revolution. It seems that their writers are simply taking their cues from history. Mahoney puts it very well,
One can hardly pick up a journal in computing today without encountering some sort of revolution in the making, usually proclaimed by someone with something to sell. Critical readers recognise most of it as hype based on future promise than present performance…
The problem with revolutions, as Mahoney notes, is that they attempt to erase (or rewrite) history, ignoring the real continuities and connections between present and the past,
Nothing is in fact unprecedented, if only because we use precedents tot recognise, accommodate and shape the new…
CIOs and other decision makers, take note!
But what about software?
The standard history of computing doesn’t say much about software,
To the extent that the standard narrative covers software, the story follows the generations of machines, with an emphasis on systems software, beginning with programming languages and touching—in most cases, just touching—on operating systems, at least up to the appearance of time-sharing. With a nod toward Unix in the 1970s, the story moves quickly to personal computing software and the story of Microsoft, seldom probing deep enough to reveal the roots of that software in the earlier period.
As far as applications software is concerned –whether in construction, airline ticketing or retail – the only accounts that exist are those of pioneering systems such as the Sabre reservation system. Typically these efforts focus on the system being built, excluding any context and connection to the past. There are some good “pioneer style” histories: an example is Scott Rosenberg’s book Dreaming in Code – an account of the Chandler software project. But these are exceptions rather than the rule.
In the revolutionary model, people react to computers. In reality, though, it’s the opposite: people figure out ways to use computers in their areas of expertise. They design and implement programs to make computers do useful things. In doing so, they make choices:
Hence, the history of computing, especially of software, should strive to preserve human agency by structuring its narratives around people facing choices and making decisions instead of around impersonal forces pushing people in a predetermined direction. Both the choices and the decisions are constrained by the limits and possibilities of the state of the art at the time, and the state of the art embodies its history to that point.
The early machines of the 1940s and 50s were almost solely dedicated to numerical computations in the mathematical and physical sciences. Thereafter, as computing became more “mainstream” other communities of practitioners started to look at how they might use computers:
These different groups saw different possibilities in the computer, and they had different experiences as they sought to realize those possibilities, often translating those experiences into demands on the computing community, which itself was only taking shape at the time.
But these different communities have their own histories and ways of doing things – i.e. their own, unique worlds. To create software that models these worlds, the worlds have to be translated into terms the computer can “understand” and work with. This translation is the process of software design. The software models thus created embody practices that have evolved over time. Hence, the models also reflect the histories of the communities that create them.
Models are imperfect
There is a gap between models and reality, though. As Mahoney states,
…Programming is where enthusiasm meets reality. The enduring experience of the communities of computing has been the huge gap between what we can imagine computers doing and what we can actually make them do.
This lead to the notion of a “software crisis: and calls to reform the process of software development, which in turn gave rise to the discipline of software engineering. Many improvements resulted: better tools, more effective project management, high-level languages etc. But all these, as Brooks pointed out in his classic paper, addressed issues of implementation (writing code) not those of design (translating reality into computable representations). As Mahoney state,
…putting a portion of the world into the computer means designing an operative representation of that portion of the world that captures what we take to be its essential features. This has proved, as I say, no easy task; on the contrary it has proved difficult, frustrating and in some cases disastrous.
The problem facing the software historian is that he or she has to uncover the problem context and reality as perceived by the software designer, and thus reach an understanding of the design choices made. This is hard to do because it is implicit in the software artefact that the historian studies. Documentation is rarely any help here because,
…what programs do and what the documentation says they do are not always the same thing. Here, in a very real sense, the historian inherits the problems of software maintenance: the farther the program lies from its creators, the more difficult it is to discern its architecture and the design decisions that inform it.
There are two problems here:
- That software embodies a model of some aspect of reality.
- The only explanation of the model is the software itself.
As Mahoney puts it,
Legacy code is not just old code, but rather a continuing enactment, an operative representation, of the domain knowledge embodied in it. That may explain the difficulties software engineers have experienced in upgrading and replacing older systems.
Most software professionals will recognise the truth of this statement.
The legacy of legacy code
The problem is that new systems promise much, but are expensive and pose too many risks. As always continuity must be maintained, but this is nigh impossible because no one quite understands the legacy bequeathed by legacy code: what it does, how it does it and why it was designed so. So, customers play it safe and legacy code lives on. Despite all the advances in software engineering, software migrations and upgrades remain fraught with problems.
Mahoney concludes with the following play on the word “legacy”,
This situation (the gap between the old and the new) should be of common interest to computer people and to historians. Historians will want to know how it developed over several decades and why software systems have not kept pace with advances in hardware. That is, historians are interested in the legacy. Even as computer scientists wrestle with a solution to the problem the legacy poses, they must learn to live with it. It is part of their history, and the better they understand it, the better they will be able to move on from it.
This last point should be of interest to those running software development projects in corporate IT environments (and to a lesser extent those developing commercial software). An often unstated (but implicit) requirement is that the delivered software must maintain continuity between the past and present. This is true even for systems that claim to represent a clean break from the past; one never has the luxury of a completely blank slate, there are always arbitrary constraints placed by legacy systems. As Fred Brooks mentions in his classic article No Silver Bullet,
…In many cases, the software must conform because it is the most recent arrival on the scene. In others, it must conform because it is perceived as the most conformable. But in all cases, much complexity comes from conformation to other interfaces…
So, the legacy of legacy software is to add complexity to projects intended to replace it. Mahoney’s concluding line is therefore just as valid for project managers and software designers as it is for historians and computer scientists: project managers and software designers must learn to live with and understand this complexity before they can move on from it.
Eight to Late 