There’s trouble ahead: early warning signs of project failure
Introduction
I’ve written a number of articles on project failure, covering topics ranging from definitions of success to the role of biases in project failure. As interesting as these issues are, they are somewhat removed from the day-to-day concerns of a project manager who is more interested in avoiding failure than defining or analyzing it. In a paper entitled, Early warning signs of IT project failure: the dominant dozen, Leon Kappelman et. al. outline the top twelve risks associated with IT project failures. This post summarises the paper and lists the top twelve signs of impending trouble on projects.
Background and research methodology
The authors focus on early warning signs – i.e. those that occur within the initial 20% of the planned schedule. Further, to ensure comprehensive coverage of risks, their conclusions are based on inputs from academic and industry journals as well as from experienced IT project managers. The paper provides a detailed explanation of their research methodology, which I’ll quote directly from the paper:
The research team first searched the literature extensively to develop a preliminary list of early warning signs (EWSs). The two authors experienced in IT project management then added several EWSs based on their personal experience. Next, 19 IT project management experts were asked to assess the list. On the basis of their feedback, we added new items and modified others to develop a list of 53 EWSs. Finally, the research team invited 138 experienced IT project managers (including the original 19 experts) to participate in rating the 53 EWSs using a scale from 1 (extremely unimportant) to 7 (extremely important). Fifty-five (55) of these managers completed the survey, yielding a response rate of nearly 40 percent. The respondents had an average of more than 15 years of IT project management experience. The budgets of the largest IT projects they managed ranged from 3 million to 7 billion dollars. About 30 percent held the title of program or project manager and nearly 20 percent had consultant titles. Director or program analyst titles accounted for about 15 percent each, 10 percent were vice presidents, and the rest held titles such as CEO, CIO, chief scientist, chief technologist, or partner.
Although the list and the rankings were based on the subjective opinions of experts, the large number of participants ensures a degree of consensus regarding the most important factors.
The troublesome twelve
After ranking the fifty odd risks, the authors focused on those that had scores above 6 (out of a maximum possible of 7 as discussed above). There were 17 risks that satisfied this (somewhat arbitrary) criterion. Some of these were similar, so they could be combined. For example, the four risks:
- No documented milestone deliverables and due dates.
- No project status progress process
- Schedule deadline not reconciled to the project schedule
- Early project delays are ignored — no revision to the overall project schedule
were combined into: ineffective schedule planning and/or management.
This process of combining the top 17 items resulted in twelve risks, half of which turned out to be people-related and the other half process-related. I discuss each of the risks in detail below.
People-related early warning signs
1. Lack of top management support: This was the number one risk out of the fifty three that the authors listed. This isn’t surprising – a project that lacks executive support is unlikely to get the financial, material or human resources necessary to make it happen.
2. Ineffective project manager: Project managers who lack the communication and managerial skills needed to move the project ahead pose a serious risk to projects. The authors point out that this is a common risk on IT projects because project managers are often technical folks who have been promoted to managers. As such they may lack the interest, aptitude and/or skills to manage projects. Interestingly, the authors do not comment on the converse problem – whether the project manager’s lack of technical/domain knowledge contributes to project failure.
3. No stakeholder involvement and/or participation: A large number of projects proceed with minimal involvement of key stakeholders. Such folks often lose interest in projects when more immediate matters consume their attention. In such situations a project manager may find it hard to get the resources he or she needs to get the project done. Stakeholder or sponsor apathy is an obvious warning sign that a project is headed for trouble.
4. Uncommitted project team: The commitment (preferably, full-time) of a team is essential for the success of a project. Management needs to ensure that team members are given the time (and incentives) to work on the project. A point that is often left unconsidered is the intrinsic motivation of the team – see this post for a detailed discussion of motivation in project management.
5. Lack of technical knowledge/skills: Project teams need to have the technical skills and knowledge that is relevant to the project. Managers sometimes wrongly assume that project staff can pick up the required skills whilst working on a project. Another common management misconception is that project personnel can master new technologies solely by attending training courses. Getting contractors to do the work is one solution to the problem. However, the best option is to give the team enough time to get familiar with the technology prior to the project or, failing this, to switch to a technology that the team is familiar with.
6. Subject matter experts are not available: It is often assumed that subject matter experts can provide adequate inputs into projects whilst doing their regular jobs. This seldom works – when there’s a choice between the project and their jobs, the latter always wins. Project sponsors need to ensure that subject matter experts are freed up to work on the project.
Process-related early warning signs
1. Unclear scope: The authors label this one as “Lack of documented requirements and/or success criteria.” However I think it is better described by the phrase I’ve used. All project management methodologies emphasise the importance of clear, well-documented requirements and success criteria – and with good reason too. Lack of clarity regarding project scope means that no one knows where the project is headed – a sure sign of trouble ahead.
2. No change control process: As the cliché reminds us, change is the only constant in business environments. It is therefore inevitable that project scope will change. Changes to scope –however minor they may seem- need to be assessed for their impact on the project. The effect of several small (unanalyzed) scope changes on the project schedule should not be underestimated! Many project managers have a hard time pushing back on scope changes foisted on them by senior executives. Hence it is important that the change control process applies across the board – to everyone regardless of their authority.
3. Ineffective scheduling and schedule management: Many schedules are built on little more than guesswork and an unhealthy dose of optimism, often because they are drawn up without input from the folks who’ll actually do the work (see my article on estimation errors for more on this). Schedules need to be rooted in reality. For this to happen, they must be based on reliable estimates, preferably from those responsible for creating the deliverables. Once the schedule is created, it is the project manager’s responsibility to update it continually, reflecting all the detours and road-bumps that have occurred along the way. A common failing is that time overruns are not properly recorded, leading to a false illusion of progress.
4. Communication breakdown: Project communication is the art of getting people on the same page when they are reading different books. In my post on obstacles to project communication, I have discussed some generic difficulties posed by differences in stakeholder backgrounds and world-views. One of the key responsibilities of a project manager is to ensure that everyone on the project has a shared understanding of the project goals and shared commitment to achieving them. This is as true in the middle or the end of a project as it is at the start.
5. Resources assigned to another project: In my experience resources are rarely reassigned wholesale to other projects. What usualy happens is that they are reassigned on a part time basis, as in “we’ll take 20 % of Matt’s time and 10% of Nick’s time.” The problem with this is that Matt and Nick will end up spending most of their time on the other project, leaving the one on hand bereft.
6. No business case: A not uncommon refrain in corporate hallways is, “Why are we doing this project?” No project should be given the go-ahead without a well-articulated business case. Further still, since an understanding reason(s) for doing the project are central to its success, these should be made available to every stakeholder: a shared understanding of the goals of the project is a prerequisite to a shared understanding of the rationale behind it.
I’m sure there aren’t any surprises in this list – most project managers would agree that these are indeed common (and often ignored) early warning signs of failure. However, I suspect that there will be substantial differences of opinion regarding their ranking. Wisely, the authors have refrained from attempting to rank the risks – the list is not in order of importance.
Conclusion
Good projects managers anticipate potential problems and take action to avoid them. Although the risks listed above are indeed obvious , they are often ignored. Affected projects then limp on to oblivion because those responsible failed to react to portents of trouble. Granted, it can be hard to see problems from within the system, particularly when the system is a high-pressure project. That’s where such lists are useful: they can warn the project manager of potential trouble ahead.
More than just talk: rational dialogue in project environments
Prologue
The meeting had been going on for a while but it was going nowhere. Finally John came out and said it, “There is no way I can do this in 2 days,” he said. “It will take me at least a week.”
There it was, the point of contention between the developer and his manager. Now that it was out in the open, the real work of meeting could begin; the two could start talking about a realistic delivery date.
The manager, let’s call him Jack, was not pleased, “Don’t tell me a simple two-page web app – which you have done several times before I should add – will take you a week to do. ”
“OK, let me walk you through the details,” said John.
….and so it went for another half hour or so, Jack and John arguing about what would be a reasonable timeframe for completing the requested work.
Dialogue, rationality and action
Most developers, designers and indeed most “doers” on project teams– would have had several conversations similar to the one above. These folks spend a fair bit of time discussing matters relating to the projects they work on. In such discussions, the aim is to come to a shared understanding of the issue under discussion and a shared commitment on future action. In the remainder of this post I’ll take a look at project discussions from a somewhat philosophical perspective, with a view to understanding some of the obstacles to open dialogue and how they can be addressed.
When we participate in discussions we want our views to be taken seriously. Consequently, we present our views through statements that we hope others will see as being rational – i.e. based on sound premises and logical thought. One presumes that John – when he made his claim about the delivery date being unrealistic – was willing to present arguments that would convince Jack that this was indeed so. The point is that John is judged (by Jack and others in the meeting) based on the validity of the statements he (John) makes. When Jack’s validity claims are contested, debate ensues with the aim of getting to some kind of agreement.
The philosophy underlying such a process of discourse (which is simply another word for “debate” or “dialogue”) is described in the theory of communicative rationality proposed by the German philosopher Jurgen Habermas. The basic premise of communicative rationality is that rationality (or reason) is tied to social interactions and dialogue. In other words, the exercise of reason can occur only through dialogue. Such communication, or mutual deliberation, ought to result in a general agreement about the issues under discussion. Only once such agreement is achieved can there be a consensus on actions that need to be taken. Habermas refers to the latter as communicative action, i.e. action resulting from collective deliberation.
[Note: Just to be clear: I have not read Habermas’ books, so my discussion is based entirely on secondary sources: papers by authors who have studied Habermas in detail. Incidentally, the Wikipedia article on the topic is quite good, and well worth a read.]
Validity Claims
Since the objective in project discussions is to achieve shared understanding of issues and shared commitment on future action, one could say that such discussions are aimed at achieving communicative action. The medium through which mutual understanding is achieved is speech – i.e. through statements that a speaker makes based on his or her perceptions of reality. Others involved in the dialogue do the same, conveying their perceptions (which may or may not match the speaker’s).
Now, statements made in discussions have implicit or explicit validity claims – i.e. they express a speaker’s belief that something is true or valid, at least in the context of the dialogue. Participants who disagree with a speaker are essentially contesting claims. According to the theory of communicative action, every utterance makes the following validity claims:
- It makes a claim about objective (or external) reality. John’s statement about the deadline being impossible refers to the timing of an objective event – the delivery of working software. Habermas refers to this as the truth claim.
- It says something about social reality – that is, it expresses something about the relationship between the speaker and listener(s). The relationship is typically defined by social or workplace norms, for example – the relationship between a manager and employee as in the case of John and Jack. John’s statement is an expression of disagreement with his manager. Of course, he believes his position is justified – that it ought to take about a week to deliver the software. Habermas refers to this as the rightness claim.
- It expresses something about subjective reality – that is, the speaker’s personal viewpoint. John believes, based on his experience, intutition etc., that the deadline is impossible. For communication to happen, Jack must work on the assumption that John is being honest – i.e. that John truly believes the deadline is impossible, even though Jack may not agree. Habermas refers to this as the truthfulness claim.
The validity claims and their relation to rationality are nicely summed up in the Wikipedia article on communicative rationality, and I quote:
By earnestly offering a speech act to another in communication, a speaker claims not only that what they say is true but also that it is normatively right and honest . Moreover, the speaker implicitly offers to justify these claims if challenged and justify them with reasons. Thus, if a speaker, when challenged, can offer no acceptable reasons for the normative framework they implied through the offering of a given speech act, that speech act would be unacceptable because it is irrational.
When John says that the task is going to take him a week, he implies that he can justify the statement (if required) in three ways: it will take him a week (objective), that it ought to take him a week (normative – based on rightness) and that he truly believes it will take him a week (subjective).
In all dialogues validity claims are implied, but rarely tested; we usually take what people say at face value, we don’t ask them to justify their claims. Nevertheless, it is assumed that they can offer justifications should we ask them to. Naturally, we will do so only when we have reason to doubt the validity of what they say. It is at that point that discourse begins. As Wener Ulrich puts it in this paper:
In everyday communication, the validity basis of speech is often treated as unproblematic. The purpose consists in exchanging information rather than in examining validity claims. None of the three validity claims is then made an explicit subject of discussion. It is sufficient for the partners to assume (or anticipate, as Habermas likes to say) that speakers are prepared to substantiate their claims if asked to do so, and that it is at all times possible for the participants to switch to a different mode of communication in which one or several validity claims are actually tested. Only when validity claims do indeed become problematic, as one of the participants feels compelled to dispute either the speaker’s sincerity or the empirical and/or normative content of his statements, ordinary communication breaks down and discourse begins.
Progress in project discussions actually depends on such breakdown in “ordinary communication” – good project decisions emerge from open deliberation about the pros and cons of competing approaches. Only once this is done can one move to action.
Conditions for ideal discourse
All this sounds somewhat idealistic, and it is. Habermas noted five prerequisites for open debate. They are:
- Inclusion: all affected parties should be included in the dialogue.
- Autonomy: all participants should be able to present and criticise validity claims independently.
- Empathy: participants must be willing to listen to and understand claims made by others.
- Power neutrality: power differences (levels of authority) between participants should not affect the discussion.
- Transparency: participants must not indulge in strategic actions (i.e. lying!).
In this paper Bent Flyvbjerg adds a sixth point: that the group should be able to take as long as it needs to achieve consensus – Flyvbjerg calls this the requirement of unlimited time.
From this list it is clear that open discourse (or communicative rationality) is an ideal that is difficult to achieve in practice. Nevertheless, because it is always possible to improve the quality of dialogue on projects, it behooves us as project professionals to strive towards the ideal. In the next section I’ll look at one practical way to do this.
Boundary judgements
Most times in discussions we jump straight to the point, without bothering to explain the assumptions that underpin our statements. By glossing over assumptions, however, we leave ourselves open to being misunderstood because others have no means to assess the validity of our statements. Consequently it becomes difficult for them to empathise with us. For example, when John says that it is impossible to finish the work in less than a week, he ought to support his claim by stating the assumptions he makes and how these bear on his argument. He may be assuming that he has to do the work in addition to all the other stuff he has on his plate. On the other hand, he may be assuming too much because his manager may be willing to reassign the other stuff to someone else. Unless this assumption is brought out in the open, the two will continue to argue without reaching agreement.
Werner Ulrich pointed out that the issue of tacit assumptions and unstated frameworks is essentially one of defining the boundaries within which one’s claims hold. He coined the term boundary judgement to describe facts and norms that a speaker deems relevant to his or her statements. A boundary judgement determines the context within which a statement holds and also determines the range of validity of the statement. For example, John is talking about the deadline being impossible in the context of his current work situation; if the situation changed, so might his estimate. Ulrich invented the notion of boundary critique to address this point. In essence, boundary critique is a way to uncover boundary judgements by asking the right questions. According to Ulrich, such boundary questions probe the assumptions made by various stakeholders. He classifies boundary questions into four categories. These are:
- Motivation: this includes questions such as:
- Why are we doing this project?
- Who are we doing it for?
- How will we measure the benefits of the project once it is done?
- Power: this includes questions such as:
- Who is the key decision-maker regarding scope?
- What resources are controlled by the decision-maker?
- What are the resources that cannot be controlled by the decision-maker (i.e. what are the relevant environmental factors)?
- Knowledge: This includes:
- What knowledge is needed to do this work?
- Who (i.e which professionals) have this knowledge?
- What are the key success factors – e.g. stakeholder consensus, management support, technical soundness etc?
- Legitimation: This includes:
- Who are the stakeholders (including those that are indirectly affected by the project)?
- How do we ensure that the interests of all stakeholders are taken into account?
- How can conflicting views of project objectives be reconciled?
The questions above are drawn from a paper by Ulrich. I have paraphrased them in a way that makes sense in project environments.
Many of these questions are difficult to address openly, especially those relating to power and legitimation. Answers to these often bump up against organisational politics or power. The point, however, is that once these questions are asked, such constraints become evident to all. Only after this happens can discourse proceed in the full knowledge of what is possible and what isn’t.
Before closing this section I’ll note that there are other techniques that do essentially the same thing1, but I won’t discuss them here as I’ve already exceeded a reasonable word count.
Conclusion
Someone recently mentioned to me that the problem in project meetings (and indeed any conversation) is that participants see their own positions as being rational, even when they are not. Consequently, they stick to their views, even when faced with evidence to the contrary. According to the theory of communicative rationality, however, such folks aren’t being rational because they do not subject their positions and views to “trial by argumentation”. Rationality lies in dialogue, not in individual statements or positions. A productive discussion is one in which validity claims are continually challenged until they converge on an optimal decision. The best (or most rational) position is one that emerges from such collective deliberation.
In closing, a caveat is in order – a complete discussion of dialogue in projects (or organisations) would take an entire book and more. My discussion here has merely highlighted a few issues (and a technique) that I daresay are rarely touched upon in management texts or courses. There are many more tools and techniques that can help improve the quality of discourse within organisations. Paul Culmsee and I discuss some of these in our book, The Heretic’s Guide to Best Practices.
1 Those familiar with soft systems methodology (SSM) will recognise the parallels between Ulrich’s approach and the CATWOE checklist of SSM. CATWOE is essentially a means of exposing boundary judgements.
Eight to Late 