Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘IT Management’ Category

There’s trouble ahead: early warning signs of project failure

leave a comment »

Introduction

I’ve written a number of articles on project failure, covering topics ranging from  definitions of success to the role of biases in project failure.  As interesting as these issues are, they are somewhat removed from the day-to-day concerns of a  project manager who is  more interested in avoiding failure than defining or analyzing it. In a paper entitled, Early warning signs of IT project failure: the dominant dozen,   Leon Kappelman et. al. outline the top twelve risks associated with IT project failures.  This post summarises the paper and lists the top twelve signs of impending trouble on projects.

Background and research methodology

The authors  focus on early warning signs – i.e. those that occur within the initial 20% of the planned schedule. Further, to ensure comprehensive coverage of risks, their conclusions are  based on inputs from academic and industry journals as well as from experienced IT project managers.  The paper provides a detailed explanation of their research methodology, which I’ll quote directly from the paper:

The research team first searched the literature extensively to develop a preliminary list of early warning signs (EWSs). The two authors experienced in IT project management then added several EWSs based on their personal experience. Next, 19 IT project management experts were asked to assess the list. On the basis of their feedback, we added new items and modified others to develop a list of 53 EWSs. Finally, the research team invited 138 experienced IT project managers (including the original 19 experts) to participate in rating the 53 EWSs using a scale from 1 (extremely unimportant) to 7 (extremely important). Fifty-five (55) of these managers completed the survey, yielding a response rate of nearly 40 percent. The respondents had an average of more than 15 years of IT project management experience. The budgets of the largest IT projects they managed ranged from 3 million to 7 billion dollars. About 30 percent held the title of program or project manager and nearly 20 percent had consultant titles. Director or program analyst titles accounted for about 15 percent each, 10 percent were vice presidents, and the rest held titles such as CEO, CIO, chief scientist, chief technologist, or partner.

Although the list and the rankings were based on the subjective opinions of experts, the large number of participants ensures a degree of  consensus regarding  the most important factors.

The troublesome twelve

After ranking the fifty odd risks, the authors focused on those that had scores above 6 (out of  a maximum possible of  7 as discussed above).  There were 17 risks that satisfied this (somewhat arbitrary) criterion.  Some of these were similar, so they could be combined. For example, the four risks:

  • No documented milestone deliverables and due dates.
  • No project status progress process
  • Schedule deadline not reconciled to the project schedule
  • Early project delays are ignored — no revision to the overall project schedule

were combined into: ineffective schedule planning and/or management.

This process of combining the top 17 items resulted in twelve risks, half of which turned out to be people-related and the other half process-related.  I discuss each of the risks in detail below.

People-related early warning signs

1.       Lack of top management support: This was the number one risk out of the fifty three that the authors listed. This isn’t surprising – a project that lacks executive support is unlikely to get the financial,  material or human resources necessary to make it happen.

2.       Ineffective project manager: Project managers who lack the communication and managerial skills needed to move the project ahead pose a serious risk to projects. The authors point out  that this is a common risk on IT projects because project managers  are often technical folks who have been promoted to managers. As such they may lack the interest, aptitude and/or skills to manage projects. Interestingly, the authors do not comment on the converse problem – whether the project manager’s lack of technical/domain knowledge contributes to project failure.

3.       No stakeholder involvement and/or participation: A large number of projects proceed with minimal involvement of key stakeholders. Such folks often lose interest in  projects when more immediate matters consume their attention.  In such situations a project manager may find it hard to get the resources he or she needs to get the project done. Stakeholder or sponsor apathy is an obvious warning sign that a project is headed for trouble.

4.       Uncommitted project team: The commitment (preferably, full-time) of a team is essential for the success of a project.  Management needs to ensure that team members are given the time (and incentives) to work on the project. A point that is often left unconsidered is the intrinsic motivation of the team – see this post for a detailed discussion of motivation in project management.

5.       Lack of technical knowledge/skills:  Project teams need to have the technical skills and knowledge that is relevant to the project. Managers sometimes wrongly assume that project staff can pick up the required skills  whilst working on a project.  Another common management misconception is that project personnel can master new technologies solely by attending training courses.   Getting contractors to do the work  is one solution to the problem.  However,  the best option is  to give the team enough time to get familiar with the technology prior to the project or, failing this,  to switch to a technology that the team is familiar with.

6.       Subject matter experts are not available: It is often assumed that subject matter experts can provide adequate inputs into projects whilst doing their regular jobs. This seldom works – when there’s a choice between the project and their jobs, the latter always wins.  Project sponsors need to  ensure that subject matter experts are freed up to work on the project.

Process-related early warning signs

1.       Unclear scope: The authors label this one as “Lack of documented requirements and/or success criteria.” However I think it is better described by the phrase I’ve used. All project management methodologies emphasise the importance of clear, well-documented requirements and success criteria –  and with good reason too. Lack of clarity regarding project scope means that no one knows where the project is headed – a sure sign of trouble ahead.

2.       No change control process:  As the cliché reminds us, change is the only constant in business environments.  It is therefore inevitable that project scope will change.  Changes to scope –however minor they may seem- need to be assessed for their impact on the project. The effect of several small (unanalyzed) scope changes on the project schedule should not be underestimated! Many project managers have a hard time pushing back on scope changes foisted on them by senior executives. Hence it is important that the change control process applies across the board – to everyone regardless of their authority.

3.       Ineffective scheduling and schedule management: Many schedules are built on little more than guesswork and an unhealthy dose of optimism, often because they are drawn up without input from the folks who’ll actually do the work (see my article on estimation errors for more on this). Schedules need to be rooted in reality. For this to happen, they must be based on reliable estimates, preferably from those responsible for creating the deliverables. Once the schedule is created, it is the project manager’s responsibility to update it continually, reflecting all the detours and road-bumps that have occurred along the way.  A common failing is that time overruns are not properly recorded,  leading to a false illusion of progress.

4.       Communication breakdown: Project communication is the art of getting people on the same page when they are reading different books. In my post on obstacles to project communication, I have discussed some generic difficulties posed by differences in stakeholder backgrounds and world-views. One of the key responsibilities of a project manager is to ensure that everyone on the project has a shared understanding of the project goals and shared commitment to achieving them. This is as true in the middle or the end of a project as it is at the start.

5.       Resources assigned to another project:  In my experience resources are rarely reassigned wholesale to other projects. What usualy happens is that they are  reassigned on a part time basis,  as in “we’ll take 20 % of Matt’s time and 10% of Nick’s time.” The problem with this is that Matt and Nick will end up spending most  of their time on the other project, leaving the one on hand bereft.

6.       No  business case: A not uncommon refrain in corporate hallways is, “Why are we doing this project?” No project should be given the go-ahead without a well-articulated business case. Further still, since an understanding reason(s) for doing the project are central to its success, these should be made available to every stakeholder:  a shared understanding of the goals of the project is a prerequisite to a shared understanding of the rationale behind it.

I’m sure there aren’t any surprises in this list –  most project managers would agree that these are indeed common (and often ignored) early warning signs of failure.  However, I suspect that there will be substantial differences of opinion regarding their ranking. Wisely, the authors have refrained from attempting to rank the risks – the list is not in order of importance.

Conclusion

Good projects managers  anticipate potential problems and take action to avoid them.  Although the risks listed above are indeed obvious , they are often ignored. Affected projects then  limp on to oblivion because those responsible failed to react to portents of trouble.  Granted, it can be hard to see problems from within the system,  particularly when the system is a  high-pressure project.  That’s where such lists are useful:  they can warn the project manager of potential  trouble ahead.

Written by K

January 6, 2011 at 10:42 pm

On the social construction of IS project risks

with 6 comments

Introduction

Most approaches to project risk management prescribe a structured approach to managing risks involving steps such as identification, analysis and response planning (see the PMBOK Guide, for example).  Implicit in this approach is the assumption that risks exist “out there” and   can be identified via systematic investigation.  Among other things, this leads to a view that it is possible – and indeed, desirable – to identify all possible risks on a project, and then manage these through the lifecycle of a project. Of course, new risks may emerge, but the underlying belief is that these too can be identified and managed using a structured approach.  In short, the fundamental assumption is that risks have an objective existence and can therefore be identified by an examination of the project and its environment.

In a paper entitled, The Limits of Risk Management – A Social Construction Approach, Bernd Stahl and co-authors argue that the traditional view of risk management is limited because:

  1. The assumption of objectivity leads to a false sense of security.
  2. It (often) ignores (or understates) risks that may be specific to certain organisations or situations.

Consequently, they suggest that it may be more fruitful to view risks as social constructs – perceptions of reality that are products of social interactions between stakeholders. In this post I explore this somewhat unusual view of risk. I’ll first illustrate the general idea of risk as social construct and then describe how some common IS project risks can be seen as social constructs. I’ll end with a brief look at the implications of this for project risk management.

Project risks as social constructs

The first step in classical (or standard) risk analysis is to identify all possible events that may affect the project.  However this, in principle, is impossible because there is no general method to do so. Consequently, risk managers usually compile lists of potential risks (based on prior experience, research literature, textbooks, brainstorming etc.) and use these as starting points for analysis.  The point is, all these methods are ad-hoc and, despite claims to the contrary, cannot guarantee a comprehensive coverage of all risks.  This point is made very clearly in a  paper by Lyytinen et. al. which examines four techniques of software project risk analysis and concludes that there are substantial differences between them.  Quoting from the conclusion to the Lyytinen paper:

…practitioners should be cautious with regard to the expected miracles of any risk management approach. Risk management approaches are neither complete nor risk-proof (my italics). Their value in the context of managing complex socio-technical change is that they put high premium on shaping management attention through learning new organizing schemes that help make sense of the development situations in new ways…

According to Stahl, the conclusions of the Lyytinnen (and a few other researchers)  support the idea that  risk s is a social construct that comes about as a result of  interactions and agreements between those involved in the project (and individual perceptions of these):

We believe that [the] objective concept of risk is flawed and threatens the success of the entire enterprise. A concept of objective risk raises the expectation that risks can be completely controlled. Also, it suggests that once the comprehensive list of risks is compiled, the end of the theoretical work is reached and managerial practice can take over. These factors can combine to create a false sense of security that will dull managers’ attention and can thereby create even bigger risks. We hold risk to be a social construction depending on social agreements and on individual perceptions. It must be ascribed to become real.

In my opinion there’s a big  leap of logic in the above paragraph:  I do not see how ”the expectation that risk can be completely controlled” and the “false sense of security this creates” implies that risk is a construct that depends on “social agreements  and individual perceptions.” Nevertheless, I do believe that the social constructionist view of risk is useful because most project risks have a social dimension. I’ll elaborate on this point via examples in the next section.

IS project risk as a social construct – some examples

Most of the research literature on social construction tends to be hard to read and devoid of examples that professional project managers can relate to. This is a shame because much of this work casts a critical eye on the implicit assumptions that underlie project management practice. So, instead of quoting from and paraphrasing hard-to-read papers, I’ll illustrate how some common project risks have a social dimension.

The risks I consider are drawn from a paper entitled, Early warning signs of IT project failure: the dominant dozen, by Leon Kappelman and his co workers.  The paper presents the top twelve early-stage project risks based on an analysis of data gathered from project professionals and a survey of research literature.  I’ll discuss how  four risks from Kappelman’s dominant dozen have social origins.

Lack of top management support

According to Kappelman et. al.  this is the number one risk in IS projects.  The point is that this risk most often arises because of (dysfunctional) politicking between managers. As  Kappelman et. al. state :

In many cases, IT projects get caught up in enterprise politics where there are fundamental disagreements about overall enterprise priorities. In these cases the resources and enterprise-wide commitment required for success are lacking. Middle managers do not see the project as being important to the enterprise or to their performance evaluations and therefore redirect resources and attention to activities that top management does support.

The point is: this risk is often a consequence of corporate politics, which is very much a social construct.

Lack of stakeholder involvement and/or participation

The following passage from the paper highlights the socially constructed nature of this risk:

If key project stakeholders do not participate in major review meetings, it signals they are not engaged in the project and therefore the project is not a high priority for them. Other stakeholders soon begin to disengage too. The project manager then finds it harder to get the participation and resources necessary for project success, especially from those who are not full-time members of the project team. Often such project team members get reassigned to other projects that are perceived to be more important.

The point is – key stakeholders, through their influence in the organisation, can cause a project to go awry.

Another common situation is one in which stakeholders lose interest in a project because they do not see the benefits of being involved. This point is well-recognised in traditional risk management which highlights the importance of getting stakeholder buy-in. The point here is that this risk is a consequence of individual perceptions, not of objective reality. In other words,  it is a social construct.

Lack of documented requirements and/or success criteria

This risk is an old classic, discussed and analysed by several well-known writers.  As Robert Glass  mentions in his book on the facts and fallacies of software engineering:

This problem is caused by the fact that the customers and users for the software solution are not really sure of what problem they need to have solved. They may think they know at the outset, only to discover as the project proceeds that the problem they wanted to solve is too simplistic or unrealistic or something else they weren’t expecting. Or they may really not know anything at the outset and are simply exploring solutions to a vague problem they know needs to be solved….

Regardless of the causes or the outcome, the point is that Ill defined scope means that different stakeholders have different interpretations of what’s to be delivered.  Consequently, project objectives become a matter of interpretation and opinion. This shows how even something as basic as project objectives are actually social constructs – they depend on individual stakeholder perceptions. Seen in this light it isn’t a stretch to say that there can be  as many objectives as there are stakeholders!

Communication breakdown among stakeholders

This risk is perhaps the most obvious social construct – communication is not only the lifeblood of a project, it is also the basis on which professional and social relationships are built. Communication becomes all the more important on projects environments where diverse stakeholders and ongoing change are the norm. As Kappelman et. al. state:

Any significant project has multiple stakeholders and requires an ongoing choreography of various tasks and resources.  Change over the life of the project is inevitable — business environment, competitor strategic and tactical moves, laws and regulations, management team, staff turnover, resource availability, and cost — to name just a few possibilities. If all stakeholders do not  communicate and work together on an ongoing basis, the project team will be pulled in multiple directions…

Communication breakdowns can occur for a variety reasons, but most of these arise because of conflicting viewpoints of those involved. As I have written in a post on project communication, “A shared world-view –  which includes a common understanding of tools, terminology, culture, politics etc. –  is what enables effective communication within a group.”

Implications

The above examples show how common project risks have a social dimension, even if they are not entirely social constructs.  The social aspects of risk are often ignored because they are hard to handle –   it is much easier to follow a process than to deal with stakeholders who are (or might get) upset. Consequently risks such as communication breakdown or lack of management support are not broached because of the high cost of speaking up.  I contend that many project risks remain unaddressed because of this.

The implications of a social constructionist view of risk can be summed up as follows:

  1. It is impossible, in principle, to compile authoritative lists of all possible risks.
  2. Risks should be studied in the context of a particular project and its environment. This includes technical, social and organizational aspects of the environment.
  3. Particular emphasis should be given to relationships between stakeholders as these may present barriers to an honest assessment of risks.

These implications suggest that a participatory approach involving open deliberation is mandatory for successful  risk management. As Stahl et. al. put it:

Indeed, a checklist of risks is just a starting point for an ongoing debate on risk. Management should identify the stakeholder of risky decisions and engage in a free discourse about the nature and the evaluation of risks. The stakeholder discourse could be used to define responsibilities . The stakeholders as parties interested in the process are presumably best suited to identify risk factors. Ideally this process would lead to a consensus concerning the risks. Similar approaches have been suggested in the literature, however, we wish to emphasise that only when managers understand that risk is a social construct will the complex and costly process of stakeholder discourses as a way of dealing with risk make sense (my italics).

Summing up

The mathematical and analytical machinery of risk management can obscure the fact that  managing risks  is as much an art as a science, calling for skills in dealing with people as well as probabilities. Organisational politics, individual perceptions and interactions between different stakeholder groups play a role in creating risks.  In other words, risks are not objective entities, they have a social dimension.

Written by K

December 2, 2010 at 5:33 am

Six ways in which project estimates go wrong

with 7 comments

Despite the increasing focus on  project estimation, the activity still remains  more guesswork than art or science.  In his book on the fallacies of software engineering,  Robert Glass has this to say about it:

Estimation, as you might imagine, is the process by which we determine how long a project will take and how much it will cost. We do estimation very badly in the software field. Most of our estimates are more like wishes than realistic targets. To make matters worse, we seem to have no idea how to improve on those very bad practices. And the result is, as everyone tries to meet an impossible estimation target, shortcuts are taken, good practices are skipped, and the inevitable schedule runaway becomes a technology runaway as well…

Moreover, he suggests that poor estimation is one of the top two causes of project failure.

Now, there are a number of reasons why project estimates go wrong,  but in my experience there are half-dozen standouts. Here they are, in no particular order:

1.  False analogies: Project estimates based on historical data are generally considered to be more reliable than those developed using other methods such as expert judgement (see this article, from the MS Project support site for example). This is fine and good as long as one uses data from historical projects that are identical to the one at hand in relevant ways. Problem is, one rarely knows what is relevant and what isn’t.  It is all too easy too select a project that is superficially similar to the one at hand, but actually differs in critical ways.  See my posts on false analogies and the reference class problem for more on this point.

2.  False precision: Project estimates are often quoted as single numbers rather than ranges. Such estimates are incorrect because they ignore the fact that uncertain quantities should be quantified by a range of numbers (or more accurately, a distribution) rather than  point values. As Dr. Sam Savage emphasises in his book, The Flaw of Averages: an uncertain quantity is a shape, not a number (see my review of the book for more on this point). In short, an estimate quoted as a single number is almost guaranteed to be incorrect.

3.  Estimation by decree: It should be obvious that estimation must be done by those who will do the work. Unfortunately this principle is one of the first to be sacrificed on Death March Projects. In such projects, schedules  are shoe-horned into  predetermined timelines, with estimates cooked up by those who have little or no idea of the actual effort involved in doing the work.

4.   Subjectivity: This is where estimates are plucked out of thin air and “justified” based on gut-feel and other subjective notions. Such estimates are prone to overconfidence and a range of other cognitive biases. See my post on cognitive biases as project meta-risks for a detailed discussion of how these biases manifest themselves in project estimates.

5.  Coordination neglect: Projects consist of diverse tasks that need to be coordinated and integrated carefully. Unfortunately, the time and effort needed for coordination and integration  is often underestimated (or even totally overlooked)  by project decision makers. This is referred to as coordination neglect. Coordination neglect is a problem in projects of all sizes, but  is generally more significant for projects involving large teams (see this paper for an empirical study of the effect of team size on coordination neglect).  As one might imagine, coordination neglect also becomes a significant problem in projects that consist of a large number of dependent tasks  or have a large number of external dependencies.

6.  Too coarse grained: Large tasks are made up of smaller tasks strung together in specific ways. Consequently, estimates for large tasks should be built up from estimates for the smaller sub-tasks. . Teams often short-circuit the process by attempting to estimate the large task directly. Such estimates usually turn out to be incorrect because sub-tasks  are overlooked. Another problem is coordination neglect  between sub-tasks, as discussed in the earlier point. It is true – – the devil is always in the details.

I should emphasise that the above list based on personal experience, not on any systematic study.

I’ll conclude this piece with another fragment from Glass, who  is not very optimistic about improvements in the area of project estimation. As he states in his book:

The bottom line is that, here in the first decade of the twenty-first century, we don’t know what constitutes a good estimation approach, one that can yield decent estimates with good confidence that they will really predict when a project will be completed and how much it will cost. That is a discouraging bottom line. Amidst all the clamor to avoid crunch mode and end death marches, it suggests that so long as faulty schedule and cost estimates are the chief management control factors on software projects, we will not see much improvement.

True enough,  but being aware of the ways in which estimates can go wrong is the first step towards improving them.

Written by K

October 1, 2010 at 4:55 am