Archive for the ‘Paper Review’ Category
Do project managers learn from experience?
Do project managers learn from their experiences? One might think the answer is a pretty obvious, “Yes.” However in a Harvard Business Review article entitled, The Experience Trap, Kishore Sengupta, Tarek Abdel-Hamid and Luk Van Wassenhove suggest the answer may be a negative, especially on complex projects. I found this claim surprising, as I’m sure many project managers would. It is therefore worth reviewing the article and the arguments made therein.
The article is based on a study in which several hundred experienced project managers were asked to manage a simulated software project with specified goals and constraints. Most participants failed miserably: their deliverables were late , over-budget and defect ridden. This despite the fact that most of them acknowledged that the problems encountered on the simulations were reflective of those they had previously seen on real projects. The authors suggest this indicates problems with the way project managers learn from experience. Specifically:
- When making decisions, project managers do not take into account consequences of prior actions.
- Project managers don’t change their approach, even when it is evident that it doesn’t work.
The authors identify three causes for this breakdown in learning:
- Time lags between cause and effect: In complex projects, the link between causes and effects are not immediately apparent. The main reason for this, the authors contend, is that there can be significant delays between the two – e.g. something done today might affect the project only after a month. The authors studied this effect through another simulated project in which requirements increased during implementation. The participants were asked to make hiring decisions at specified intervals in the project, based on their anticipated staffing requirements. The results showed that the ability of the participants to make good hiring decisions deteriorated as the arrival lag (time between hiring and arrival) or assimilation lag (time between arrival and assimilation) increased. This, the authors claim, shows that project managers find it hard to make causal connections when delays between causes and effects are large.
- Incorrect estimates: It is well established that software projects are notoriously hard to estimate (see my article on complexity of IT projects for more on why this is so). The authors studied how project managers handle incorrect estimates. This, again, was done through a simulation. What they found was participants tended to be overly conservative when providing estimates even when things were actually going quite well. The authors suggest this is an indication that project managers attempt to game the system to get more resources (or time), regardless of what the project data tells them.
- Initial goal bias: Through yet another another simulation, the authors studied what happens as project goals change with time. The simulation started out with a well-defined initial scope which was then changed some time after the project started. Participants were not required to reevaluate goals as scope changed, but that avenue was open to them. The researchers found that none of the participants readjusted their goals in response to the change, thus indicating that unless explicitly required to reevaluate objectives, project managers tend to stick to their original targets.
After discussing the above barriers to learning, the authors provide the following suggestions reduce these:
- Provide cognitive feedback: A good way to understand causal relationships in complex processes is to provide cognitive feedback – i.e feedback that clarifies the connection between important variables. In the simulation exercise involving arrival / assimilation delays, participants who were provided with such feedback (basically graphical displays of number of defects detected vs time) were able to make better (i.e. more timely) staffing decisions.
- Use calibrated model-based tools and guidelines: The authors suggest using decision support and forecasting tools to guide project decision-making. They warn that these tools should be calibrated to the specific industry and environment.
- Set goals based on behaviours rather than performance: Most project managers are assessed on their performance – i.e. the success of their projects. Instead, the authors suggest setting goals that promote specific behaviours. An example of such a goal might be the reduction of team attrition. Such a goal would ensure that project managers focus on things such as promoting learning within the team, protecting their team from schedule pressure etc. This, the logic goes, will lead to better team cohesion and morale, ultimately resulting in better project outcomes.
- Use project simulators: Project simulations provide a safe environment for project managers to hone their skills and learn new ones. The authors cite a case where introduction of project simulation games significantly improved the performance of managers on projects, and also lead to a better understanding of dynamic relationships in complex environments.
Although many of the problems (e.g. inaccurate estimates) and solutions (e.g. use of simulation and decision support software) discussed in the article aren’t new, the authors present an interesting and thought-provoking study on the apparently widespread failure of project managers to learn from experience. However, for reasons which now I outline, I believe their case may somewhat overstated.
Regarding the research methodology, I believe their reliance on simulations limits the strength, if not the validity, of their claims. More on this below:
- Having participated in project simulations before, I can say that simulators cannot simulate (!)important people-related factors which are always present in a real project environment. These include factors such as personal relationships and ill-defined but important notions such as organisational culture. In my experience, project managers always have to take these into account when making project decisions.
- Typically many of the important factors on real projects are “fuzzy” and have complex dependencies that are hard to disentangle. Simulations are only as good as the models they use, and these factors are hard to model.
On the solutions recommended by the authors:
- I’m somewhat sceptical about the use of software tools to supports decision making. In my experience, decision support tools require a fair bit of calibration, practice and (good) data to be of any real use. By the time one gets them working, one usually has a good handle on the problem any way. They’re also singularly useless when extrapolated to new situations – and projects (almost by definition) often involve new situations.
- Setting behavioural goals is nice in theory, but I’m not sure how it would work in practice. Essentially I have a problem with how a project manager’s performance will be measured against such goals. The causal connection between a behavioural goal such as “reduce team attrition” and improved project performance is indirect at best.
- Regarding simulators as training tools, I have used them and have been less than impressed. It is very easy to make a “wrong” decision on a simulator because information has been hidden from you. In a real life situation, a canny project manager would find ways to gather the information he or she needs to make an informed decision, even if this is hard to do. Typically, this involves using informal communication modes and generally keeping ear to the ground. The best project managers excel at this.
So, in closing, I think the authors have a point about the disconnect between project management practice and learning at the level of an individual project manager. However, I believe their thesis is somewhat diluted because it is based on the results of simulated project games which are limited in their ability to replicate complex, people-related issues that are encountered on real projects.
A path-based approach to corporate IT
CIOs the world over struggle to keep up with ever changing business requirements. At the same time, many companies feel short-changed by their IT departments, which end-users often perceive as being inflexible and slow to respond to changes in business needs. This lack of flexibility can usually be traced back to TItanic-like enterprise systems which straitjacket businesses into certain ways of doing things. In an article entitled Radically Simple IT published in the March 2008 issue of the Harvard Business Review , David Upton and Bradley Staats propose a “novel approach” to the design and development of enterprise systems. They call the approach path-based because it provides a path for systems to be developed over time, incorporating changing (or missed) requirements along the way.
The path-based approach is based on the following tenets:
- Meld IT and business strategies together: The authors reckon that IT-business alignment, as it is usually presented, is hard to achieve because of a lack of mutual understanding between the two sides (this is something I’ve alluded to in an earlier post on project communication). Most discussions between IT and business tend to focus on individual projects rather than the big picture. The latter, however, is integral to a path based approach. To ensure that both sides understand each other, the authors suggest that IT staff be encouraged to understand the business and vice versa. This recommendation is far from new – see this article from 2005, for example.
- Keep things as simple as possible: The article recommends that CIOs should strive for simplicity in their IT environments, thereby ensuring that systems remain flexible (i.e. open to change). Some ways to achieve this include:
- Adhering to a minimal set of standards, so that the company has a standard operating environment at the client and server level. Additionally, standards should be recommended for specific applications.
- Evaluating whether new functionality requested by the business can be provided by existing software.
- Looking for simple solutions before buying or building complicated ones.
- Developing a modular enterprise architecture . This is essentially the notion of loose coupling – where individual systems are built in such a way as to minimise effects and dependencies on other systems.
- Empower end-users: All project and IT managers know that the biggest resistance to new systems often comes from end-users who, quite naturally, are reluctant to change their ways of working. Without end-user buy in, any new system is doomed to fail. The authors recommend reducing such organisational resistance to new applications by rolling out changes gradually. They also recommend soliciting user feedback on a continuous basis. The feedback should be acted on (and be seen to be acted on) in a timely manner so that users know their input is taken seriously.
The above principles are hardly new or radical, and few CIOs would question them. The path based approach really amounts to a consistent application of these in a (often difficult and ever changing) business environment. The authors illustrate the approach through a case study based on the experiences of Shinsei Bank. The case study is helpful because it provides concrete examples of application of the principles discussed. For more on the Shinsei story, read this transcript of an interview with Jay Dwivedi, the CIO responsible for implementing a path-based approach at that bank.
The article is well written and makes a good case for implementing such an approach in just about any corporate IT environment. However, many CIOs – particularly those stuck in organisations that don’t appreciate the strategic value of IT – will find it hard to put these principles into practice. But try they must, else their departments will continue to be viewed as cash sinks rather than strategic assets.
The effect of organizational culture on knowledge transfer in projectized organisations
The knowledge gained during the implementation of a project is often lost after the project is completed. This loss carries a huge cost as future project teams have to rediscover lost knowledge, or reinvent the proverbial wheel. Although learnings are often (or should I say “sometimes”?) documented in project post-mortems or lessons learnt documents, these are rarely, if ever, read by project teams down the line. Management and transfer of knowledge involves complex processes which, in turn, depend on several factors that vary from organisation to organisation. One such factor is organisational culture. A recent paper entitled, Knowledge Transfer in Project Based Organizations: An Organization Culture Perspective, published in the Project Management Journal investigates obstacles to knowledge transfer in project-based organisations, with particular emphasis on the role of organisational culture. I summarise and review the paper below.
The authors begin by a brief overview of what is meant by knowledge and how it is created. They distinguish between data (unprocessed facts), information (meaningful aggregations of data) and knowledge (information that is processed and filtered on the basis of an individual’s perception, skills and experience). Knowledge involves assimilation by the human mind whereas data and information do not. The authors also draw a distinction between explicit and tacit knowledge, i.e. that which is documented and that which is undocumented, often existing only in people’s minds. According to the SECI model proposed by Nonaka and Takeuchi, new knowledge is created by an interaction between tacit and explicit knowledge through the processes of Socialisation, Externalisation, Combination and Internalisation. Other researchers claim that explicit knowledge is an extension of tacit knowledge to a new level, whereby it is “consciously known” and hence can be transferred to others.
The authors then move on to discussing how knowledge is transferred. Their discussion is based mainly on a paper by Nissen and Snider (see abstract), which describes three views of project-related knowledge transfer:
- As solution – wherein knowledge is transferred on the job – i.e. when working on projects. In this perspective, managers facilitate knowledge flow by ensuring selection of appropriate technologies and motivating individuals to use them.
- As experience – wherein knowledge is transferred by capturing experiences (by documentation, for example) for future reference. Here the emphasis is on flow of knowledge across time. An example of this is when knowledge is transferred from one project team to another.
- As socially created -wherein knowledge is created (not transferred!) through interpersonal interactions (discussions, arguments and other informal communications). The challenges associated with this form of knowledge transfer are primarily in fostering an organisational culture that encourages informal communication. Although this may be considered outside the ambit of a project manager’s responsibilities, a project manager can help by fostering a communication-friendly culture within the project team.
The authors then point out that project management has grown beyond its traditional role of planning, controlling and monitoring projects to a more strategic role of resource optimisations and inter-departmental collaboration – which ultimately ends up providing better customer service. I’m not sure why they throw this point in, as it seems like a side issue to the main focus of the paper. Perhaps it is to emphasise that effective knowledge transfer processes become more critical as project management takes on a more strategic role in organisations.
Project teams are often required to find and assimilate knowledge that exists in organisational memory. The authors suggest that this task is easier in functional organisations than in projectised ones, because in the former, knowledge is organised and stored by department (and hence, presumably, easy to find). I beg to differ: In my experience the task can be just as hard in functionally structured organisations. This is true for most of the other knowledge transfer obstacles they mention, such as: volume of documentation, no time to document etc. etc. My point: knowledge management problems listed by the authors aren’t unique to project-based organisations.
Next the paper looks at the classification of project-related knowledge. Here the authors follow the work of Conroy and Soltan, who suggested that all project-related knowledge falls into one of the following: an organisation knowledge base, a project management knowledge base and a project-specific knowledge base; each being developed (or augmented) during the implementation of projects. Since projects are the only means via which knowledge bases are enhanced, it is important that project learnings are captured effectively.
Having discussed the need to capture project knowledge and the importance of project-specific knowledge, the authors move on to a discussion of obstacles to knowledge transfer in projectised organisations. They identify the following road blocks to knowledge transfer:
-
Project constraints leave little time and resources for effective documentation of knowledge.
-
The existence of significant social and cultural barriers to knowledge transfer. These are things such as: lack of openness, no tolerance of failure, blame culture etc.
-
Lack of motivation (or incentives) to undertake project reviews. This is the well known wiifm factor.
-
Lack of leadership that accords enough importance to developing the organisation’s knowledge base.
The authors contend that these issues boil down to inadequacies in organisational culture. Stated another way: the transfer of intrinsic knowledge (which exists in people’s minds) can occur only in an organisational culture that supports it. This definitely rings true, and I don’t think any project manager would argue with it. I would add that the aforementioned obstacles are alive and well in all organisations – not just project-based ones.
Having established that a supportive organisational culture is critical to knowledge transfer, the authors move on to discussing the cultural variables that promote knowledge transfer. In particular, they discuss models by West and Schneider.
West proposes the following two dimensions of organisational culture:
- Flexibility versus control: this measures the degree of organisational control. Specifically, flexible organisations tend to have flat hierarchies, decentralised decision making as opposed to controlled organisations which have rigid (and often deep) hierarchies and centralised decision making.
- Internal versus external orientation: this measures the degree to which the organisation is affected by its cultural environment. Although organisations are situated in a greater cultural context, the extent to which they are affected by it can vary.
As compared to control-based organisations, flexible organisations are characterised by flat hierarchies and decentralised decision making. The authors mention that rigid structures and hierarchies promote efficiency, though often at the expense of innovation and collaboration. In my experience in control-based organisations, the emphasis on efficiency and control often leads to an overemphasis on process, which is ultimately what reduces any possibility of collaboration and innovation.
In contrast to West, Schneider proposes the following dimensions of organisational culture:
-
Possibility versus actuality: this relates to what the organisation focuses on – potentialities or facts.
-
Personal versus impersonal: this relates to how decisions are made in the organisation – process driven (i.e. by logic) or based on feelings, beliefs, ideals (i.e. all those “warm and fuzzy” notions).
These axes divide the universe of organisations into four core cultures:
-
Control: a process driven culture which values certainty and predictability above all.
-
Competence: a culture that values excellence (in services or products). These organisations aim to have unique or best of breed products and services.
-
Collaboration: this culture emphasises close connections with customers and collaborative working towards developing products or solutions. This is essentially the opposite of a competence culture.
-
Cultivation: this culture emphasises openness, autonomy, ideals and ideas. These organisations tend to be innovative. This is essentially the opposite of a control culture.
It is intuitively clear that the above dimensions of organisational culture will have varying effects on the efficacy of knowledge transfer. The authors, however, do not delve into these. So their coverage of models of organisational culture, as interesting as it is, seemed a little pointless in the context of the paper.
Next the authors point out that since project teams consist of individuals drawn from various professions, one needs to consider the effect of professional culture as well. This is an interesting point, which complicates matters because different professional cultures approach communication in different ways – the example of IT and business cultures being a particularly stark case in point.
Finally, the authors end with a very general discussion of how knowledge transfer can be promoted in projectised organisations. They suggest that managers should:
- Recognise different levels at which knowledge is generated – i.e. individual, group and organisational.
- Appreciate the role of organisational culture in promoting or hindering knowledge transfer between these levels. They suggest the primary barriers to knowledge transfer are cultural rather than technical (See this article by Suzanne Koudsi, for an interesting case study).
- Understand the role that management plays in fostering a culture that facilitates knowledge transfer. This is particularly true for project managers who have to deal with many different cultures (organisational, departmental and project team) simultaneously. Awareness of cultural differences can help managers find the cause of obstacles to knowledge transfer.
- Appreciate the challenges involved in transforming organisational culture.
- And finally, since projects are the coalface at which knowledge is generated, practitioners must understand the issues that need to be addressed to facilitate gathering and preserving of relevant knowledge generated during project implementation. Some examples of these include communication modes between team members, what worked well in the project, what can be improved and how it might be improved, etc.
Having covered a lot of ground (somewhat skimpily, in my opinion), the authors conclude the paper with three very general statements:
- Effective knowledge transfer can occur only if the organisational culture is open to accepting new knowledge transfer activities. Managers must therefore prepare the culture to accept, adopt and implement these activities.
- In addition to knowledge transfer, knowledge management is also about fostering an organisational culture that encourages the creation, sharing and utilisation of knowledge.
- Project managers have to merge myriad organisational, departmental and professional cultures into an effective project culture that promotes knowledge management.
Although project managers may appreciate these points (and I suspect many do), there’s little they can do about changing an organisation’s cultural mindset. All they can do is ensure that they establish a culture that fosters knowledge transfer within their own projects (point 3 above). Some ways to do this include: encourage collaboration between team members, establish a no-blame environment etc.
In conclusion, the paper was reasonably interesting but somewhat hard to read because it was thin on detail, and occasionally veered from topic to topic without proper transition or introduction. I suspect this is because the authors assumed their readership would consist entirely of academics with a strong grounding in project management theory. This is a pity because the paper, if better written, could have been of considerable interest to practising project managers. So I end this review with a plea to journal paper authors: when writing your paper, please remember that project management isn’t just an academic discipline.
References:
Ajmal, M. M. and Koskinen, K.U., Knowledge Transfer in Project-Based Organizations: An Organizational Culture Perspective, Project Management Journal, 39 (1), 7-15. (2008).
Eight to Late 