Archive for the ‘Organizations’ Category
Symptoms, not causes: a systems perspective on project failure
Introduction
A quick search reveals that the topic of project failure gets a fair bit of attention on the Internet. Many of the articles identify factors such as lack of executive support or incomplete/misunderstood requirements as the prime causes of failure. In this post I use concepts from systems theory to argue that commonly identified “causes” of project failure – such as the ones noted above – are symptoms rather than causes. I then surface the real causes of project failure by taking a systems perspective – a viewpoint that considers the project and the hosting organisation as a whole.
As I will argue, project failures can often be traced back to dysfunctional structures and processes within the organisation. These factors usually have little to do with the project directly and are therefore not always obvious at first sight.
Setting the scene
Readers who have waded through the vast literature on failed projects will have noted that there are diverse opinions on the prime reasons for failure. It would take me too long to wade through these articles, so I’ll just pick a source that is well known even if not entirely credible: The Chaos Report by the Standish Group. As per this summary the Chaos Report 2009 claimed the following were the top three causes of project failure:
- Lack of user input.
- Incomplete or changing requirements/specifications.
- Lack of executive support.
Although the report lists the top ten factors, in the interests of space I’ll focus on just these three.
I should mention that the report refers to the above as “Project Challenged Factors.” Grammar issues aside, this is a somewhat strange way of putting it. Anyway, I interpret this phrase to mean reasons (or driving factors) for failure.
Systems theory and projects
First up, what exactly is a system?
Here is a jargon-free definition from the wonderful book by Donella Meadows entitled, Thinking in Systems: A Primer. Incidentally, I highly recommend the book as an easy-to-read and engaging introduction to systems theory:
A system is a set of things interconnected in such a way that they produce their own pattern of behaviour over time. The system may be buffeted, constricted , triggered or driven by outside forces. But its response to these is characteristic of itself and is seldom simple in the real world. (italics mine)
The word interconnected is important because it tells us that when we study something from a systems perspective, we must identify all the important connections it has with its environment. In the case of projects, the important connections are fairly obvious. A project is usually carried out within an organisational setting so it would have many connections to the hosting organisation. Chief amongst these is the fact that a project is staffed and resourced by the hosting organisation (or by an organisation designated by it). Another important connection is that a project will affect different stakeholder groups within the hosting organisation in different ways. However, since all stakeholders have ongoing organisational roles that go beyond the project, the project is not their only interest. This is a key point to which we will return later.
The phrases “own pattern of behaviour over time” and “characteristic of itself” tell us that systems have a unique pattern of behaviour that is characteristic to them. The important point to note is that characteristic behaviour implies that different systems that have the same signature – i.e. identifying features – will tend to behave in the same way. From the perspective of projects this tells us that projects within similar organisations will evolve in similar ways.
A systems view of the causes of project failure
With only this very brief look at systems theory we are now in a position to get some insights into the real causes of project failure. As mentioned earlier, we will focus on the top three reasons ala Standish.
Lack of user input
First up, consider lack of user input. Systems theory tells us that we need to look at the issue from the perspective of the project and the organisation. From this point of view it is clear that users who have been asked to work on the project in addition to their normal duties will view the project as a burden rather than something that may benefit them in the future.
This by itself is not a new insight. In fact, project management gurus have talked themselves hoarse about the need to free up resources etc. However, the point is that organisational structures typically work against this. Firstly, people who are asked to work on a project know that it is an initiative that will end in a finite (usually, reasonably short) time. Therefore, their long terms interests lies in their ongoing roles rather than in projects. Secondly, most organisations are still structured along functional lines and people’s identities are anchored within their functional reporting lines rather than ephemeral project hierarchies.
Changing requirements
The issue of changing requirements and specifications can also be understood from a systems point of view. A characteristic of many systems is that they are stable – i.e. they resist change. Organisations are typically stable systems – they tend to retain their identity despite changes in their environment. One of the characteristics of such organisations is that people in them tend to think and act in set ways – that is, their actions and thinking processes follow well worn patterns to the point where they do not need to think about what they do too deeply.
One of the consequences of this is that when they are asked for requirements users often provide incomplete description of what they do, leaving out significant items that are obvious to them (but not to the analysts who are gathering requirements!). Although I don’t have the figures to back this – I speculate that a fair proportion of changes in requirements are the result of inadequate detail or thought put into developing initial requirements. The point is users and sponsors don’t necessarily see these as changes, but project teams do.
Lack of executive support
Finally, let’s look at the the problem of lack of executive support. Project sponsors usually hold important executive-level positions within the hosting organisation. By virtue of their positions, they have a number of important things that compete for their attention. A project – even an important one – is only one of many things going on in an organisation at any one time. Moreover, organisational priorities do shift, perhaps more often than executives may want to admit. So a project that was the key focus yesterday may be superseded by other priorities today.
There are of course many other ways in which project sponsors can be distracted, but I think I’ve made my point which is that lack of executive support is due to features that are inherent in organisations. So no amount of forcing executives to pay attention to their projects is going to work unless the entire system (project + organisation) changes. And this is difficult if not impossible to achieve because stable systems such as organisations tend to resist change, and therefore continue to display their characteristic patterns of behaviour.
Discussion
So we see that the causes of project failure can be traced back to the organisations in which they are embedded. Specifically, they lie in unwritten norms and formal policies that dictate how the hierarchy operates and how things are done within the organisation. The most important consequence of this is that standard fixes (of encouraging user input and executive support, or instituting change management, say) will not cure the problem of project failure because they do not address the dysfunctional norms and policies that are the root cause of failure.
The above is not news. In fact, the matrix organisation structure was proposed as a response to the need for “project friendly” organisations. I’m no expert on matrix organisations so I will leave it to others to comment on how successful they are. The only point I would make is that in my experience multiple reporting lines (even if dotted and solid) do not work too well. There are always conflicting interests that cause divided loyalties and conflicting interests.
So, the natural question is : what – if anything – can we do about this? The answer is implicit in the foregoing paragraphs. One has to align the project with the organisation, not just at the level of objectives or structure, but also in operational matters such as timelines, budget and resources – the very things that make up the so-called iron triangle. The best time to do this is at the front-end of the project, i.e. the start. At this time, the person(s) driving the project have to engage all stakeholders who will be affected by the initiative and find out their motivations, interests and – most importantly – their concerns regarding the project. If this discussion happens in an open and frank manner, it should surface the issues highlighted In the previous section. Since the discussion takes place even before the project starts, there is at least some hope of addressing these concerns.
There are many ways to structure and facilitate such discussions. Check out this post for an introduction to one and have a look at my book co-authored with Paul Culmsee for much more. That said, one doesn’t need any particular technique – the willingness to discuss difficult matters openly and an openness to other points of view is all that’s needed. That, however, is not always easy to come by…
Conclusion
We have seen that the top causes of project failure can be traced back to the hierarchies and incentive systems of the hosting organisation. Therefore, superficial attempts to fix the problem at the level of individual projects (or even a PMO) will not work. The only hope of addressing the root causes of project failure is to focus on the systemic dysfunctions that cause them.
The paradox of the learning organisation
Introduction
The term learning organisation refers to an organisation that continually modifies itself in response to changes in its environment. Ever since Peter Senge coined the term in his book, The Fifth Discipline, assorted consultants and academics have been telling us that a learning organisation is an ideal worth striving for. The reality, however, is that most organisations that undertake the journey actually end up in a place far removed from this ideal. Among other things, the journey may expose managerial hypocrisies that contradict the very notion of a learning organisation. In this post, I elaborate on the paradoxes of learning organisations, drawing on an excellent and very readable paper by Paul Tosey entitled, The Hunting of the Learning Organisation: A Paradoxical Journey.
(Note: I should point out that the term learning organisation should be distinguished from organisational learning: the latter refers to processes of learning whereas the former is about an ideal type of organisation. See this paper for more on the distinction.)
The journey metaphor
Consultants and other experts are quick to point out that the path to a learning organisation is a journey towards an ideal that can never be reached. Quoting from this paper, Tosey writes, “we would talk about the fact that, in some ways, the learning organization represented all of our collective best wishes for Utopia in the workplace.” As another example, Peter Senge writes of it being, “a journey in search of the experience of being a member of `a great team.” Elsewhere, Senge suggests that the learning organisation is a vision that is essentially unattainable.
The metaphor of a journey seems an apt one at first, but there are a couple of problems with it. Firstly, the causal connection between initiatives that purport to get one to the goal and actual improvements in an organisation’s capacity to learn is tenuous and impossible to establish. This suggests the journey is one without a map. Secondly, the process of learning about learning within the organisation – how it occurs, and how it is perceived by different stakeholders – can expose organisational hypocrisies and double-speak that may otherwise have remained hidden. Thus instead of progressing towards the the ideal one may end up moving away from it. Tosey explores these paradoxes by comparing the journey of a learning organisation to the one described in Lewis Carroll’s poem, The Hunting of The Snark.
Hunting the Snark (and the learning organisation)
Carroll’s poem tells the story of ten characters who set of in search of a fabulous creature called a Snark. After many trials and tribulations, they end up finding out that the Snark is something else: a not-so-pleasant creature called a Boojum. Tosey comments that the quest described in the poem is a superb metaphor for the journey towards a learning organisation. As he states:
Initially, when reflecting on personal experience of organizational events… I was struck by the potential of the dream-like voyage of fancy on which Carroll’s characters embarked as an allegory of the quest for the learning organization. Pure allegory has limitations. Through writing and developing the article I came to view the poem more as a paradigm of the consequences of human desire for, and efforts at, progress through the striving for ideals. In other words the poem expresses something about our `hunting’. In this respect it may represent a mythological theme,a profound metaphor more than a mere cautionary moral tale.
There are many interesting parallels between the hunt for the Snark and the journey towards a learning organisation. Here are a few:
The expedition to find the Snark is led by a character called the Bellman who asserts: “What I tell you three times is true.” This is akin to the assurances (pleas?) from experts who tell us (several times over) that it is possible to transform our organisations into ones that continually learn.
The journey itself is directionless because the Bellman’s map is useless. In Carroll’s words:
Other maps are such shapes, with their islands and capes!
But we’ve got our brave Captain to thank:
(So the crew would protest) “that he’s bought us the best—
A perfect and absolute blank!
Finally, the Snark is never found. In its stead, the crew find a scary creature called a Boojum that has the power to make one disappear. Quoting from the poem:
In the midst of the word he was trying to say,
In the midst of his laughter and glee,
He had softly and suddenly vanished away—
For the Snark was a Boojum, you see.
The journey towards a learning organisation often reveals the Boojum-like dark side of organisations. One common example of this is when the process of learning surfaces questions that are uncomfortable for those in power. Tosey relates the following tale which may be familiar to some readers,
…a multinational company intending to develop itself as a learning organization ran programmes to encourage managers to challenge received wisdom and to take an inquiring approach. Later, one participant attended an awayday, where the managing director of his division circulated among staff over dinner. The participant raised a question about the approach the MD had taken on a particular project; with hindsight, had that been the best strategy? `That was the way I did it’, said the MD. `But do you think there was a better way?’, asked the participant. `I don’t think you heard me’, replied the MD. `That was the way I did it’. `That I heard’, continued the participant, `but might there have been a better way?’. The MD fixed his gaze on the participants’ lapel badge, then looked him in the eye, saying coldly, `I will remember your name’, before walking away.
One could argue that a certain kind of learning – that of how the organisation learns – occurred here: the employee learnt that certain questions were out of bounds. I think it is safe to say, though, that this was not the kind of learning that was intended by those who initiated the program.
In the preface to the poem, Carroll notes that the Bellman there is a rule – Rule 42 – which states, “No one shall speak to the Man at the Helm,” to which the Bellman (the leader) added, “and the Man at the Helm shall speak to no one.” This rendered communication between the helmsman and the crew impossible. In such periods the ship was not steered. The parallels between this and organisational life are clear: there is rarely open communication between the those steering the organisational ship and rank and file employees. Indeed, Tosey reformulates Rule 42 in organisational terms as, “the organization shall not speak to the supervision, and the supervision shall not speak to the organization.” This, he tells us, interrupts the feedback loop between individual experience and the organisations which renders learning impossible.
(Note: I can’t help but wonder if Douglas Adams’ famous answer to the life universe and everything was inspired by Carroll’s rule 42…)
In the poem, the ship sometimes sailed backwards when Rule 42 was in operation. Tosey draws a parallel between “sailing backwards” and unexpected or unintended consequence of organisational rules. He argues that organisational actions can result in learning even if those actions were originally intended to achieve something else. The employee in the story above learnt something about the organisational hierarchy and how it worked.
Finally, it is a feature of Rule-42-like rules that they cannot be named. The employee in the story above could not have pointed out that the manager was acting in a manner that was inconsistent with the intent of the programme – at least not without putting his own position at risk. Perhaps that in itself is a kind of learning, though of a rather sad kind.
Conclusion
Experts and consultants have told us many times over that the journey towards a learning organisation is one worth making….and as the as the Bellman in Carroll’s poem says: “What I tell you three times is true.” Nevertheless, the reality is that instances in which learning actually occurs tend to be more a consequence of accident than plan, and tend to be transient than lasting. Finally, and perhaps most important, the Snark may turn out to Boojum: people may end up learning truths that the organisation would rather remained hidden. And therein lies the paradox of the learning organisation.
A stupidity-based theory of organisations – a paper review
Introduction
The platitude “our people are our most important asset” reflects a belief that the survival and evolution of organisations depends on the intellectual and cognitive capacities of the individuals who comprise them. However, in view of the many well documented examples of actions that demonstrate a lack of foresight and/or general callousness about the fate of organisations or those who work in them, one has to wonder if such a belief is justified, or even if it is really believed by those who spout such platitudes.
Indeed, cases such as Enron or Worldcom (to mention just two) seem to suggest that stupidity may be fairly prevalent in present day organisations. This point is the subject of a brilliant paper by Andre Spicer and Mats Alvesson entitled, A stupidity based theory of organisations. This post is an extensive summary and review of the paper.
Background
The notion that the success of an organization depends on the intellectual and rational capabilities of its people seems almost obvious. Moreover, there is a good deal of empirical research that seems to support this. In the opening section of their paper, Alvesson and Spicer cite many studies which appear to establish that developing the knowledge (of employees) or hiring smart people is the key to success in an ever-changing, competitive environment.
These claims are mirrored in theoretical work on organizations. For example Nonaka and Takeuchi’s model of knowledge conversion acknowledges the importance of tacit knowledge held by employees. Although there is still much debate about tacit/explicit knowledge divide, models such as these serve to perpetuate the belief that knowledge (in one form or another) is central to organisational success.
There is also a broad consensus that decision making in organizations, though subject to bounded rationality and related cognitive biases, is by and large a rational process. Even if a decision is not wholly rational, there is usually an attempt to depict it as being so. Such behaviour attests to the importance attached to rational thinking in organization-land.
At the other end of the spectrum there are decisions that can only be described as being, well… stupid. As Rick Chapman discusses in his entertaining book, In Search of Stupidity, organizations occasionally make decisions that are plain dumb However, such behaviour seldom remains hidden because of its rather obvious negative consequences for the organisation. Such stories thus end up being immortalized in business school curricula as canonical examples of what not to do.
Functional stupidity
Notwithstanding the above remarks on obvious stupidity, there is another category of foolishness that is perhaps more pervasive but remains unnoticed and unremarked. Alvesson and Spicer use the term functional stupidity to refer to such “organizationally supported lack of reflexivity, substantive reasoning, and justitication.”
In their words, functional stupidity amounts to the “…refusal to use intellectual resources outside a narrow and ‘safe’ terrain.” It is reflected in a blinkered approach to organisational problems, wherein people display an unwillingness to consider or think about solutions that lie outside an arbitrary boundary. A common example of this is when certain topics are explicitly or tacitly deemed as being “out of bounds” for discussion. Many “business as usual” scenarios are riddled with functional stupidity, which is precisely why it’s often so hard to detect.
As per the definition offered above, there are three cognitive elements to functional stupidity:
- Lack of reflexivity: this refers to the inability or unwillingness to question claims and commonly accepted wisdom.
- Lack of substantive reasoning: This refers to reasoning that is based on a small set of concerns that do not span the whole issue. A common example of this sort of myopia is when organisations focus their efforts on achieving certain objectives with little or no questioning of the objectives themselves.
- Lack of justification: This happens when employees do not question managers or, on the other hand, do not provide explanations regarding their own actions. Often this is a consequence of power relationships in organisations. This may, for example, dissuade employees from “sticking their necks out” by asking questions that managers might deem out of bounds.
It should be noted that functional stupidity has little to do with limitations of human cognitive capacities. Nor does it have anything to do with ignorance, carelessness or lack of thought. The former can be rectified through education and/or the hiring of consultants with the requisite knowledge, and the latter via the use of standardised procedures and checklists.
It is also important to note that functional stupidity is not necessarily a bad thing. For example, by placing certain topics out of bounds, organisations can avoid discussions about potentially controversial topics and can thus keep conflict and uncertainty at bay. This maintains harmony, no doubt, but it also strengthens the existing organisational order which in turn serves to reinforce functional stupidity.
Of course, functional stupidity also has negative consequences, the chief one being that it prevents organisations from finding solutions to issues that involve topics that have been arbitrarily deemed as being out of bounds.
Examples of functional stupidity
There are many examples of functional stupidity in recent history, a couple being the irrational exuberance in the wake of the internet boom of the 1990s, and the lack of critical examination of the complex mathematical models that lead to the financial crisis of last decade.
However, one does not have to look much beyond one’s own work environment to find examples of functional stupidity. Many of these come under the category of “business as usual” or “that’s just the way things are done around here” – phrases that are used to label practices that are ritually applied without much thought or reflection. Such practices often remain unremarked because it is not so easy to link them to negative outcomes. Indeed, the authors point out that “most managerial practices are adopted on the basis of faulty reasoning, accepted wisdom and complete lack of evidence.”
The authors cite the example of companies adopting HR practices that are actually detrimental to employee and organisational wellbeing. Another common example is when organisations place a high value on gathering information which is then not used in a meaningful way. I have discussed this “information perversity” at length in my post on entitled, The unspoken life of information in organisations, so I won’t rehash it here. Alvesson and Spicer point out that information perversity is a consequence of the high cultural value placed on information: it is seen as a prerequisite to “proper” decision making. However, in reality it is often used to justify questionable decisions or simply “hide behind the facts.”
These examples suggest that functional stupidity may be the norm rather than the exception. This is a scary thought…but I suspect it may not be surprising to many readers.
The dynamics of stupidity
Alvesson and Spicer claim that functional stupidity is a common feature in organisations. To understand why it is so pervasive, one has to look into the dynamics of stupidity – how it is established and the factors that influence it. They suggest that the root cause lies in the fact that organisations attempt to short-circuit critical thinking through what they call economies of persuasion, which are activities such as corporate culture initiatives, leadership training or team / identity building, relabelling positions with pretentious titles – and many other such activities that are aimed at influencing employees through the use of symbols and images rather than substance. Such symbolic manipulation, as the authors calls it, is aimed at increasing employees’ sense of commitment to the organisation.
As they put it:
Organizational contexts dominated by widespread attempts at symbolic manipulation typically involve managers seeking to shape and mould the ‘mind-sets’ of employees . A core aspect of this involves seeking to create some degree of good faith and conformity and to limit critical thinking
Although such efforts are not always successful, many employees do buy in to them and thereby identify with the organisation. This makes employees uncritical of the organisation’s goals and the means by which these will be achieved. In other words, it sets the scene for functional stupidity to take root and flourish.
Stupidity management and stupidity self-management
The authors use the term stupidity management to describe managerial actions that prevent or discourage organisational actors (employees and other stakeholders) from thinking for themselves. Some of the ways in which this is done include the reinforcement of positive images of the organisation, getting employees to identify with the organisation’s vision and myriad other organisational culture initiatives aimed at burnishing the image of the corporation. These initiatives are often backed by organisational structures (such as hierarchies and reward systems) that discourage employees from raising and exploring potentially disruptive issues.
The monitoring and sanctioning of activities that might disrupt the positive image of the organisation can be overt (in the form of warnings, say). More often, though, it is subtle. For example, in many meetings, participants participants know that certain issues cannot be raised. At other times, discussion and debate may be short circuited by exhortations to “stop thinking and start doing.” Such occurrences serve to create an environment in which stupidity flourishes.
The net effect of managerial actions that encourage stupidity is that employees start to cast aside their own doubts and questions and behave in corporately acceptable ways – in other words, they start to perform their jobs in an unreflective and unquestioning way. Some people may actually internalise the values espoused by management; others may psychologically distance themselves from the values but still act in ways that they are required to. The net effect of such stupidity self-management (as the authors call it) is that employees stop questioning what they are asked to do and just do it. After a while, doubts fade and this becomes the accepted way of working. The end result is the familiar situation that many of us know as “business as usual” or “that’s just the way things are done around here.”
The paradoxes and consequences of stupidity
Functional stupidity can cause both feelings of certainty and dissonance in members of an organisation. Suppressing critical thinking can result in an easy acceptance of the way things are. The feelings of certainty that come from suppressing difficult questions can be comforting. Moreover, those who toe the organisational line are more likely to be offered material rewards and promotions than those who don’t. This can act to reinforce functional stupidity because others who see stupidity rewarded may also be tempted to behave in a similar fashion.
That said, certain functionally stupid actions, such as ignoring obvious ethical lapses, can result in serious negative outcomes for an organisation. This has been amply illustrated in the recent past. Such events can prompt formal inquiries at the level of the organisation, no doubt accompanied by informal soul-searching at the individual level. However, as has also been amply illustrated, there is no guarantee that inquiries or self-reflection lead to any major changes in behaviour. Once the crisis passes, people seem all too happy to revert to business as usual.
In the end , though, when stark differences between the rhetoric and reality of the organisation emerge – as they eventually will– employees will see the contradictions between the real organisation and the one they have been asked to believe in. This can result in alienation from and cynicism about the organisation and its objectives. So, although stupidity management may have beneficial outcomes in the short run, there is a price to be paid in the longer term.
Nothing comes for free, not even stupidity…
Conclusion
The authors main message is that despite the general belief that organisations enlist the cognitive and intellectual capacities of their members in positive ways, the truth is that organisational behaviour often exhibits a wilful ignorance of facts and/or a lack of logic. The authors term this behaviour functional stupidity.
Functional stupidiy has the advantage of maintaining harmony at least in the short term, but its longer term consequences can be negative. Members of an organisation “learn” such behaviour by becoming aware that certain topics are out of bounds and that they broach these at their own risk. Conformance is rewarded by advancement or material gain whereas dissent is met with overt or less obvious disciplinary action. Functional stupidity thus acts as a barrier that can stop members of an organisation from developing potentially interesting perspectives on the problems the organisations face.
The paper makes an interesting and very valid point about the pervasiveness of wilfully irrational behaviour in organisations. That said, I can’t help but think that the authors have written it with tongue firmly planted in cheek.
Eight to Late 