Archive for the ‘Project Management’ Category
On the unintended consequences of organisational change
Introduction
Change, as the cliché goes, is the only constant. At any given time, most organisations are either planning or implementing changes of some kind. Perhaps because of its ubiquity, the rationale and results of change are not questioned as deeply as they ought to be. In this post I describe some unintended effects of organisational change, drawing on Barbara Czarniawska’s book, A Theory of Organizing and other sources. I also briefly discuss some ways in which these side effects can be avoided.
I’ll begin with a few words about terminology. In this article planned changes (also referred to as reforms) are changes instituted in order to achieve specific goals. The goals of reforms are referred to as planned effects – that is, planned effects are intended results of change. As I discuss below, although planned effects may eventually be achieved, change initiatives have a host of unforeseen but significant consequences. These are referred to as unplanned, unintended or side effects.
This article is organised as follows: I’ll begin by describing some of the positive and negative side effects of change, following which I’ll discuss why side effects come about and how they can be managed.
Advantageous side effects of change
Although, the term side effect has a negative connotation, some side effects of change can actually be advantageous. These include:
- Questioning of the status quo: In most organisations, processes and structures are taken for granted, rarely is the status quo questioned. Organisational change presents an opportunity to pose those “How can we do this better?” type questions that challenge the way things are done. Such questioning is unplanned in that it generally occurs spontaneously.
- Opportunities for reflection: This is a consequence of the previous point: questioning the status quo can cause people to reflect on how things can be done better. Again, this is an unintended consequence of a reform, not part of its planned goals. Also, it should be noted that although opportunities for reflection arise often, they are generally ignored because of time pressures.
- Spontaneous inventions: Finally, questioning of and reflecting on the status quo can trigger ideas for improvement.
Most people would agree that the above points are indeed Good Things that ought to be encouraged. However, the important point is that people who are in the throes of a planned change seldom have the time or motivation to pursue these opportunities.
Harmful side effects of change
The negative side effects of planned changes are insidious because they tend to occur as a result of inaction – i.e. by not taking corrective actions to counter the detrimental effects of change. The following side effects serve to illustrate this point:
- The aims of reform become cast in stone: The objectives of a change initiative are formulated based on an understanding of a situation as it exists at a particular point in time. Problem is, as time evolves the original objectives maybecome irrelevant or obsolete. Yet, in many (most?) change initiatives, objectives are rarely reviewed and adjusted.
- The means get confused with the ends: Following from the previous point, a change initiative becomes pointless when its objectives are no longer relevant. However, a common reaction in such situations is to continue the initiative, justifying it as a worthwhile end in itself. For example, if the benefits of, say, a restructuring initiative become moot, the restructuring itself becomes the objective rather than the benefits that were supposed to flow from it. This helps save face as the project can be declared a success once the restructuring is completed, regardless of whether or not the promised benefits are realised.
- Improvisations and spontaneous inventions are suppressed: As I have discussed at length in this post, planning and improvisation are complementary but contradictory aspects of organizational work. A negative aspect of planned change initiatives is that they are inimical to improvisations: those responsible for overseeing the change tend to ignore, even suppress any improvisations that arise because they are seen as getting in the way of achieving the objectives of the primary change.
Planned change initiatives are generally implemented through programs or projects. In fact, most major projects in organisations – restructurings, enterprise system implementations etc – are aimed at implementing reforms of some kind. However, although the raison d’etre of such projects is to achieve the planned objectives, many suffer from the negative side effects mentioned above. In her book Czarniawska states, “Planned change rarely, if ever, leads to planned effects.” Although this claim may be a tad exaggerated, the significant proportion of large projects that fail suggests there is at least a whiff of truth about it.
In the next two sections I take a brief look at why planned changes fail and what can be done about it.
The origin of the side effects of change
Most structures and processes within organisations have a complex, path-dependent history. Among other things, they develop in ways that are unique to an organisation and are often deeply intertwined with each other. As a result, it is impossible to be certain about the consequences of changing processes or structures – there are just too many variables and dependencies involved.
There are two related points that flow from this:
Firstly, those who plan changes need to have a good understanding of legacy: the history of the issues that the change aims to fix and those that it may create in the future. The problem is most of the people involved in planning, initiating and executing reforms have little appreciation of such issues.
Secondly, most major changes are conceived by a small number of people who hold positions of authority within organisations. These folks have a tendency to gloss over complexities, and often fail to involve those who have a detailed knowledge of the affected processes and structures. Consequently, their plans overlook dependencies and possible knock-on effects that can arise from them. This results in the negative side effects discussed in the previous section.
..and what can be done about them
Czarniawska recommends the following informal rules for successful change:
- Be willing to modify the objectives of the change and your path to get there as your understanding of it evolves.
- Implement lightweight processes, avoid bureaucratic procedures.
- Be open to improvisations.
This is good advice as it goes, but how exactly does one use it?
In our recently published book, The Heretic’s Guide to Best Practices, Paul Culmsee and I discuss how issues of legacy and lack of inclusiveness can be addressed.
Firstly, we suggest that apart from time, cost and scope (the classic iron triangle), project decision-makers would be well served by considering legacy as a separate variable in projects (also see this post on Paul’s blog for more on this point). More importantly, we describe techniques that can be used to surface hidden assumptions and aspects of history that could have a bearing on the project and those that might cause problems in the future.
Secondly, we discuss how one can work towards creating an environment in which a diverse group of stakeholders can air and reconcile their viewpoints. Such a discussion is a prerequisite to creating a plan that: a) considers as many viewpoints (variables) as possible and b) has the support of all stakeholders. Without this, any implementation is bound to have side-effects because of overlooked variables and/or the actions (or non-actions) of stakeholders who do not support the plan.
Of course, inclusiveness sounds great but it can be difficult in practice, especially in large organisations. What can decision-makers do in such cases? The answer comes from a slightly different, if rather obvious direction.
In his very illuminating book on decision-making, James March notes that organisations face messy and inconsistent environments. Given this, decisions made and implemented at lower levels have a better chance of success than those made in rarefied air of board-rooms. Paraphrasing a statement from his book:
Since knowledge of local conditions and specialized competencies are both essential and more readily found in decentralized units, control over the details of policy implementation and adaptation of general policies to local conditions are [best] delegated to local units. From the standpoint of general management, the strategy is usually seen as one of gaining the informational and motivational advantages of using people with local involvement, [but] at the cost of accentuating problems of central coordination and control.
Indeed, most of the nasty side effects of planned change arise from over-centralisation of coordination and control. The solution is to devolve control and decision-making authority down to the level at which the changes are to be implemented.
Conclusion
Planned change fails to achieve its goals because planners cannot foresee all the consequences of change or even know which factors may be important in determining these. Moreover, individuals will view changes through the lens of their background, biases and interests. Since organisations consist of many individuals with different views, managing change is essentially a wicked problem.
To sum up, those who initiate large-scale changes should keep in mind the law of unintended consequences: any planned action will have consequences that are not intended, or even foreseen. These consequences can be managed by getting a better appreciation of the factors that affect the processes and the structures to be changed. One can gain an understanding of these factors through a consideration of legacy and/or via dialogue involving all those who work with the processes and structures that are to be changed. The simplest way to achieve both is by delegating decision making and implementation authority down to where it belongs – with the people who work at the coalface of the organisation.
Models and messes in management – from best practices to appropriate practices
Scientific models and management
Physicists build mathematical models that represent selected aspects of reality. These models are based on a mix of existing knowledge, observations, intuition and mathematical virtuosity. A good example of such a model is Newton’s law of gravity according to which the gravitational force between two objects (planets, apples or whatever) varies in inverse proportion to the square of the distance between them. The model was a brilliant generalization based on observations made by Newton and others (Johannes Kepler, in particular), supplemented by Newton’s insight that the force that keeps the planets revolving round the sun is the same as the one that made that mythical apple fall to earth. In essence Newton’s law tells us that planetary motions are caused by gravity and it tells us – very precisely – the effects of the cause. In short: it embodies a cause-effect relationship.
[Aside: The validity of a physical model depends on how well it stands up to the test of reality. Newton’s law of gravitation is remarkably successful in this regard: among many other things, it is the basis of orbital calculations for all space missions. The mathematical model expressed by Newton’s law is thus an established scientific principle. That said, it should be noted that models of the physical world are always subject to revision in the light of new information. For example, Newton’s law of gravity has been superseded by Einstein’s general theory of relativity. Nevertheless for most practical applications it remains perfectly adequate.]
Given the spectacular success of modeling in the physical and natural sciences, it is perhaps unsurprising that early management theorists attempted to follow the same approach. Fredrick Taylor stated this point of view quite clearly in the introduction to his classic monograph, The Principles of Scientific Management. Here are the relevant lines:
This paper has been written…to prove that the best management is a true science, resting upon clearly defined laws, rules and principles, as a foundation. And further to show that the fundamental principles of scientific management are applicable to all human activities, from our simplest individual activities to the work of great corporations, which call for the most elaborate cooperation. And briefly, through a series of illustrations, to convince the reader that whenever these principles are correctly applied, results must follow which are truly astounding…
From this it appears that Taylor’s intent was to prove that management could be reduced to a set of principles that govern all aspects of work in organizations.
The question is: how well did it work?
The origin of best practices
Over time, Taylor’s words were used to justify the imposition of one-size-fits-all management practices that ignored human individuality and uniqueness of organisations. Although, Taylor was aware of these factors, he believed commonalities were more important than differences. This thinking is well and alive to this day: although Taylor’s principles are no longer treated as gospel, their spirit lives on in the notion of standardized best practices.
There are now a plethora of standards or best practices for just about any area of management. They are often sold using scientific language, terms such as principles and proof. Consider the following passage taken from from the Official PRINCE2 site:
Because PRINCE2 is generic and based on proven principles, organisations adopting the method as a standard can substantially improve their organisational capability and maturity across multiple areas of business activity – business change, construction, IT, mergers and acquisitions, research, product development and so on.
There are a couple of other things worth noting in the above passage. First, there is an implied cause-effect relationship between the “proven principles” and improvements in “organizational capability and maturity across multiple areas of business activity.” Second, as alluded to above, the human factor is all but factored out – there is an implication that this generic standard can be implemented by anyone anywhere and the results will inevitably be as “truly astounding” as Taylor claimed.
Why best practices are not the best
There are a number of problems with the notion of a best practice. I discuss these briefly below.
First, every organisation is unique. Yes, much is made of commonalities between organisations, but it is the differences that make them unique. Arguably, it is also the differences that give organisations their edge. As Stanley Deetz mentioned in his 2003 Becker lecture:
In today’s world unless you have exceptionally low labor costs, competitive advantage comes from high creativity, highly committed employees and the ability to customize products. All require a highly involved, participating workforce. Creativity requires letting differences make a difference. Most high-end companies are more dependent on the social and intellectual capital possessed by employees than financial investment.
Thoughtless standardization through the use of best practices is a sure way to lose those differences that could make a difference.
Second, in their paper entitled, De-Contextualising Competence: Can Business Best Practice be Bundled and Sold, Jonathan Wareham and Han Gerrits pointed out that organisations operate in vastly varying cultural and social environments. It is difficult to see how best practice approaches with their one-and-a-half-size –fits-all approach would work.
Third , Wareham and Gerrits also pointed out that best practice is often tacit and socially embedded. This invalidates the notion that it can be transferred from an organization in which it works and to another without substantial change. Context is all important.
Lastly, best practices are generally implemented in response to a perceived problem. However, they often address the symptoms rather than the root cause of the problem. For example, a project management process may attempt to improve delivery by better estimation and planning. However, the underlying cause – which may be poor communication or a dysfunctional relationship between users and the IT department –remains unaddressed.
In his 2003 Becker lecture, Stanley Deetz illustrated this point via the following fable:
… about a company formed by very short people. Since they were all short and they wanted to be highly efficient and cut costs, they chose to build their ceiling short and the doorways shorter so that they could have more work space in the same building. And, they were in fact very successful. As they became more and more successful, however, it became necessary for them to start hiring taller people. And, as they hired more and more tall people, they came to realize that tall people were at a disadvantage at this company because they had to walk around stooped over. They had to duck to go through the doorways and so forth. Of course, they hired organizational consultants to help them with the problem.
Initially they had time-and-motion experts come in. These experts taught teams of people how to walk carefully. Tall members learned to duck in stride so that going through the short doors was minimally inconvenient. And they became more efficient by learning how to walk more properly for their environment. Later, because this wasn’t working so well, they hired psychological consultants. These experts taught greater sensitivity to the difficulties of tall members of the organization. Long-term short members learned tolerance knowing that the tall people would come later to meetings, would be somewhat less able to perform their work well. They provided for tall people networks for support…
The parable is an excellent illustration of how best practices can end up addressing symptoms rather than causes.
Ambiguity + the human factor = a mess
Many organisational problems are ambiguous in that cause-effect relationships are unclear. Consequently, different stakeholders can have wildly different opinions as to what the root cause of a problem is. Moreover, there is no way to conclusively establish the validity of a particular point of view. For example, executives may see a delay in a project as being due to poor project management whereas the project manager might see it as being a consequence of poor scope definition or unreasonable timelines. The cause depends on who you ask and there is no way to establish who is right! Unlike problems in physics, organisational problems have a social dimension.
The visionary Horst Rittel coined the evocative term wicked problem to describe problems that involve many stakeholder groups with diverse and often conflicting perspectives. This makes such problems messy. Indeed, Russell Ackoff referred to wicked problems as messes. In his words, “every problem interacts with other problems and is therefore part of a set of interrelated problems, a system of problems…. I choose to call such a system a mess”
Consider an example that is quite common in organisations: the question of how to improve efficiency. Management may frame this issue in terms of tighter managerial control and launch a solution that involves greater oversight. In contrast, a workgroup within the organisation may see their efficiency being impeded by bureaucratic control that results from increased oversight, and thus may believe that the road to efficiency lies in giving workgroups greater autonomy. In this case there is a clear difference between the aims of management (to exert greater control) and those of workgroups (to work autonomously). Ideally, the two ought to talk it over and come up with a commonly agreed approach. Unfortunately they seldom do. The power structure in organisations being what it is, management’s solution usually prevails and, as a consequence, workgroup morale plummets. See this post for an interesting case study on one such situation.
Summing up: a need for appropriate practice, not best practice
The great attraction of best practices, and one of the key reasons for their popularity, is that they offer apparently straightforward solutions to complex problems. However, such problems typically have a social dimension because they affect different stakeholders in different ways. They are messes whose definition depends on who you ask. So there is no agreement on what the problem is, let alone its solution. This fact by itself limits the utility of the best practice approach to organisational problem solving. Purveyors of best practices may use terms like “proven”, “established”, “measurable” etc. to lend an air of scientific respectability to their wares, but the truth is that unless all stakeholders have a shared understanding of the problem and a shared commitment to solving it, the practice will fail.
In our recently published book entitled, The Heretic’s Guide to Best Practices, Paul Culmsee and I describe in detail the issues with the best practice approach to organisational problem-solving. More important, we provide a practical approach that can help you work with stakeholders to achieve a shared understanding of a problem and a shared commitment to a commonly agreed course of action. The methods we discuss can be used in small settings or larger one, so you will find the book useful regardless of where you sit in your organisation’s hierarchy. In essence our book is a manifesto for replacing the concept of best practice with that of appropriate practice – practice with a human face that is appropriate for you in your organisation and particular situation.
Book Announcement: The Heretic’s Guide to Best Practices
After two years of chewing up most of my free time, I’m delighted and relieved (in about equal measure) to announce that the book I’ve written with Paul Culmsee is finally out: The Heretics Guide to Best Practices is now available through Amazon (also on Kindle) and iUniverse.
In this post I present the first couple of paragraphs from the book to give you a flavour of the content and style. I also include some quotes from reviewers who read drafts of the book.
The first few paragraphs
The following lines are taken from the start of the book:
Have you ever noticed that infomercials trying to sell you the latest ab-sculpting, fat burning, home fitness device with three easy credit card payments, always start with questions designed in such a way that the answer is invariably “Yes”? We have too—so as a tribute to these infomercials, we are starting this book with some seriously loaded questions.
- Have you ever had the feeling that something is not quite right in your workplace, yet you cannot articulate why?
- Are you required to perform tasks that you instinctively feel are of questionable value?
- Have you ever questioned an approach, only to be told that it is a best practice and therefore cannot be questioned?
- Have you ever sighed and blamed the ills of your organisation on “culture” or “that’s just the way things are done here”?
- Have you ever lamented to others that “If only we got ourselves organised”, we would stop chasing our tails and being so reactive?
If you answered “No” to these questions then, seriously—you are holding the wrong book. What’s more, if you manage staff and you answered ”No” to these questions, then chances are your staff gave you this book to read in the hope that you might learn a few home truths.
For those who said a hopefully emphatic “Yes!”—and we are hoping that’s a fair chunk of our readers—this book might offer you some answers and put some names to some of the things that make your organisational “spider senses” tingle. Bear in mind, you are not going to get any glib “Seven Steps to Organisational Nirvana” type stuff here. Instead, you are about to undertake a varied and, at times, heretical journey into the fun-filled world of organisational problem solving. Not only will this book provide you with some juicy ammunition in relation to organisational debates about the validity of best practices, but the practical tools and approaches that we cover might also give you some insights in how to improve things.
Quotes from reviewers
Here are some pre-publication quotes from reviewers who read draft versions of the book:
“In Paul and Kailash I have found kindred spirits who understand how messed up most organizations are, and how urgent it is that organizations discover what Buddhists call ‘expedient means’—not more ‘best practices’ or better change management for the enterprise, but transparent methods and theories that are simple to learn and apply, and that foster organizational intelligence as a natural expression of individual intelligence. This book is a bold step forward on that path, and it has the wonderful quality, like a walk at dawn through a beautiful park, of presenting profound insights with humor, precision, and clarity.”
—Jeff Conklin, Director, Cognexus Institute
—
“Hugely enjoyable, deeply reflective, and intensely practical. This book is about weaving human artistry and improvisation, with appropriate methods and technologies, in order to pool collective intelligence and wisdom under pressure.”
—Simon Buckingham Shum, Knowledge Media Institute, The Open University, UK.
—
“This is a terrific piece of work: important, insightful, and very entertaining. Culmsee and Awati have produced a refreshing take on the problems that plague organisations, the problems that plague attempts to fix organisations, and what can be done to make things better. If you’re trying to deal with wicked problems in your organisation, then drop everything and read this book.”
—Tim Van Gelder, Principal Consultant, Austhink Consulting
—
“This book has been a brilliantly fun read. Paul and Kailash interweave forty years of management theory using entertaining and engaging personal stories. These guys know their stuff and demonstrate how it can be used via real world examples.
As a long time blogger, lecturer and consultant/practitioner I have always been served well by contrarian approaches, and have sought stories and case studies to understand the reasons why my methods have worked. This book has helped me understand why I have been effective in dealing with complex business problems. Moreover, it has encouraged me to delve into the foundations of various management practices…”
—Craig Brown, Director, Evaluator
—
“Paul and Kailash have written a book that largely mirrors what I have learned through my own (sometimes painful) experience: at the foundation of every technical solution there should be a clear understanding of the business problem. It amazes me how many projects proceed without this basic planning building block, and yet the percentage of projects that fail remains fairly constant. This book provides an informative and entertaining look at the role of the business analyst, with guidance on how to improve your problem-solving skills.”
—Christian Buckley, Director, Product Evangelism at Axceler
—
“Paul and Kailash have done a fantastic job of pulling together many areas of research and presenting this in an accessible and compelling way. They walk you through their discovery process, helping you gain a real understanding of way things work but more importantly why things work, and then apply these in the real world. If you have ever been told something is a best practice, you owe it to yourself to read this book.”
—Andrew Woodward, Founder and CEO, 21Apps
—
We think there is something is the book for most professionals, regardless of where they sit in their organisation’s hierarchy. But in the end it is your opinion that counts. We would love to know what you think of our effort, and look forward to hearing from you – either via a comment on our blogs or a review on bookseller websites.
Note (added on 31 Jan 2012):
Check out the customer reviews on Amazon.
Eight to Late 