“The Heretic’s Guide to Best Practices” wins bronze at the 5th Annual Axiom Business Book Awards.
I’m delighted to announce that the book that Paul Culmsee and I published recently has been awarded a bronze medal at the Axiom Business Book Awards for 2012, under the category Operations Management/Lean/Continuous Improvement.
We are truly honoured that the panel found our efforts worthy of an award.
If you are interested in finding out more about the book, please check out the review by Shim Marom and the one by Scott McCrickard.
There are also a number of customer reviews on Amazon.
The Heretic’s Guide is a self-published book with no big publisher marketing behind it, so we’d greatly appreciate your spreading the word!
On the unintended consequences of organisational change
Introduction
Change, as the cliché goes, is the only constant. At any given time, most organisations are either planning or implementing changes of some kind. Perhaps because of its ubiquity, the rationale and results of change are not questioned as deeply as they ought to be. In this post I describe some unintended effects of organisational change, drawing on Barbara Czarniawska’s book, A Theory of Organizing and other sources. I also briefly discuss some ways in which these side effects can be avoided.
I’ll begin with a few words about terminology. In this article planned changes (also referred to as reforms) are changes instituted in order to achieve specific goals. The goals of reforms are referred to as planned effects – that is, planned effects are intended results of change. As I discuss below, although planned effects may eventually be achieved, change initiatives have a host of unforeseen but significant consequences. These are referred to as unplanned, unintended or side effects.
This article is organised as follows: I’ll begin by describing some of the positive and negative side effects of change, following which I’ll discuss why side effects come about and how they can be managed.
Advantageous side effects of change
Although, the term side effect has a negative connotation, some side effects of change can actually be advantageous. These include:
- Questioning of the status quo: In most organisations, processes and structures are taken for granted, rarely is the status quo questioned. Organisational change presents an opportunity to pose those “How can we do this better?” type questions that challenge the way things are done. Such questioning is unplanned in that it generally occurs spontaneously.
- Opportunities for reflection: This is a consequence of the previous point: questioning the status quo can cause people to reflect on how things can be done better. Again, this is an unintended consequence of a reform, not part of its planned goals. Also, it should be noted that although opportunities for reflection arise often, they are generally ignored because of time pressures.
- Spontaneous inventions: Finally, questioning of and reflecting on the status quo can trigger ideas for improvement.
Most people would agree that the above points are indeed Good Things that ought to be encouraged. However, the important point is that people who are in the throes of a planned change seldom have the time or motivation to pursue these opportunities.
Harmful side effects of change
The negative side effects of planned changes are insidious because they tend to occur as a result of inaction – i.e. by not taking corrective actions to counter the detrimental effects of change. The following side effects serve to illustrate this point:
- The aims of reform become cast in stone: The objectives of a change initiative are formulated based on an understanding of a situation as it exists at a particular point in time. Problem is, as time evolves the original objectives maybecome irrelevant or obsolete. Yet, in many (most?) change initiatives, objectives are rarely reviewed and adjusted.
- The means get confused with the ends: Following from the previous point, a change initiative becomes pointless when its objectives are no longer relevant. However, a common reaction in such situations is to continue the initiative, justifying it as a worthwhile end in itself. For example, if the benefits of, say, a restructuring initiative become moot, the restructuring itself becomes the objective rather than the benefits that were supposed to flow from it. This helps save face as the project can be declared a success once the restructuring is completed, regardless of whether or not the promised benefits are realised.
- Improvisations and spontaneous inventions are suppressed: As I have discussed at length in this post, planning and improvisation are complementary but contradictory aspects of organizational work. A negative aspect of planned change initiatives is that they are inimical to improvisations: those responsible for overseeing the change tend to ignore, even suppress any improvisations that arise because they are seen as getting in the way of achieving the objectives of the primary change.
Planned change initiatives are generally implemented through programs or projects. In fact, most major projects in organisations – restructurings, enterprise system implementations etc – are aimed at implementing reforms of some kind. However, although the raison d’etre of such projects is to achieve the planned objectives, many suffer from the negative side effects mentioned above. In her book Czarniawska states, “Planned change rarely, if ever, leads to planned effects.” Although this claim may be a tad exaggerated, the significant proportion of large projects that fail suggests there is at least a whiff of truth about it.
In the next two sections I take a brief look at why planned changes fail and what can be done about it.
The origin of the side effects of change
Most structures and processes within organisations have a complex, path-dependent history. Among other things, they develop in ways that are unique to an organisation and are often deeply intertwined with each other. As a result, it is impossible to be certain about the consequences of changing processes or structures – there are just too many variables and dependencies involved.
There are two related points that flow from this:
Firstly, those who plan changes need to have a good understanding of legacy: the history of the issues that the change aims to fix and those that it may create in the future. The problem is most of the people involved in planning, initiating and executing reforms have little appreciation of such issues.
Secondly, most major changes are conceived by a small number of people who hold positions of authority within organisations. These folks have a tendency to gloss over complexities, and often fail to involve those who have a detailed knowledge of the affected processes and structures. Consequently, their plans overlook dependencies and possible knock-on effects that can arise from them. This results in the negative side effects discussed in the previous section.
..and what can be done about them
Czarniawska recommends the following informal rules for successful change:
- Be willing to modify the objectives of the change and your path to get there as your understanding of it evolves.
- Implement lightweight processes, avoid bureaucratic procedures.
- Be open to improvisations.
This is good advice as it goes, but how exactly does one use it?
In our recently published book, The Heretic’s Guide to Best Practices, Paul Culmsee and I discuss how issues of legacy and lack of inclusiveness can be addressed.
Firstly, we suggest that apart from time, cost and scope (the classic iron triangle), project decision-makers would be well served by considering legacy as a separate variable in projects (also see this post on Paul’s blog for more on this point). More importantly, we describe techniques that can be used to surface hidden assumptions and aspects of history that could have a bearing on the project and those that might cause problems in the future.
Secondly, we discuss how one can work towards creating an environment in which a diverse group of stakeholders can air and reconcile their viewpoints. Such a discussion is a prerequisite to creating a plan that: a) considers as many viewpoints (variables) as possible and b) has the support of all stakeholders. Without this, any implementation is bound to have side-effects because of overlooked variables and/or the actions (or non-actions) of stakeholders who do not support the plan.
Of course, inclusiveness sounds great but it can be difficult in practice, especially in large organisations. What can decision-makers do in such cases? The answer comes from a slightly different, if rather obvious direction.
In his very illuminating book on decision-making, James March notes that organisations face messy and inconsistent environments. Given this, decisions made and implemented at lower levels have a better chance of success than those made in rarefied air of board-rooms. Paraphrasing a statement from his book:
Since knowledge of local conditions and specialized competencies are both essential and more readily found in decentralized units, control over the details of policy implementation and adaptation of general policies to local conditions are [best] delegated to local units. From the standpoint of general management, the strategy is usually seen as one of gaining the informational and motivational advantages of using people with local involvement, [but] at the cost of accentuating problems of central coordination and control.
Indeed, most of the nasty side effects of planned change arise from over-centralisation of coordination and control. The solution is to devolve control and decision-making authority down to the level at which the changes are to be implemented.
Conclusion
Planned change fails to achieve its goals because planners cannot foresee all the consequences of change or even know which factors may be important in determining these. Moreover, individuals will view changes through the lens of their background, biases and interests. Since organisations consist of many individuals with different views, managing change is essentially a wicked problem.
To sum up, those who initiate large-scale changes should keep in mind the law of unintended consequences: any planned action will have consequences that are not intended, or even foreseen. These consequences can be managed by getting a better appreciation of the factors that affect the processes and the structures to be changed. One can gain an understanding of these factors through a consideration of legacy and/or via dialogue involving all those who work with the processes and structures that are to be changed. The simplest way to achieve both is by delegating decision making and implementation authority down to where it belongs – with the people who work at the coalface of the organisation.
On the nature of decision-making in organisations
Introduction
Decision-making is a key activity at all levels in an organisation. All employees make decisions: from the front-line employee who has to decide how to handle a difficult customer to an executive who has to choose between projects that are competing for funding. Given this it is no surprise that a vast body of knowledge – decision theory – has been developed to support the process of decision making.
Decision theory concerns itself with rational decision making– that is, decisions that are based on an objective evaluation of available options and their consequences, leading to a choice that is made on the basis of such an evaluation alone. In reality, though, many decisions are not made this way. In this post I look at the different ways in which decisions are actually made in organisations, drawing on a brilliant essay by James March entitled, How Decisions Happen in Organizations.
Decision making as a rational process
The standard view of decision making is that it is a process of rational choice based on:
- Knowledge of alternatives
- Knowledge of the consequences of each of the alternatives
- Ordered preferences by which consequences can be evaluated
- Rule(s) by which a particular alternative can be selected
In its basic form, decision theory assumes that each of the above is fully known. As March states:
In the most familiar form of the model, we assume that all alternatives, the probability distribution of consequences conditional on each alternative, and the subjective value of each possible consequence are known; we assume a choice is made by selecting the alternative with the highest expected value. This emphasis on expected value may be moderated by a risk preference (i.e.,some value associated with the variability of the outcome distribution).
However, there are a number of challenges to this ideal picture of decision-making. These include:
- Uncertainty about consequences of actions: The standard theory of rational choice assumes that decision-makers have knowledge of all possible outcomes of actions. However, this is not possible because humans are boundedly rational – their ability to seek and process information is limited by their cognitive abilities and available resources. Quite often it happens that consequences reveal themselves only after a decision has been made and implemented. As March mentions, “…management requires tolerance of the idea that the meaning of yesterday’s action will be discovered in the experiences and interpretations of today…”
- Uncertainty about preferences: The standard theory assumes that preferences are stable and consistent. Quite often, it happens that preferences change with time and different preferences can be inconsistent with each other.
- The role of risk: Typically, in theories of rational decision making risk appetite (of an individual or organisation) is treated as a single fixed number. In reality, it varies with situational factors such as level of threat to survival, excess resources available etc. Moreover, it also depends on the (often unarticulated) hopes and fears of individuals who are making the decision.
- Conflict between decision makers: Rational theories of decision making assume that conflict between decision makers can be resolved by (rationally!) evaluating conflicting alternatives and choosing the best one based on an agreed decision rule. The problem is that in such situations it is often impossible to come up with such a decision rule. Negotiations over criteria can go on interminably and conclude without agreement. March suggests that the reasons why decisions get made despite this is that people rely on trust and reputation rather than formal agreements in order to reach a consensus.
So as we see, the rational view of decision making has less practical relevance than one might expect. It is part of the story of decision making, but definitely not the whole tale.
Decision making as a rule-based activity
An alternate logic of decision making is that of following rules, obligations and duties; doing what is appropriate rather than what is rational. As March puts it:
Much of the decision-making behavior we observe reflects the routine way in which people do what they are supposed to do. For example, most of the time, the majority of people in organizations follow rules, even when it is not obviously in their self-interest to do so. Much of the behavior in an organization is specified by standard operating procedures, professional standards, cultural norms, and institutional structures. The terminology is one of duties and roles rather than anticipatory, consequential choice.
Within a logic of appropriateness, people make decisions by mapping the aspects of the decision they are required to make to what is appropriate in such situations. In particular, they consider the following:
- Situation: what kind of a situation is this?
- Identity: who am I? What kind of position do I hold in the organisation?
- Determining an appropriate choice: What should a person like me (or in my position) do in this kind of situation?
In such a process the focus is on doing what is right (as per the rules) rather than searching for rationally determined best choice.
The interesting question is how these rules come into existence. March describes three ways in which this happens:
- Rules are developed through experience and are modified by feedback on what worked well and what didn’t. In this view organisations create rules.
- Rules are selected (rather than developed) based on their suitability for a group or organisation. In this view, rules have an existence independent of organisations.
- Rules spread from organisation to organisation – much like “fads or measles.” In this view, rules are created in idiosyncratic ways (through an innovative or quirky choices made by an individual, say) and then, if they are successful, are copied others. Many popular management practices have their roots in such fads.
Summarising: decisions can be based on appropriate choices rather than rational ones.
Decision making as a contingent event
The views of decision making embodied in the logic of rationality and appropriateness assume that the cause-effect relationships between decisions and outcomes are well understood and that organizational rules and hierarchies actually control outcomes. However, in reality things tend to be less straightforward. For example:
- Many things happen at the same time, each competing for the attention of decision makers. The attention a decision maker gives to a problem thus depends on the other things that are on her mind at the time.
- Individual perceptions of situations vary, thus making the formulation of a decision problem difficult (in effect, making it a wicked problem).
In cases such as these, the decisions are contingent on factors that have nothing to do with the decision itself. As examples, an executive who is distracted by personal problems may not give enough attention to a decision problem at hand and a bunch of stakeholders who cannot agree may end up making a decision that cannot be justified via rationality or appropriateness.
Decision making as a byproduct of other factors
The assumption underlying the foregoing discussion is that decisions affect outcomes and hence that decisions matter. However, as March points out:
Descriptions of decision arenas often seem to make little sense in such terms. Information that is ostensibly gathered for decisions is often ignored. Contentiousness of the policies of an organization is often followed by apparent indifference about their implementation. Individuals fight for the right to participate in decision processes, but then do not exercise the right. Studies of managers consistently indicate that very little time is spent making decisions. Rather, managers seem to spend time meeting people and executing managerial performances.
Based on the above, March makes the interesting point decision making is often a ritual activity that has little to do with the actual decision itself. The process of making a choice provides decision makers opportunities to do other things such as:
- Presenting and justifying their viewpoints to their peers.
- Distributing credit or blame for what has occurred.
- Reaffirming loyalties and friendships
- Socialising!
An aspect of decision making made highlighted in the previous section is that there are many competing demands on a decision maker’s attention – for example, family, friends or personal goals. This is true in general: in the course of our lives, we are presented with a steady stream of choices, opportunities and problems. The degree to which each of these hold our attention depends on a host of factors including (but not restricted to) our values, duties and priorities. Because of these concurrent or nearly concurrent issues, the attention we give to a decision problem is closely linked to events that have recently occurred or are anticipated in the near future, and the priorities we assign to these. In such situations, the logic of decision making is temporal (dictated by time) rather than consequential or rule-based. In other words, our decisions depend on recent events and our immediate (or recent) environment.
Conclusion
In this article I have summarised various views of decision making drawing on the work of James March. We have seen that the official line about decision making being a rational process that is concerned with optimizing choices on the basis of consequences and preferences is not the whole story. Our decisions are influenced by a host of other factors, ranging from the rules that govern our work lives to our desires and fears, or even what happened at home yesterday. In short: the choices we make often depend on things we are only dimly aware of.
Eight to Late 