Archive for the ‘Best Practice’ Category
The illusion of enterprise risk management – a paper review
Introduction
Enterprise risk management (ERM) refers to the process by which uncertainties are identified, analysed and managed from an organization-wide perspective. In principle such a perspective enables organisations to deal with risks in a holistic manner, avoiding the silo mentality that plagues much of risk management practice. This is the claim made of ERM at any rate, and most practitioners accept it as such. However, whether the claim really holds is another matter altogether. Unfortunately, most of the available critiques of ERM are written for academics or risk management experts. In this post I summarise a critique of ERM presented in a paper by Michael Power entitled, The Risk Management of Nothing.
I’ll begin with a brief overview of ERM frameworks and then summarise the main points of the paper along with some of my comments and annotations.
ERM Frameworks and Definitions
What is ERM?
The best way to answer this question is to look at a couple of well-known ERM frameworks, one from the Casualty Actuarial Society (CAS) and the other from the Committee of Sponsoring Organisations of the Treadway Commission (COSO).
CAS defines ERM as:
… the discipline by which an organization in any industry assesses, controls, exploits, finances, and monitors risks from all sources for the purpose of increasing the organization’s short- and long-term value to its stakeholders.
See this article for an overview of ERM from actuarial perspective.
COSO defines ERM as:
…a process, effected by an entity’s board of directors, management and other personnel, applied in strategy setting and across the enterprise, designed to identify potential events that may affect the entity, and manage risk to be within its risk appetite, to provide reasonable assurance regarding the achievement of entity objectives.
The term risk appetite in the above definition refers to the risk an organisation is willing to bear. See the first article in the June 2003 issue of Internal Auditor for more on the COSO perspective on ERM.
In both frameworks, the focus is very much on quantifying risks through (primarily) financial measures and on establishing accountability for managing these risks in a systematic way.
All this sounds very sensible and uncontroversial. So, where’s the problem?
The problems with ERM
The author of the paper begins with the observation that the basic aim of ERM is to identify risks that can affect an organisation’s objectives and then design controls and mitigation strategies that reduce these risks (collectively) to below a predetermined value that is specified by the organisation’s risk appetite. Operationally, identified risks are monitored and corrective action is taken when they go beyond limits specified by the controls, much like the operation of a thermostat.
In this view, risk management is a mechanistic process. Failures of risk management are seen more as being due to “not doing it right” (implementation failure) or politics getting in the way (organizational friction), rather than a problem with the framework itself. The basic design of the framework is rarely questioned.
Contrary to common wisdom, the author of the paper believes that the design of ERM is flawed in the following three ways:
- The idea of a single, organisation-wide risk appetite is simplistic.
- The assumption that risk can be dealt with by detailed, process-based rules (suitable for audit and control) is questionable.
- The undue focus on developing financial metrics and controls blind it to “bigger picture”, interconnected risks because these cannot be quantified or controlled by such mechanisms.
We’ll now take a look at each of the above in some detail
Appetite vs. appetisation
As mentioned earlier, risk appetite is defined as the risk the organisation is willing to bear. Although ERM frameworks allow for qualitative measures of risk appetite, most organisations implementing ERM tend to prefer quantitative ones. This is a problem because the definition of risk appetite can vary significantly across an organization. For example, the sales and audit functions within an organisation could (will!) have different appetites for risk. As another example, familiar to anyone who reads the news, is that there is usually a big significant gap between the risk appetites of financial institutions and regulatory authorities.
The difference in risk appetites of different stakeholder groups is a manifestation of the fact that risk is a social construct – different stakeholder groups view a given risk in different ways, and some may not even see certain risks as risks (witness the behaviour of certain financial “masters of the universe”)
Since a single, organisation-wide risk appetite is difficult to come up with, the author suggests a different approach – one which takes into account the multiplicity of viewpoints in an organisation; a process he calls “risk appetizing”. This involves getting diverse stakeholders to achieve a consensus / agreement on what constitutes risk appetite. Power argues that this process of reconciling different viewpoints of risk would lead to a more realistic view of the risk the organization is willing to bear. Quoting from the paper:
Conceptualising risk appetising as a process might better direct risk management attention to where it has likely been lacking, namely to the multiplicity of interactions which shape operational and ethical boundaries at the level of organizational practice. COSO-style ERM principles effectively limit the concept of risk appetite within a capital measurement discourse. Framing risk appetite as the process through which ethics and incentives are formed and reformed would not exclude this technical conception, but would bring it closer to the insights of several decades of organization theory.
Explicitly acknowledging the diversity of viewpoints on risk is likely to be closer to reality because:
…a conflictual and pluralistic model is more descriptive of how organizations actually work, and makes lower demands on organizational and political rationality to produce a single ‘appetite’ by explicitly recognising and institutionalising processes by which different appetites and values can be mediated.
Such a process is difficult because it involves getting people who have different viewpoints to agree on what constitutes a sensible definition of risk appetite.
A process bias
A bigger problem, in Power’s view, is that the ERM frameworks overemphasise financial / accounting measures and processes as a means of quantifying and controlling risk. As he puts it ERM:
… is fundamentally an accounting-driven blueprint which emphasises a controls-based approach to risk management. This design emphasis means that efforts at implementation will have an inherent tendency to elaborate detailed controls with corresponding documents trails.
This is a problem because it leads to a “rule-based compliance” mentality wherein risks are managed in a mechanical manner, using bureaucratic processes as a substitute for real thought about risks and how they should be managed. Such a process may work in a make-believe world where all risks are known, but is unlikely to work in one in which there is a great deal of ambiguity.
Power makes the important point that rule-based compliance chews up organizational resources. The tangible effort expended on compliance serves to reassure organizations that they are doing something to manage risks. This is dangerous because it lulls them into a false sense of security:
Rule-based compliance lays down regulations to be met, and requires extensive evidence, audit trails and box ‘checking’. All this demands considerable work and there is daily pressure on operational staff to process regulatory requirements. Yet, despite the workload volume pressure, this is also a cognitively comfortable world which focuses inwards on routine systems and controls. The auditability of this controls architecture can be theorized as a defence against anxiety and enables organizational agents to feel that their work conforms to legitimised principles.
In this comfortable, prescriptive world of process-based risk management, there is little time to imagine and explore what (else) could go wrong. Further, the latter is often avoided because it is a difficult and often uncomfortable process:
…the imagination of alternative futures is likely to involve the production of discomfort, as compared with formal ‘comfort’ of auditing. The approach can take the form of scenario analysis in which participants from different disciplines in an organization can collectively track the trajectory of potential decisions and events. The process begins as an ‘encounter’ with risk and leads to the confrontation of limitation and ambiguity.
Such a process necessarily involves debate and dialogue – it is essentially a deliberative process. And as Power puts it:
The challenge is to expand processes which support interaction and dialogue and de-emphasise due process – both within risk management practice and between regulator and regulated.
This is right of course, but that’s not all: a lot of other process-focused disciplines such as project management would also benefit by acknowledging and responding to this challenge.
A limited view of embeddedness
One of the imperatives of ERM is to “embed” risk management within organisations. Among other things, this entails incorporating risk management explicitly into job descriptions, and making senior managers responsible for managing risks. Although this is a step in the right direction, Power argues that the concept of embeddeness as articulated in ERM remains limited because it focuses on specific business entities, ignoring the wider environment and context in which they exist. The essential (but not always obvious) connections between entities are not necessarily accounted for. As Power puts it:
ERM systems cannot represent embeddedness in the sense of interconnectedness; its proponents seem only to demand an intensification of embedding at the individual entity level. Yet, this latter kind of embedding of a compliance driven risk management, epitomised by the Sarbanes-Oxley legislation, is arguably a disaster in itself, by tying up resources and, much worse, cognition and attention in ‘auditized’ representations of business processes.
In short: the focus on following a process-oriented approach to risk management – as mandated by frameworks – has the potential to de-focus attention from risks that are less obvious, but are potentially more significant.
Addressing the limitations
Power believes the flaws in ERM can be addressed by looking to the practice of business continuity management (BCM). BCM addresses the issue of disaster management – i.e. how to keep an organisation functioning in the event of a disaster. Consequently, there is a significant overlap between the aims of BCM and ERM. However, unlike ERM, BCM draws specialists from different fields and emphasizes collective action. Such an approach is therefore more likely to take a holistic view of risk, and that is the real point.
Regardless of the approach one takes, the point is to involve diverse stakeholders and work towards a shared (enterprise-wide) understanding of risks. Only then will it be possible to develop a risk management plan that incorporates the varying, even contradictory, perspectives that exist within an organisation. There are many techniques to work towards a shared understanding of risks, or any other issues for that matter. Some of these are discussed at length in my book.
Conclusion
Power suggests that ERM, as articulated by bodies such as CAS and COSO, flawed because:
- It attempts to quantify risk appetite at the organizational level – an essentially impossible task because different organizational stakeholders will have different views of risk. Risk is a social construct.
- It advocates a controls and rule-based approach to managing risks. Such a prescriptive “best” practice approach discourages debate and dialogue about risks. Consequently, many viewpoints are missed and quite possibly, so are many risks.
- Despite the rhetoric of ERM, implemented risk management controls and processes often overlook connections and dependencies between entities within organisations. So, although risk management appears to be embedded within the organisation, in reality it may not be so.
Power suggests that ERM practice could learn a few lessons from Business Continuity Management (BCM), in particular about the interconnected nature of business risks and the collective action needed to tackle them. Indeed, any approach that attempts to reconcile diverse risk viewpoints will be a huge improvement on current practice. Until then ERM will continue to be an illusion, offering false comfort to those who are responsible for managing risk.
The king’s son – a project management fable
Once upon a time there was a king who was much loved by his people. The people loved him because he did many Good Things: he built roads for those who needed to travel long distances, houses for those who lacked a place to live and even initiated software projects to keep geeks in gainful employment.
All the Good Things the king did needed money and although the king was rich, his resources were not unlimited. Naturally, the king’s treasurer wanted to ensure that the funds flowing out of the state coffers were being put to good use.
One day, at a council meeting the treasurer summoned up his courage and asked the king, “Your highness, I know your intentions are good, but how do we know that all the money we spend is being used properly?”
“It must be so because the people are happy,” replied the king.
“Yes they are happy and that is good,” said the treasurer, “but how do we know that money we spend is not being wasted? Is it not possible that we could save money by coordinating, planning and monitoring the Good Things we do in an organized manner?”
The king (who was known to think from time to time) mulled over this for a few days.
After much mulling, he summoned his treasurer and said, “You are right. We should be more organized in the way we do all the Good Things we do. This task is so important that I will ask my second son to oversee the Good Things we do. He is, after all, a Prince Too.”
The second son (who was a Prince Too) took to his new role with relish. His first act was to set up a Governance Committee to oversee and direct all the Good Things that were being done. He ordered the board to come up with a process that would ensure that the Good Things being done would be done in an efficient and transparent way. His second act was to publish a decree, declaring that all those who did not follow the process would be summarily terminated.
Many expensive consultants and long meetings later, the Governance Committee announced they had a methodology (they could coin a word or two…) which, if followed to the letter, would ensure that all the Good Things being done were done efficiently, in a way to ensure value for the state. They had the assurance of those expensive consultants that the methodology was tested and proven so they believed this would happen as a matter of course. Moreover, the rates that the consultants charged convinced the Governance Committee that this must indeed be so.
In keeping with penchant of committees to name things, they gave the methodology the name of the king’s son (who, as we have seen earlier, was a Prince Too).
And so it came to pass that all the Good Things being done followed a process. Those who managed the Good Things and those who actually did them, underwent rigorous training in the foundations and practice of the methodology (which meant more revenue for the consultants). The planners and the doers then went out and applied the methodology in their work.
And for a while, everyone was happy: the king, the treasurer, the Governance Committee ….and of course, the Prince Too.
After sometime, however, the treasurer noticed that the flow of money out of his coffers and into the Good Things had not lessened – on the contrary, it seemed to have increased. This alarmed him, so he requested a meeting with the king’s son to discuss the matter. The king’s son, on hearing the treasurer’s tale, was alarmed too (his father would not be happy if he heard that methodology had made the matter worse…).
The king’s son summoned the Governance Committee and demanded an Explanation Now! Yes, this was how he said it, he was very, very angry.
The Governance Committee were at a loss to explain the paradox. They were using a tested and proven methodology (as the expensive consultants assured them), yet their cost of all the Good Things they were doing was rising. “What gives?” they wondered. Try as they did, they could not find an answer. After much cogitation they called in the expensive consultants and demanded an explanation.
The consultants said that the methodology was Tested and Proven. It was simply not possible that it wasn’t working. To diagnose the problem they recommended a month long audit of all the Good Things that had been done since the methodology was imposed.
The Governance Committee agreed; they had little choice (unless they preferred summary termination, which they didn’t).
The audit thus proceeded.
A month later the consultant reported back to the Governance Committee. “We know what the problem is,” they said. “Those who do Good Things aren’t following the methodology to the letter. You must understand that the benefits of the methodology will be realised only if it is implemented properly. We recommend that everyone undergoes refresher training in the methodology so that they understand it properly .”
The Governance Committee went to the treasurer, explained the situation and requested that funds be granted for refresher courses.
On hearing this, the treasurer was livid. “What? We have to spend more money to fix this problem? You must be joking.” He was very angry but he could see no other way; the consultants were the only ones who could see them out of this mess.
The money was sanctioned and the training conducted. More Good Things were done but, unfortunately, the costs did not settle down. Things, in fact, got so bad that the treasurer went directly to the king and mentioned the problem.
The king said, “Summon my second son,” he said imperiously, “I must have Words with him.”
The second son (who was a Prince Too) was summoned and arrived post-haste. His retainers had warned him that the king was very very angry.
“Father, you requested my presence?” He asked, a tad tremulously.
“Damn right, I requested your presence. I asked you to ensure that my money is being well spent on creating Good Things, and now I find that you are spending even more than we did before I put you in charge. I demand an explanation,” thundered the king.
The king’s son knew he was in trouble, but he was a quick thinker. “Father,” he said, “I am as disappointed as you are with the performance of the Governance Committee; so disappointed am I that I shall terminate them summarily.”
“You do that son,” said the king, “and staunch the flow of funds from my coffers. I don’t know much, but I do know that when the treasurer tells me that I am running out of money, I have a serious problem.”
And so the Governance Committee was terminated. The expensive consultants, however, lived on as did the king’s son (who was after all a Prince Too ). He knew he would try again, but with a more competent Governance Committee. He had no choice – the present bunch of incompetents had been summarily terminated.
Acknowledgement
This piece was inspired by Craig Brown’s New Prince2 Hypothesis.
Models and messes in management – from best practices to appropriate practices
Scientific models and management
Physicists build mathematical models that represent selected aspects of reality. These models are based on a mix of existing knowledge, observations, intuition and mathematical virtuosity. A good example of such a model is Newton’s law of gravity according to which the gravitational force between two objects (planets, apples or whatever) varies in inverse proportion to the square of the distance between them. The model was a brilliant generalization based on observations made by Newton and others (Johannes Kepler, in particular), supplemented by Newton’s insight that the force that keeps the planets revolving round the sun is the same as the one that made that mythical apple fall to earth. In essence Newton’s law tells us that planetary motions are caused by gravity and it tells us – very precisely – the effects of the cause. In short: it embodies a cause-effect relationship.
[Aside: The validity of a physical model depends on how well it stands up to the test of reality. Newton’s law of gravitation is remarkably successful in this regard: among many other things, it is the basis of orbital calculations for all space missions. The mathematical model expressed by Newton’s law is thus an established scientific principle. That said, it should be noted that models of the physical world are always subject to revision in the light of new information. For example, Newton’s law of gravity has been superseded by Einstein’s general theory of relativity. Nevertheless for most practical applications it remains perfectly adequate.]
Given the spectacular success of modeling in the physical and natural sciences, it is perhaps unsurprising that early management theorists attempted to follow the same approach. Fredrick Taylor stated this point of view quite clearly in the introduction to his classic monograph, The Principles of Scientific Management. Here are the relevant lines:
This paper has been written…to prove that the best management is a true science, resting upon clearly defined laws, rules and principles, as a foundation. And further to show that the fundamental principles of scientific management are applicable to all human activities, from our simplest individual activities to the work of great corporations, which call for the most elaborate cooperation. And briefly, through a series of illustrations, to convince the reader that whenever these principles are correctly applied, results must follow which are truly astounding…
From this it appears that Taylor’s intent was to prove that management could be reduced to a set of principles that govern all aspects of work in organizations.
The question is: how well did it work?
The origin of best practices
Over time, Taylor’s words were used to justify the imposition of one-size-fits-all management practices that ignored human individuality and uniqueness of organisations. Although, Taylor was aware of these factors, he believed commonalities were more important than differences. This thinking is well and alive to this day: although Taylor’s principles are no longer treated as gospel, their spirit lives on in the notion of standardized best practices.
There are now a plethora of standards or best practices for just about any area of management. They are often sold using scientific language, terms such as principles and proof. Consider the following passage taken from from the Official PRINCE2 site:
Because PRINCE2 is generic and based on proven principles, organisations adopting the method as a standard can substantially improve their organisational capability and maturity across multiple areas of business activity – business change, construction, IT, mergers and acquisitions, research, product development and so on.
There are a couple of other things worth noting in the above passage. First, there is an implied cause-effect relationship between the “proven principles” and improvements in “organizational capability and maturity across multiple areas of business activity.” Second, as alluded to above, the human factor is all but factored out – there is an implication that this generic standard can be implemented by anyone anywhere and the results will inevitably be as “truly astounding” as Taylor claimed.
Why best practices are not the best
There are a number of problems with the notion of a best practice. I discuss these briefly below.
First, every organisation is unique. Yes, much is made of commonalities between organisations, but it is the differences that make them unique. Arguably, it is also the differences that give organisations their edge. As Stanley Deetz mentioned in his 2003 Becker lecture:
In today’s world unless you have exceptionally low labor costs, competitive advantage comes from high creativity, highly committed employees and the ability to customize products. All require a highly involved, participating workforce. Creativity requires letting differences make a difference. Most high-end companies are more dependent on the social and intellectual capital possessed by employees than financial investment.
Thoughtless standardization through the use of best practices is a sure way to lose those differences that could make a difference.
Second, in their paper entitled, De-Contextualising Competence: Can Business Best Practice be Bundled and Sold, Jonathan Wareham and Han Gerrits pointed out that organisations operate in vastly varying cultural and social environments. It is difficult to see how best practice approaches with their one-and-a-half-size –fits-all approach would work.
Third , Wareham and Gerrits also pointed out that best practice is often tacit and socially embedded. This invalidates the notion that it can be transferred from an organization in which it works and to another without substantial change. Context is all important.
Lastly, best practices are generally implemented in response to a perceived problem. However, they often address the symptoms rather than the root cause of the problem. For example, a project management process may attempt to improve delivery by better estimation and planning. However, the underlying cause – which may be poor communication or a dysfunctional relationship between users and the IT department –remains unaddressed.
In his 2003 Becker lecture, Stanley Deetz illustrated this point via the following fable:
… about a company formed by very short people. Since they were all short and they wanted to be highly efficient and cut costs, they chose to build their ceiling short and the doorways shorter so that they could have more work space in the same building. And, they were in fact very successful. As they became more and more successful, however, it became necessary for them to start hiring taller people. And, as they hired more and more tall people, they came to realize that tall people were at a disadvantage at this company because they had to walk around stooped over. They had to duck to go through the doorways and so forth. Of course, they hired organizational consultants to help them with the problem.
Initially they had time-and-motion experts come in. These experts taught teams of people how to walk carefully. Tall members learned to duck in stride so that going through the short doors was minimally inconvenient. And they became more efficient by learning how to walk more properly for their environment. Later, because this wasn’t working so well, they hired psychological consultants. These experts taught greater sensitivity to the difficulties of tall members of the organization. Long-term short members learned tolerance knowing that the tall people would come later to meetings, would be somewhat less able to perform their work well. They provided for tall people networks for support…
The parable is an excellent illustration of how best practices can end up addressing symptoms rather than causes.
Ambiguity + the human factor = a mess
Many organisational problems are ambiguous in that cause-effect relationships are unclear. Consequently, different stakeholders can have wildly different opinions as to what the root cause of a problem is. Moreover, there is no way to conclusively establish the validity of a particular point of view. For example, executives may see a delay in a project as being due to poor project management whereas the project manager might see it as being a consequence of poor scope definition or unreasonable timelines. The cause depends on who you ask and there is no way to establish who is right! Unlike problems in physics, organisational problems have a social dimension.
The visionary Horst Rittel coined the evocative term wicked problem to describe problems that involve many stakeholder groups with diverse and often conflicting perspectives. This makes such problems messy. Indeed, Russell Ackoff referred to wicked problems as messes. In his words, “every problem interacts with other problems and is therefore part of a set of interrelated problems, a system of problems…. I choose to call such a system a mess”
Consider an example that is quite common in organisations: the question of how to improve efficiency. Management may frame this issue in terms of tighter managerial control and launch a solution that involves greater oversight. In contrast, a workgroup within the organisation may see their efficiency being impeded by bureaucratic control that results from increased oversight, and thus may believe that the road to efficiency lies in giving workgroups greater autonomy. In this case there is a clear difference between the aims of management (to exert greater control) and those of workgroups (to work autonomously). Ideally, the two ought to talk it over and come up with a commonly agreed approach. Unfortunately they seldom do. The power structure in organisations being what it is, management’s solution usually prevails and, as a consequence, workgroup morale plummets. See this post for an interesting case study on one such situation.
Summing up: a need for appropriate practice, not best practice
The great attraction of best practices, and one of the key reasons for their popularity, is that they offer apparently straightforward solutions to complex problems. However, such problems typically have a social dimension because they affect different stakeholders in different ways. They are messes whose definition depends on who you ask. So there is no agreement on what the problem is, let alone its solution. This fact by itself limits the utility of the best practice approach to organisational problem solving. Purveyors of best practices may use terms like “proven”, “established”, “measurable” etc. to lend an air of scientific respectability to their wares, but the truth is that unless all stakeholders have a shared understanding of the problem and a shared commitment to solving it, the practice will fail.
In our recently published book entitled, The Heretic’s Guide to Best Practices, Paul Culmsee and I describe in detail the issues with the best practice approach to organisational problem-solving. More important, we provide a practical approach that can help you work with stakeholders to achieve a shared understanding of a problem and a shared commitment to a commonly agreed course of action. The methods we discuss can be used in small settings or larger one, so you will find the book useful regardless of where you sit in your organisation’s hierarchy. In essence our book is a manifesto for replacing the concept of best practice with that of appropriate practice – practice with a human face that is appropriate for you in your organisation and particular situation.
Eight to Late 