Eight to Late

Sensemaking and Analytics for Organizations

Archive for the ‘Wicked Problems’ Category

Beyond Best Practices: a paper review and the genesis of a collaboration

with 10 comments

Introduction

The fundamental premise behind best practices  is that it is possible to reproduce the successes of those who excel by imitating them. At first sight this assumption seems obvious and uncontroversial. However, most people who have lived through an implementation of a best practice know that following such prescriptions does not guarantee success. Actually, anecdotal evidence suggests the contrary:  that most attempts at implementing best practices fail. This paradox remains unnoticed by managers and executives who continue to commit their organisations to implementing best practices that are, at best, of dubious value.

Why do best practices fail?   There has been a fair bit of research on the shortcomings of best practices, and the one thing it tells us is that there is no simple answer to this question.  In this post I’ll discuss this issue, drawing upon an old (but still very relevant) paper by Jonathan Wareham and Han Cerrits entitled, De-Contextualising Competence: Can Best Practice be Bundled and Sold.   Note that I will not cover the paper in its entirety;  my discussion will focus only on those aspects that relate to the question raised above.

I may as well say it here:  I have a secondary aim  (or more accurately,  a vested interest)  in discussing this paper.  Over the last few months Paul Culmsee and I have been working on a book that discusses reasons why best practices fail and proposes some practical techniques to address their shortcomings.  I’ll end this post with a brief discussion of the background and content of the book (see this post for Paul’s take on the book). But let’s look at the paper first…

Background

On the first page of the paper the authors state:

Although the concept of ‘imitating excellent performers’ may seem quite banal at first glance, the issue, as we will argue, is not altogether that simple after deeper consideration. Accordingly, the purpose of the paper is to explore many of the fundamental, often unquestioned, assumptions which underlie the philosophy and application of Business Best Practice transfer. In illuminating the central empirical and theoretical problems of this emerging discipline, we hope to refine our expectations of what the technique can yield, as well as contribute to theory and the improvement of practice.

One of the most valuable aspects of the paper is that it lists some of the implicit assumptions that are often glossed over by consultants and  others who sell  and implement best practice methodologies.  It turns out that these assumptions are not valid in most practical situations, which renders the practices themselves worthless.

The implicit assumptions

According to Wareham and Cerrits, the unstated premises behind best practices include:

  1. Homogeneity of organisations: Most textbooks and courses on best practices present the practices as though they have an existence that is independent of organizational context.  Put another way: they assume that all organisations are essentially the same. Clearly, this isn’t the case – organisations are defined by their differences.
  2. Universal yardstick: Best practices assume that there is a universal definition of what’s best, that what’s best for one is best for all others. This assumption is clearly false as organisations have different (dare I say, unique) environments, objectives and strategies. How can a universal definition of “best” fit all?
  3. Transferability:  Another tacit assumption in the best practice business is that practices can be transplanted on to receiving organisations wholesale. Sure, in recent years it has been recognized that such transplants are successful only if a) the recipient organisation undertakes the changes necessary for the transplant to work and b) the practice itself is adapted to the recipient organisation. The point is in most successful cases, the change or adaptation is so great that it no longer resembles that original  best practice. This is an important point – to have a hope in hell of working, best practices have to be adapted extensively. It is also worth mentioning that  such adaptations will succeed only if they are  made in consultation with those who will be affected by the practices. I’ll say more about this later in this post
  4. Alienability and stickiness: These are concepts that relate to the possibility of extracting relevant knowledge pertaining to a best practice from a source and transferring it without change to a recipient. Alienability refers to the possibility of extracting relevant knowledge from the source. Alienability is difficult because best practice knowledge is often tacit, and is therefore difficult to codify. Stickiness refers to the willingness of the recipient to learn this knowledge, and his or her ability to absorb it. Stickiness highlights the importance of obtaining employee buy-in before implementing best practices. Unfortunately most best practice implementations gloss over the issues of alienability and stickiness.
  5. Validation: Wareham and Cerrits contend that best practices are rarely validated. More often than not,  recipient organisations simply believe that they will work, based on their consultants’ marketing spiel. See this short piece by Paul Strassman for more on the dangers of doing so.

What does  “best” mean anyway?

After listing the implicit assumptions, Wareham and Cerrits argue that the conceptual basis for defining a particular practice as being “best” is weak.  Their argument hinges on the observation that it is impossible to attribute the superior performance of a firm to specific managerial practices. Why? Well, because one cannot perform a control experiment to see what would happen if those practices weren’t used.

Related to the above is the somewhat subtle point that it is impossible to say, with certainty, whether practices, as they exist within model organisations, are consequences of well-thought out managerial action or whether they are merely adaptations to changing environments. If the latter were true, then there is no best practice, because the practices as they exist in model organisations are essentially random responses to organizational stimuli.

Wareham and Cerrits also present an economic perspective on best practice acquisition and transfer, but I’ll omit this as it isn’t of direct relevance to the question of why best practices fail.

Implications

The authors draw the following conclusions from their analysis:

  1. The very definition of best practices is fraught with pitfalls.
  2. Environmental factors have a significant effect on the evolution and transfer(ability) of “best” practices. Consequently, what works in one organisation may not work in another.

So, can anything be salvaged?  Wareham and Cerrits think so. They suggest an expanded view of best practices which includes things such as:

  1. Using best practices as guides for learning new technologies or new ways of working.
  2. Using best practices to generate creative insight into how business processes work in practice.
  3. Using best practices as a guide for change – that is, following the high-level steps, but not necessarily the detailed prescriptions.

These are indeed sensible and reasonable statements. However, they  are much weaker than the usual hyperbole-laden claims that accompany best practices.

Discussion

Cerrits and Johnson focus on the practices themselves, not the problems they are used to solve. In my opinion, another key reason why best practices fail is that they are applied without a comprehensive understanding of the problem that they are intended to address.

I’ll clarify this using an example:  in a quest to improve efficiency an organisation might go through a major restructure. All too often, such organisations will not think through all the consequences of the restructuring (what are the long-term consequences of outsourcing certain functions, for instance). The important point to realize is that a comprehensive understanding of the consequences is possible only if all stakeholders – management and employees – are involved in planning the restructure.  Unfortunately, such a bottom-up approach is rarely taken because of the effort involved, and the  wrong-headed perception that chaos may ensue from management actually talking to people on the metaphorical shop floor.   So  most organizations take a top-down approach, dictating what will be done, with little or no employee involvement.

Organisations focus on how to achieve a particular end. The end itself, the reasons for wanting to achieve it and the consequences of doing so remain unexplored; it is assumed that these are obvious to all stakeholders. To put it in aphoristically: organizations focus on the “how” not the on the “what” or why.”

The heart of the matter

The key to understanding why best practices do not work is to realise that many organizational problems are wicked problems: i.e., problems that are hard to define, let alone solve’s  (see this paper for a comprehensive discussion of wicked problems).  Let’s look at organizational efficiency, for example.   What does it really mean to improve organizational efficiency?   More to the point, how can one arrive at a generally agreed way to improve organizational efficiency?  By generally agreed, I mean a measure that all stakeholders understand and agree on. Note that “efficiency “is just an example here – the same holds for most other matters of strategic importance to organizations: organisational strategy is a wicked problem.

Since wicked problems are hard to pin down (because they mean different things to different people), the first step to solving them is to ensure that all stakeholders have a common (or shared) understanding of what the problem is. The next step  is to achieve a shared commitment to solving that problem. Any technique that could help achieve a shared understanding of wicked problems and commitment to solving them  would  truly deserve to be called the one best practice to rule them all.

The genesis of a collaboration

About a year ago, in a series of landmark posts entitled The One Best Practice to Rule Them All, Paul Culmsee wrote about his search for a practical method to manage wicked problems.  In the articles he made a convincing case that dialogue mapping can help a diverse group of stakeholders achieve a shared understanding of such problems.  Paul’s writings inspired me to learn dialogue mapping and use it at work. I was impressed – here, finally, was a technique that didn’t claim to be a best practice, but had the potential to address some of the really complex problems that organisations face.

Since then, Paul and I have had several conversations about the failure of best practices in to tackling issues ranging from organizational change to project management. Paul is one of those rare practitioners with an excellent grounding in theory and practice.  I learnt a lot from him in those conversations. Among other things, he told me about his experiences in using dialogue mapping to tackle apparently intractable problems (see this case study from Paul’s company, for example).

Late last year, we thought of writing up some of the things we’d been talking about in a series of joint blog posts. Soon we realised that we had much more to say than would fit into a series of posts – we probably had  enough for a book.  We’re a few months into writing that book, and are quite pleased with the way it’s turning out.

Here’s a very brief summary of the book. The first part analyses why best practices fail.  Our analysis  touches  upon diverse areas like organizational rhetoric, cognitive bias, memetics and scientific management (topics that both Paul and I have written about on our blogs).  The second part of the book presents a series of case studies that illustrate some techniques that address complex problems that organizations face. The case studies are based on our experiences in using dialogue mapping and other techniques to tackle wicked problems relating to organizational strategy and project management.  The techniques we discuss go beyond the rhetoric of best practices – they work because they use a bottom-up approach that takes into account the context and environment in which the problems live.

Now, Paul writes way better than I do. For one, his writing is laugh-out-loud funny, mine  isn’t. Those who have read his work and mine may be wondering how our very different styles will combine.  I’m delighted to report that the book is way more conversational and entertaining than my blog posts.  However, I should also emphasise that we are trying to be as rigorous as we can by backing up our claims by references to research papers and/or case studies.

We’re learning a lot in the process of writing, and are enthused and excited about the book . Please stay tuned – we’ll post occasional updates on how it is progressing.

Update (16 June 2010):

An excerpt from the book has been published here.

Update (27 Nov 2011):

The book, which has a new title, is currently in the final round of proofs. Hopefully it will be available for pre-order in a month or two.

Update (05 Dec 2011):

It’s out!

Get your copy via Amazon or Book Depository.

The e-book can be obtained from iUniverse (PDF or Epub formats) or Amazon (Kindle).

 

IBIS, dialogue mapping, and the art of collaborative knowledge creation

with 23 comments

Introduction

In earlier posts  I’ve described a notation called IBIS (Issue-based information system), and demonstrated its utility in visualising reasoning and resolving complex issues through dialogue mapping.  The IBIS notation consists of just three elements (issues, ideas and arguments) that can be connected in a small number of ways. Yet, despite these limitations, IBIS has been found to enhance creativity when used in collaborative design discussions.  Given the simplicity of the notation and grammar, this claim is surprising,  even paradoxical.  The present post  resolves this paradox by viewing  collaborative knowledge creation as an art, and considers the aesthetic competencies required to facilitate this art.

Knowledge art

In a position paper entitled, The paradox of the “practice level” in collaborative design rationale, Al Selvin draws an analogy between design  discussions using Compendium (an open source IBIS-based argument mapping tool)  and art.  He uses the example of the artist Piet Mondrian, highlighting the difference in  style between Mondrian’s earlier and later work. To quote from the paper,

Whenever I think of surfacing design rationale as an intentional activity — something that people engaged in some effort decide to do (or have to do), I think of Piet Mondrian’s approach to painting in his later years. During this time, he departed from the naturalistic and impressionist (and more derivative, less original) work of his youth (view an image here) and produced the highly abstract geometric paintings (view an image here) most associated with his name…

Selvin points out that the difference between the first and the second paintings is essentially one of abstraction: the first one is almost instantly recognisable as a depiction of dunes on a beach whereas the second one, from Mondrian’s minimalist period, needs some effort to understand and appreciate, as it uses a very small number of elements to create a specific ambience. To quote from the paper again,

“One might think (as many in his day did) that he was betraying beauty, nature, and emotion by going in such an abstract direction. But for Mondrian it was the opposite. Each of his paintings in this vein was a fresh attempt to go as far as he could in the depiction of cosmic tensions and balances. Each mattered to him in a deeply personal way. Each was a unique foray into a depth of expression where nothing was given and everything had to be struggled for to bring into being without collapsing into imbalance and irrelevance. The depictions and the act of depicting were inseparable. We get to look at the seemingly effortless result, but there are storms behind the polished surfaces. Bringing about these perfected abstractions required emotion, expression, struggle, inspiration, failure and recovery — in short, creativity…”

In analogy, Selvin contends that a group of people who work through design issues using a minimalist notation such as IBIS can generate creative new ideas. In other words:  IBIS, when used in a group setting such as dialogue mapping,  can become a vehicle for collaborative creativity. The effectiveness of the tool, though, depends on those who wield it:

“…To my mind using tools and methods with groups is a matter of how effective, artistic, creative, etc. whoever is applying and organizing the approach can be with the situation, constraints, and people. Done effectively, even the force-fitting of rationale surfacing into a “free-flowing” design discussion can unleash creativity and imagination in the people engaged in the effort, getting people to “think different” and look at their situation through a different set of lenses. Done ineffectively, it can impede or smother creativity as so many normal methods, interventions, and attitudes do…”

Although Selvin’s discussion is framed in the context of design discussions using Compendium,  this is but dialogue mapping by another name.  So,  in essence, he  makes a case for viewing the collaborative generation of knowledge (through dialogue mapping or any other means) as an art.  In fact, in another article, Selvin uses the term knowledge art to describe both the process and the product of creating knowledge as discussed above.   Knowledge Art as he sees it, is a marriage of the two forms of discourse that make up the term. On the one hand, we have knowledge which, “… in an organizational setting, can be thought of as what is needed to perform work; the tacit and explicit concepts, relationships, and rules that allow us to know how to do what we do.” On the other, we have art which “… is concerned with heightened expression, metaphor, crafting, emotion, nuance, creativity, meaning, purpose, beauty, rhythm, timbre, tone, immediacy, and connection.”   

Facilitating collaborative knowledge creation

In the business world, there’s never enough time to deliberate or think through ideas (either individually or collectively): everything is done in a hurry and the result is never as good as it should or could be; the picture never quite complete.  However, as Selvin says,

“…each moment (spent discussing or thinking through ideas or designs) can yield a bit of the picture, if there is a way to capture the bits and relate them, piece them together over time. That capturing and piecing is the domain of Knowledge Art. Knowledge Art requires a spectrum of skills, regardless of how it’ practiced or what form it takes. It means listening and paying attention, determining the style and level of intervention, authenticity, engagement, providing conceptual frameworks and structures, improvisation, representational skill and fluidity, and skill in working with electronic information…”

So,  knowledge art requires a wide range of technical and non-technical skills.  In previous posts  I’ve discussed some of  technical skills required – fluency with IBIS, for example.  Let’s now look at  some of the non-technical competencies.

What are the competencies needed for collaborative knowledge creation?  Palus and Horth offer some suggestions in their paper entitled, Leading Complexity; The Art of Making Sense.  They define the concept of  creative leadership as making shared sense out of complexity and chaos and the crafting of meaningful action.  Creative leadership is akin to dialogue mapping, which Jeff Conklin describes as  a means to achieve a shared understanding of wicked problems  and a shared commitment to solving them.  The connection between creative leadership and dialogue mapping is apparent once one notices the similarity between their definitions.  So the  competencies  of creative leadership should apply to dialogue mapping (or collaborative knowledge creation)  as well.

Palus  and Horth describe  six basic competencies of creative leadership. I outline these below, mentioning  their relevance to dialogue mapping:

Paying Attention:  This refers to the ability to slow down discourse  with the aim of  achieving a deep understanding of the issues at hand. A skilled dialogue mapper has to be able to listen; to pay attention to what’s being said.

Personalizing:  This refers to the ability to draw upon personal experiences, interests and passions whilst engaged in work. Although the connection to dialogue mapping isn’t immediately evident, the point Palus and Horth make is that the ability to make connections between work and one’s interests and passions helps increase involvement, enthusiasm and motivation in tackling work challenges.

Imaging:  This refers to the ability to visualise problems so as  to understand them better,  using metaphors, pictures stories etc to stimulate imagination, intuition and understanding. The connection to dialogue mapping is clear and needs no elaboration.

Serious play: This refers to the ability to experiment with new ideas; to learn by trying and doing in a non-threatening environment. This is something that software developers do when learning new technologies. A group engaged in a dialogue mapping must have a sense of play; of trying out new ideas, even if they seem somewhat unusual.

Collaborative enquiry: This refers to the ability to  sustain productive dialogue in a diverse group of stakeholders. Again, the connection to dialogue mapping is evident.

Crafting: This refers to the ability to synthesise issues, ideas, arguments and actions into coherent, meaningful wholes. Yet again, the connection to dialogue mapping is clear – the end product is ideally a shared understanding of the problem and a shared commitment to a meaningful solution.

Palus and Horth suggest that these competencies have been ignored in the business world because:

  1. They are seen as threatening the status quo (creativity is to feared because it invariably leads to changes).
  2. These competencies are aesthetic, and the current emphasis on scientific management devalues competencies that are not rational or analytical.

The irony is that creative scientists have these aesthetic competencies (or qualities) in spades. At the most fundamental level science is an art – it is about constructing theories or designing experiments that make sense of the world. Where do the ideas for these new theories or experiments come from? Well, they certainly aren’t out there in the objective world; they come from the imagination of the scientist. Science, in the real sense of the word, is knowledge art. If these competencies are useful in science, they should be more than good enough for the business world.

Summing up

To sum up:  knowledge creation in an organisational context is best viewed as an art – a collaborative art.  Visual representations such as IBIS provide a medium to capture snippets of knowledge and relate them, or  piece them together over time. They provide the canvas, brush and paint to express knowledge as art  through the process of dialogue mapping.

Visualising arguments using issue maps – an example and some general comments

with 16 comments

The aim of an opinion piece writer is to convince his or her readers that a particular idea or point of view is reasonable or right.  Typically, such pieces  weave facts , interpretations and reasoning into prose, wherefrom it can be hard to pick out the essential thread of argumentation.  In an earlier post I showed how an issue map can help in clarifying the central arguments in a “difficult” piece of writing by mapping out Fred Brooks’ classic article No Silver Bullet.  Note that I use the word “difficult” only because the article has, at times, been misunderstood and misquoted; not because it is particularly hard to follow.  Still, Brooks’ article borders on the academic; the arguments presented therein are of interest to a relatively small group of people within the software development community. Most developers and architects aren’t terribly interested in the essential difficulties of the profession – they just want to get on with their jobs. In the present post, I develop an issue map of a piece that is of potentially wider interest to the IT community – Nicholas Carr’s 2003 article, IT Doesn’t Matter.

The main point of Carr’s article is that IT is becoming a utility,  much like electricity, water or rail. As this trend towards commoditisation gains momentum, the strategic advantage offered by in-house IT will diminish, and organisations will be better served by buying IT services from “computing utility” providers than by maintaining their own IT shops.  Although Carr makes a persuasive case, he glosses over a key difference between IT and other utilities (see this post for more). Despite this, many business and IT leaders have taken his words as the way things will be. It is therefore important for all IT professionals to understand Carr’s arguments. The consequences are likely to affect them some time soon, if they haven’t already.

Some preliminaries before proceeding with the map. First, the complete article is available here – you may want to have a read of it before proceeding (but this isn’t essential). Second, the discussion assumes a basic knowledge of  IBIS (Issue-Based Information System) –  see  this post for a quick tutorial on IBIS.  Third, the map is constructed using the open-source tool Compendium which can be downloaded here.

With the preliminaries out of the way, let’s get on with issue mapping Carr’s article.

So, what’s the root  (i.e. central) question that Carr poses in the article?  The title of the piece is  “IT Doesn’t Matter” – so one possible root question is, “Why doesn’t IT matter?” But there are other candidates:   “On what basis is IT an infrastructural technology?” or  “Why is the strategic value of IT diminishing?” for example. From this it should be clear that there’s a fair degree of subjectivity at every step of constructing an issue map. The visual representation that I construct here is but one interpretation of Carr’s argument.

Out of the above (and many other possibles),  I choose  “Why doesn’t IT matter?” as the root question. Why? Well,  in my view the whole  point of the piece  is to convince the reader that IT doesn’t matter because it is an infrastructural technology and consequently has no strategic significance. This point should become clearer as our development of the issue map progresses.

The ideas that respond to this question aren’t immediately obvious. This isn’t unusual:  as I’ve mentioned elsewhere, points can only be made sequentially – one after the other – when expressed in prose.  In some cases one may have to read a piece in its entirety to figure out the elements that respond to a root (or any other) question.

In the case at hand, the response to the root question stands out clearly after a quick browse through the article. It is:  IT is an infrastructural technology.

The map with the root question and the response is shown in Figure 1.

Figure 1: Issue Map Stage 1

Figure 1: Issue Map Stage 1

Moving on, what arguments does Carr offer for (pros) and against (cons) this idea? A reading of the article reveals one con and four pros. Let’s look at the cons first:

  1. IT (which I take to mean software) is complex and malleable, unlike other infrastructural technologies. This point is mentioned, in passing, on the third page of the paper: “Although more complex and malleable than its predecessors, IT has all the hallmarks of an infrastructural technology…”

The arguments supporting the idea that IT is an infrastructural technology are:

  1. The evolution of IT closely mirrors that of other infrastructural technologies such as electricity and rail. Although this point encompasses the other points made below, I think it merits a separate mention because the analogies are quite striking. Carr makes a very persuasive, well-researched case supporting this point.
  2. IT is highly replicable. This is point needs no further elaboration, I think.
  3. IT is a transport mechanism for digital information. This is true, at least as far as network and messaging infrastructure is concerned.
  4. Cost effectiveness increases as IT services are shared. This is true too, providing it is understood that flexibility is lost when services are shared.

The map, incorporating the pros and cons is shown in Figure 2.

Figure 2: Issue Map Stage 2

Figure 2: Issue Map Stage 2

Now that the arguments for and against the notion that IT is an infrastructural technology are laid out, lets look at the article again, this time with an eye out for any other issues  (questions)  raised.

The first question is an obvious one: What are the consequences of IT being an infrastructural technology?   

Another point to be considered is the role of proprietary technologies, which – by definition – aren’t infrastructural. The same holds true for  custom built applications. So, this begs the question, if IT is an infrastructural technology, how do proprietary and custom built applications fit in?

The map, with these questions  added in is shown in Figure 3.

Figure 3: Issue Map Stage 3

Figure 3: Issue Map Stage 3

Let’s now look at the ideas that respond to these two questions.

A point that Carr makes early in the article is that the strategic value of IT is diminishing. This is essentially a consequence of the notion that IT is an infrastructural technology. This idea is supported by the following arguments:

  1. IT is ubiquitous – it is everywhere, at least in the business world.
  2. Everyone uses it in the same way. This implies that no one gets a strategic advantage from using it.

What about proprietary technologies and custom apps?.  Carr reckons these are:

  1. Doomed to economic obsolescence. This idea is supported by the argument that these apps are too expensive and are hard to maintain.
  2. Related to the above, these will be replaced by generic apps that incorporate best practices. This trend is already evident in the increasing number of enterprise type applications that offered as services. The advantages of these are that they a) cost little b) can be offered over the web and c) spare the client all those painful maintenance headaches.

The map incorporating these ideas and their supporting arguments is shown in Figure 4.

Figure 4: Issue Map Stage 4

Figure 4: Issue Map Stage 4

Finally, after painting this somewhat gloomy picture (to a corporate IT minion, such as me) Carr asks and answers the question: How should organisations deal with the changing role of IT (from strategic to operational)? His answers are:

  1. Reduce IT spend.
  2. Buy only proven technology – follow don’t lead.
  3. Focus on (operational) vulnerabilities rather than (strategic) opportunities.

The map incorporating this question and the ideas that respond to it is shown in Figure 5, which is also the final map (click on the graphic to view  a full-sized image).

Figure 5: Final Issue Map

Figure 5: Issue Map Stage 5

Map completed, I’m essentially done with this post. Before closing, however, I’d like to mention a couple of general points that arise from issue mapping of prose pieces.

Figure 5 is my interpretation of the article. I should emphasise that my interpretation may not coincide with what Carr intended to convey (in fact, it probably doesn’t). This highlights an important, if obvious, point: what a writer intends to convey in his or her writing may not coincide with how readers interpret it. Even worse, different readers may interpret a piece differently. Writers need to write with an awareness of the potential for being misunderstood.  So, my  first point is that issue maps can help writers clarify and improve the quality of their reasoning  before they cast it in prose.

Issue maps sketch out the logical skeleton or framework of argumentative prose. As such, they  can help highlight weak points of arguments. For example, in the above article Carr glosses over the complexity and malleability of software. This is a weak point of the argument, because it is a key difference between IT and traditional infrastructural technologies. Thus my second point is that issue maps can help readers visualise weak links in arguments which might have been obscured by rhetoric and persuasive writing.

To conclude,   issue maps are valuable to writers and readers:  writers can use  issue maps to  improve the quality of their  arguments before committing them in writing, and  readers can use such maps to understand arguments that have been thus committed.