Archive for the ‘Project Management’ Category
Dark clouds, silver linings: a transaction cost perspective on the outsourcing of information systems
Introduction
Some years ago, I wrote a post applying ideas of transaction cost economics to the question of outsourcing IT development work. In that post I argued that evaluating costs on the basis of vendor quotes alone is highly misleading. One has to also factor in transaction costs – i.e. costs relating to things such as search, bargaining and contract enforcement.
In the present post I discuss transaction cost theory as it applies to the question of whether or not organisations should outsource their enterprise systems (such as ERP and CRM applications) to Software as a Service (SaaS) vendor. With the increasing number of offerings on the market, this issue is one that is high on the agenda of IT decision makers.
To be sure, there are many well-known (and massively hyped!) benefits of cloud-based solutions. To name just one: organisations that take the SaaS route do not have to worry about maintenance, upgrades etc. as these are handled by the vendor. As we shall see, however, when it comes to costs, things are not so clear.
Setting the context
Today’s IT landscape is considerably more complex than that of a couple of decades ago. Improvements in infrastructural technology now offer the IT decision maker choices that were simply not available then. One of the basic choices a decision maker is faced when implementing a new enterprise system is whether to develop (or customize), host and support it in-house or opt for a cloud-based offering. Most often decisions on these matters are made on the basis of vendor-quoted costs alone. A typical discussion between a supporter and skeptic of outsourcing may go as follows:
Supporter: We should outsource our CRM system because we will save costs. We can save X million dollars on licensing and hardware…and then there are potential savings on personnel costs in the longer term.
Skeptic: Yes, but we lose flexibility to modify the application to suit our needs.
Supporter: Flexibility is overrated. We have done a gap-fit analysis and have found that most of core business needs can be satisfied by the three shortlisted cloud offerings. We are not a complex business: we call on customers, sell them our product and then follow-up from time to time to see how they are going and whether there are potential opportunities to sell them upgrades. It’s pretty much what everyone else in the business does.
Skeptic: What about vendor lock-in?
Supporter: There is no lock in we pay as we go; per user per month.
….and so the conversation goes. The supporter seems to have an answer for every question so the skeptic may eventually concede. However, the latter may still be left with a vague sense of unease that something’s been overlooked, and indeed something has…which brings us to the next topic.
Transaction costs
A firm has two choices for any economic activity: performing the activity in-house or going to market. In either case, the cost of the activity can be decomposed into:
- Production costs, which are direct (easily quantifiable) costs of producing the good or service
- Transaction costs, which are other (indirect) costs incurred in performing the economic activity.
Production costs include things such as per user cost etc. These are typically quoted upfront and are easy to contractualise (i.e. put into a contract). Things are more complicated when it comes to transactions costs. To see this, let’s take a quick look at the different types of transaction costs:
- Search /selection costs: These are the costs associated with searching for and shortlisting vendors.
- Bargaining costs: These are costs associated with negotiations for a mutually acceptable contract.
- Maintenance costs: These are expected costs (i.e. those that were foreseen when the contract drawn up) associated with ongoing support, enhancements or upgrades.
- Costs of enforcement and change: These are costs associated with enforcing the terms of the contract and those associated with change.
These costs are typically hard to estimate upfront, and nearly impossible to contractualise. Moreover, in most cases, they are largely borne by the client.
Dark clouds
The difficulties associated with contractualising transaction costs arise from the following:
- Unexpected events: Unforeseen and unforeseeable changes in the customer / vendor organisations or in the business environment may entail major changes in the application functionality or even the outsourcing arrangement.
- Bounded rationality: Business environments are complex and it is impossible for the human mind to anticipate everything that can possibly occur. As a consequence, every contract is necessarily incomplete; there is bound to be something that is overlooked. It is impossible contractualise every eventuality.
- Strategic / opportunistic behaviour: the vendor or customer may deliberately withold certain information from the other party at the outset in order to secure the contract at a favourable rate. Typical vendor behaviour may include not revealing certain limitations of the software on the other hand, the customer may attempt to include unfair penalty clauses into the contract or squeeze the vendor on margins.
Now before I’m accused of being unduly alarmist I should mention that certain kinds of enterprise systems are not subject to the above difficulties, or at least not significantly. Typically these are infrastructural services such as servers, operating systems, databases and even simple, context-independent applications such as email. These can be outsourced successfully without any problems because the services to be provided can be specified accurately upfront and thus contractualised unambiguously.
However, for enterprise applications such as ERP and CRM the situation is different because these systems are more or less unique to a given organisation – in other words, they are context-dependent. Their uniqueness renders them vulnerable to the points mentioned above because it is impossible to foresee all possibilities. As a consequence it is inevitable that any outsourcing deal involving such systems will eventually be affected by one or more of the above uncertainties.
Silver linings
None of the above points is new: both vendors and customers are at least somewhat aware that application outsourcing deals have many unknowns. Consequently, both parties try to minimise their exposure to uncertainty. Unfortunately, they usually go about this in exactly the wrong way: they attempt to build safety for themselves at the cost of the other party. They do this by attempting to contractualise all possible things that can go wrong from their point of view. This is impossible because the future cannot be predicted. Contracts, however detailed, will necessarily be incomplete.
The way out is simple. As Oliver Williamson, winner of the 2009 Economics Nobel Prize, tells us:
…important to the transaction-cost economics enterprise is the assumption that contracts, albeit incomplete, are interpreted in a farsighted manner, according to which economic actors look ahead, perceive potential hazards and embed transactions in governance structures that have hazard-mitigating purpose and effect. Also, most of the governance action works through private ordering with courts being reserved for purposes of ultimate appeal.
As he tells us, contracts need to be interpreted in a farsighted manner and governance action should work through private ordering (directly between the customer and vendor). Essentially, the success of an outsourcing deal is to a large extent the joint responsibility of the customer and vendor. Above all, it calls for a relationship based on old-fashioned notion of trust.
Interestingly, Elinor Ostrom, who jointly won the 2009 Economics Nobel with Williamson, had much to say about informal contracts, private ordering and trust. Her extensive studies on collectives lead her to conclude that successful collective endeavours have the following two elements in common:
- High levels of face-to-face communication
- Innovative governance structures that are designed and enforced by the participants rather than external authorities.
The relevance of these to an outsourcing arrangement is clear: face to face communication builds trust, and innovative internal governance structures that are enforceable without legal recourse will discourage opportunistic behaviour from both parties. Although the latter may sound a bit utopian there are proven ways to set up such governance structures (see this paper for an example).
Conclusion
In this post I have argued that a proper analysis of SaaS deals must include transaction costs. Although these are difficult to quantify, a consideration of the different categories of transaction costs will at least lead to a more realistic appraisal of such arrangements. It is my belief that many outsourcing deals go sour because transaction costs are overlooked. Given that it is impossible to foresee the future, the best course of action is to develop a business relationship that is based on trust.
To sum up: most important factor in enterprise application outsourcing is not cost, but trust – an ineffable element that can neither be quantified nor contractualised.
Symptoms, not causes: a systems perspective on project failure
Introduction
A quick search reveals that the topic of project failure gets a fair bit of attention on the Internet. Many of the articles identify factors such as lack of executive support or incomplete/misunderstood requirements as the prime causes of failure. In this post I use concepts from systems theory to argue that commonly identified “causes” of project failure – such as the ones noted above – are symptoms rather than causes. I then surface the real causes of project failure by taking a systems perspective – a viewpoint that considers the project and the hosting organisation as a whole.
As I will argue, project failures can often be traced back to dysfunctional structures and processes within the organisation. These factors usually have little to do with the project directly and are therefore not always obvious at first sight.
Setting the scene
Readers who have waded through the vast literature on failed projects will have noted that there are diverse opinions on the prime reasons for failure. It would take me too long to wade through these articles, so I’ll just pick a source that is well known even if not entirely credible: The Chaos Report by the Standish Group. As per this summary the Chaos Report 2009 claimed the following were the top three causes of project failure:
- Lack of user input.
- Incomplete or changing requirements/specifications.
- Lack of executive support.
Although the report lists the top ten factors, in the interests of space I’ll focus on just these three.
I should mention that the report refers to the above as “Project Challenged Factors.” Grammar issues aside, this is a somewhat strange way of putting it. Anyway, I interpret this phrase to mean reasons (or driving factors) for failure.
Systems theory and projects
First up, what exactly is a system?
Here is a jargon-free definition from the wonderful book by Donella Meadows entitled, Thinking in Systems: A Primer. Incidentally, I highly recommend the book as an easy-to-read and engaging introduction to systems theory:
A system is a set of things interconnected in such a way that they produce their own pattern of behaviour over time. The system may be buffeted, constricted , triggered or driven by outside forces. But its response to these is characteristic of itself and is seldom simple in the real world. (italics mine)
The word interconnected is important because it tells us that when we study something from a systems perspective, we must identify all the important connections it has with its environment. In the case of projects, the important connections are fairly obvious. A project is usually carried out within an organisational setting so it would have many connections to the hosting organisation. Chief amongst these is the fact that a project is staffed and resourced by the hosting organisation (or by an organisation designated by it). Another important connection is that a project will affect different stakeholder groups within the hosting organisation in different ways. However, since all stakeholders have ongoing organisational roles that go beyond the project, the project is not their only interest. This is a key point to which we will return later.
The phrases “own pattern of behaviour over time” and “characteristic of itself” tell us that systems have a unique pattern of behaviour that is characteristic to them. The important point to note is that characteristic behaviour implies that different systems that have the same signature – i.e. identifying features – will tend to behave in the same way. From the perspective of projects this tells us that projects within similar organisations will evolve in similar ways.
A systems view of the causes of project failure
With only this very brief look at systems theory we are now in a position to get some insights into the real causes of project failure. As mentioned earlier, we will focus on the top three reasons ala Standish.
Lack of user input
First up, consider lack of user input. Systems theory tells us that we need to look at the issue from the perspective of the project and the organisation. From this point of view it is clear that users who have been asked to work on the project in addition to their normal duties will view the project as a burden rather than something that may benefit them in the future.
This by itself is not a new insight. In fact, project management gurus have talked themselves hoarse about the need to free up resources etc. However, the point is that organisational structures typically work against this. Firstly, people who are asked to work on a project know that it is an initiative that will end in a finite (usually, reasonably short) time. Therefore, their long terms interests lies in their ongoing roles rather than in projects. Secondly, most organisations are still structured along functional lines and people’s identities are anchored within their functional reporting lines rather than ephemeral project hierarchies.
Changing requirements
The issue of changing requirements and specifications can also be understood from a systems point of view. A characteristic of many systems is that they are stable – i.e. they resist change. Organisations are typically stable systems – they tend to retain their identity despite changes in their environment. One of the characteristics of such organisations is that people in them tend to think and act in set ways – that is, their actions and thinking processes follow well worn patterns to the point where they do not need to think about what they do too deeply.
One of the consequences of this is that when they are asked for requirements users often provide incomplete description of what they do, leaving out significant items that are obvious to them (but not to the analysts who are gathering requirements!). Although I don’t have the figures to back this – I speculate that a fair proportion of changes in requirements are the result of inadequate detail or thought put into developing initial requirements. The point is users and sponsors don’t necessarily see these as changes, but project teams do.
Lack of executive support
Finally, let’s look at the the problem of lack of executive support. Project sponsors usually hold important executive-level positions within the hosting organisation. By virtue of their positions, they have a number of important things that compete for their attention. A project – even an important one – is only one of many things going on in an organisation at any one time. Moreover, organisational priorities do shift, perhaps more often than executives may want to admit. So a project that was the key focus yesterday may be superseded by other priorities today.
There are of course many other ways in which project sponsors can be distracted, but I think I’ve made my point which is that lack of executive support is due to features that are inherent in organisations. So no amount of forcing executives to pay attention to their projects is going to work unless the entire system (project + organisation) changes. And this is difficult if not impossible to achieve because stable systems such as organisations tend to resist change, and therefore continue to display their characteristic patterns of behaviour.
Discussion
So we see that the causes of project failure can be traced back to the organisations in which they are embedded. Specifically, they lie in unwritten norms and formal policies that dictate how the hierarchy operates and how things are done within the organisation. The most important consequence of this is that standard fixes (of encouraging user input and executive support, or instituting change management, say) will not cure the problem of project failure because they do not address the dysfunctional norms and policies that are the root cause of failure.
The above is not news. In fact, the matrix organisation structure was proposed as a response to the need for “project friendly” organisations. I’m no expert on matrix organisations so I will leave it to others to comment on how successful they are. The only point I would make is that in my experience multiple reporting lines (even if dotted and solid) do not work too well. There are always conflicting interests that cause divided loyalties and conflicting interests.
So, the natural question is : what – if anything – can we do about this? The answer is implicit in the foregoing paragraphs. One has to align the project with the organisation, not just at the level of objectives or structure, but also in operational matters such as timelines, budget and resources – the very things that make up the so-called iron triangle. The best time to do this is at the front-end of the project, i.e. the start. At this time, the person(s) driving the project have to engage all stakeholders who will be affected by the initiative and find out their motivations, interests and – most importantly – their concerns regarding the project. If this discussion happens in an open and frank manner, it should surface the issues highlighted In the previous section. Since the discussion takes place even before the project starts, there is at least some hope of addressing these concerns.
There are many ways to structure and facilitate such discussions. Check out this post for an introduction to one and have a look at my book co-authored with Paul Culmsee for much more. That said, one doesn’t need any particular technique – the willingness to discuss difficult matters openly and an openness to other points of view is all that’s needed. That, however, is not always easy to come by…
Conclusion
We have seen that the top causes of project failure can be traced back to the hierarchies and incentive systems of the hosting organisation. Therefore, superficial attempts to fix the problem at the level of individual projects (or even a PMO) will not work. The only hope of addressing the root causes of project failure is to focus on the systemic dysfunctions that cause them.
Eight to Late 