Archive for the ‘Project Management’ Category
On the accuracy of group estimates
Introduction
The essential idea behind group estimation is that an estimate made by a group is likely to be more accurate than one made by an individual in the group. This notion is the basis for the Delphi method and its variants. In this post, I use arguments involving probabilities to gain some insight into the conditions under which group estimates are more accurate than individual ones.
An insight from conditional probability
Let’s begin with a simple group estimation scenario.
Assume we have two individuals of similar skill who have been asked to provide independent estimates of some quantity, say a project task duration. Further, let us assume that each individual has a probability 
Based on the above, the probability that they both make a correct estimate, 
This is a consequence of our assumption that the individual estimates are independent of each other.
Similarly, the probability that they both get it wrong, 
Now we can ask the following question:
What is the probability that both individuals make the correct estimate if we know that they have both made the same estimate?
This can be figured out using Bayes’ Theorem, which in the context of the question can be stated as follows:
In the above equation, 
Similarly, 
Question: Why?
Answer: If both estimators are correct then they must have made the same estimate (i.e. they must both within be an acceptable range of the right answer).
Finally, 
Now lets apply Bayes’ theorem to the following two cases:
- Both individuals are good estimators – i.e. they have a high probability of making a correct estimate. We’ll assume they both have a 90% chance of getting it right (
).
- Both individuals are poor estimators – i.e. they have a low probability of making a correct estimate. We’ll assume they both have a 30% chance of getting it right (
)
Consider the first case. The probability that both estimators get it right given that they make the same estimate is:
Thus we see that the group estimate has a significantly better chance of being right than the individual ones: a probability of 0.9878 as opposed to 0.9.
In the second case, the probability that both get it right is:
The situation is completely reversed: the group estimate has a much smaller chance of being right than an individual estimate!
In summary: estimates provided by a group consisting of individuals of similar ability working independently are more likely to be right (compared to individual estimates) if the group consists of competent estimators and more likely to be wrong (compared to individual estimates) if the group consists of poor estimators.
Assumptions and complications
I have made a number of simplifying assumptions in the above argument. I discuss these below with some commentary.
- The main assumption is that individuals work independently. This assumption is not valid for many situations. For example, project estimates are often made by a group of people working together. Although one can’t work out what will happen in such situations using the arguments of the previous section, it is reasonable to assume that given the right conditions, estimators will use their collective knowledge to work collaboratively. Other things being equal, such collaboration would lead a group of skilled estimators to reinforce each others’ estimates (which are likely to be quite similar) whereas less skilled ones may spend time arguing over their (possibly different and incorrect) guesses. Based on this, it seems reasonable to conjecture that groups consisting of good estimators will tend to make even better estimates than they would individually whereas those consisting of poor estimators have a significant chance of making worse ones.
- Another assumption is that an estimate is either good or bad. In reality there is a range that is neither good nor bad, but may be acceptable.
- Yet another assumption is that an estimator’s ability can be accurately quantified using a single numerical probability. This is fine providing the number actually represents the person’s estimation ability for the situation at hand. However, typically such probabilities are evaluated on the basis of past estimates. The problem is, every situation is unique and history may not be a good guide to the situation at hand. The best way to address this is to involve people with diverse experience in the estimation exercise. This will almost often lead to a significant spread of estimates which may then have to be refined by debate and negotiation.
Real-life estimation situations have a number of other complications. To begin with, the influence that specific individuals have on the estimation process may vary – a manager who is a poor estimator may, by virtue of his position, have a greater influence than others in a group. This will skew the group estimate by a factor that cannot be estimated. Moreover, strategic behaviour may influence estimates in a myriad other ways. Then there is the groupthink factor as well.
…and I’m sure there are many others.
Finally I should mention that group estimates can depend on the details of the estimation process. For example, research suggests that under certain conditions competition can lead to better estimates than cooperation.
Conclusion
In this post I have attempted to make some general inferences regarding the validity of group estimates based on arguments involving conditional probabilities. The arguments suggest that, all other things being equal, a collective estimate from a bunch of skilled estimators will generally be better than their individual estimates whereas an estimate from a group of less skilled estimators will tend to be worse than their individual estimates. Of course, in real life, there are a host of other factors that can come into play: power, politics and biases being just a few. Though these are often hidden, they can influence group estimates in inestimable ways.
Acknowledgement
Thanks go out to George Gkotsis and Craig Brown for their comments which inspired this post.
Not on the same page, not even reading the same book
In the course of a project it is not uncommon to have stakeholders with conflicting viewpoints on a particular issue. Some examples of this include:
- The sponsor who wants a set of reports done in a day and the report writer who reckons it will take a week.
- The project manager who believes that tasks can be tracked to a very fine level and the developer who “knows” they can’t.
- The developer who is convinced that method A is the best way to go and her colleague who is equally certain that method B is the way to go.
These are but a small selection of the conflicts I have encountered in my work. Most project professionals would undoubtedly have had similar experiences. It can be difficult to reconcile such conflicting viewpoints because they are based on completely different worldviews. Unless these are made explicit, it is difficult to come to for those involved to understand each other let alone agree.
Consider, for example, the first case above: the sponsor’s worldview is likely based on his reality, perhaps a deadline imposed on him by his boss , whereas the report writer’s view is based on what she thinks is a reasonable time to create the reports requested.
Metaphorically, the two parties are not on the same page. Worse, they are not even reading the same book. The sponsor’s reality – his “book” – is based on an imposed deadline whereas the report writer’s is based on an estimate.
So, how does one get the two sides to understand each other’s point of view?
The metaphor gives us a clue – we have to first get them to understand that they are “reading from different books.” Only then do they have a hope in hell of understanding each other’s storylines.
This isn’t easy because people tend to believe their views are reasonable (even when they aren’t!). The only way to resolve these differences are through dialogue or collective deliberation. As I have written in my post on rational dialogue in project environments:
Someone recently mentioned to me that the problem in project meetings (and indeed any conversation) is that participants see their own positions as being rational, even when they are not. Consequently, they stick to their views, even when faced with evidence to the contrary. However, such folks aren’t being rational because they do not subject their positions and views to “trial by argumentation.” Rationality lies in dialogue, not in individual statements or positions. A productive discussion is one in which conflicting claims are debated until they converge on an optimal decision. The best (or most rational) position is one that emerges from such collective deliberation.
The point is a simple one: we have to get the two sides talking to each other, with each one accepting that their views may need to be revised in the light of the arguments presented by the other. Dialogue Mapping, which I have discussed in many posts on this blog is a great way to facilitate such dialogue.
In our forthcoming book entitled, The Heretic’s Guide to Best Practices, Paul Culmsee and I describe Dialogue Mapping and a host of other techniques that can help organisations tackle problems associated with people who are “not on the same page” or “reading different books.”
The book is currently in the second round of proofs. We’ll soon be putting up a website with excerpts, review comments, pricing, release dates and much more – stay tuned!
The pragmatics of project communication
Introduction
Much of the research literature and educational material on project communication focuses on artefacts such as business cases, status reports and lessons learned reports. In an earlier post I discussed how these seemingly unambiguous documents are open to being interpreted in different or even contradictory ways. However, documents are only a small part of the story. Much of the communication that takes place in a project involves direct interaction via dialogue between stakeholders. In this post, I discuss this interactional aspect of project communication, drawing on a book by Paul Watzlawick entitled, The Pragmatics of Human Communication.
The pragmatics of communication
Those who have done a formal course on communication may already be familiar with Watzlawick’s book. I have to say, I was completely ignorant of his work until I stumbled on it a few months ago. Although the book was published in 1967, it remains a popular text and an academic bestseller. As such, it is a classic that should be mandatory reading for project managers and others who work in group settings.
Much of the communication literature focuses on syntactics (the rules of constructing messages) and semantics (the content, or information contained in messages). Watzlawick tells us that there is a third aspect, one that is often neglected: pragmatics, which refers to the behavioural or interactional aspect of communication. An example might help clarify what this means.
Let’s look at the case of a project manager who asks a team member about the status of a deliverable. The way the question is asked and the nature of the response says a lot about the relationship between the project manager and his or her team. Consider the following dialogues, for example:
“What is the status of the module? “ Asks the manager
“There have been some delays; I may be a couple of days late.”
“That’s unacceptable,” says the manager, shaking his head.
….
As opposed to:
“What is the status of the module? “ Asks the manager.
“There have been some delays. I may be a couple of days late.”
“ Is there anything I can do to help speed things up?”
….
Among other things, the book presents informal rules or axioms that govern such exchanges.
The axioms of interactional communication and their relevance to project communication
In this section I discuss the axioms of interactional communication, using the example above to demonstrate their relevance to project communication.
In the presence of another person, it is impossible not to communicate: This point is so obvious as to often be overlooked: silence amounts to communicating that one does not want to communicate. For example, if in the first conversation above, the team member chooses not to respond to his manager’s comment that the delay is unacceptable, the manager is likely to see it as disagreement or even insubordination. The point is, there is nothing the team member can do that does not amount to a response of some kind. Moreover, the response the team member chooses to give determines the subsequent course of the conversation.
Every communication has two aspects to it: content and relationship: Spoken words and how they are strung together form the content of communication. Most communication models (such as sender-receiver model) focus on the coding, transmission and decoding (or interpretation) of content. However, communication is more than just content; what matters is not only what is said, but how it is said and the context in which it is said. For instance, the initial attitude of the manager in the above example sets the tone for the entire exchange: if he takes an adversarial attitude, the team member is likely to be defensive; on the other hand, if his approach is congenial the team member is more likely to look for ways to speed things up. What is really important is that relationship actually defines content. In other words, how a message is understood depends critically on the relationship between participants.
The relationship is defined by how participants perceive a sequence of exchanges: A dialogue consists of a sequence of exchanges between participants. However, the participants will punctuate the sequence differently. What the word punctuate means in this context can be made clear by referring back to our example above. If the team member feels (from previous experience) that the manager’s query is an assertion of authority, he may respond by challenging the basis of the question. For instance, he may say that he had to deal with other work that was more important. This may provoke the project manager to assert his authority even more strongly, thereby escalating discord…and so on. This leads to a situation that can be represented graphically as shown in Figure 1.
Figure 1: Illustration of exchange
The important point here is that both participants believe they are reacting to the other’s unreasonableness: the team member perceives groupings 1-2-3 , 3-4-5 , where his challenges are a consequence of the “over-assertive” behaviour of the project managers etc. whereas the project manager perceives groupings 2-3-4, 4-5-6 etc., where his assertive behaviour is a consequence of the team member’s “gratuitous” challenges. In other words, each participant punctuates the sequence of events in a way that rationalizes their responses. The first step to resolving this problem lies in developing an understanding of the other’s punctuation – i.e. in reaching a shared understanding of the reason(s) behind the differing views.
Human communication consists of verbal and non-verbal elements: This axiom asserts that communication is more than words. The non verbal elements include (but are not limited to) gestures, facial expressions etc. Since words can either be used or not used, verbal communication has an binary (on/off) aspect to it. Watzlawick refers to verbal communication as digital communication (and yes, it seems strange to use the term digital in this context, but the book was published in 1967). In contrast, non-verbal communication is more subtle; a frown may convey perplexity or anger in varying intensities, depending on other expressions and/or gestures that are used. Watzlawick termed such communication as analogic.
In the context of our example, the digital aspects of the communication refer to the words spoken by the team member and the project manager whereas the analogic aspects refer to all other non-verbal cues – including emotions – that the participants choose to display. The important point to note is that digital communication has a highly developed syntax but lacks the semantics to express relationships, whereas analogic communication has the semantics to express relationships well, but lacks the syntax. In lay terms, words cannot express how I feel; my gestures and facial expressions can, but they can also be easily misunderstood. This observation accounts for many of the misunderstandings that occur in project and other organizational dialogues.
All communicational interchanges are either symmetrical or complementary, depending on the relationship between those involved: Symmetry and complementarity refer to whether the relationship is based on equality of the participants or differences between them. For, example the relationship illustrated in figure 1 is symmetrical – the PM and the team member communicate in a manner that suggests they see each other as peers. On the other hand, if the team member had taken a submissive attitude towards the PM, the exchange would have been complementary. Seen in this light, symmetrical interactions are based on minimization of differences between the two communicators and complementary relationships are based on maximization of differences. It should be noted that one type of interaction is in no way better than the other – they are simply two different ways in which communication-based interactions occur
Communication can be improved by strengthening relationships
In the interactional approach to communication, the relationship between participants is considered to be more important than the content of their communication. Unfortunately, the relational aspects are the hardest to convey because of the ambiguity in sequence punctuation and the semantics of analogic communication. These ambiguities are the cause of many vicious cycles of communication – an example being the case illustrated in Figure 1.
Indeed, the interactional view questions the whole notion of an objective reality of a particular communicative situation. In the end, it matters little as to whose view is the “right” one. What’s more important is the recognition that a person’s perception of a particular communicative situation depends critically on how he or she punctuates it. As Watzlawick puts it:
In the communicational perspective, the question whether there is such a thing as an objective reality of which some people are more clearly aware than others is of relatively little importance compared to the significance of different views of reality due to different punctuations.
In their book, they also point out that it is impossible for participants to be fully aware of the relational aspects of their communication (such as punctuation) because it is not possible to analyse a relationship objectively when one is living it. As they put it:
… awareness of how one punctuates is extremely difficult owing to another basic property of communication. Like all other complex conceptual systems which attempt to make assertions about themselves (e.g. language, logic, mathematics) communication typically encounters the paradoxes of self-reflexivity when trying to apply itself to itself. What this amounts to is that the patterns of communication existing between oneself and others cannot be fully understood, for it is simply impossible to be both involved in a relationship (which is indispensable in order to be related) and at the same time stand outside it as a detached, uninvolved observer…
The distinction between content and relationship is an important one. Among other things, it explains why those with opposing viewpoints fail to reach a genuine shared understanding even when they understand the content of the other positions. The difficulty arises because they fail to relate to each other in an empathetic way. Techniques such as dialogue mapping help address relational issues by objectifying issues, ideas and arguments. Such approaches can take some of the emotion out of the debate and thus help participants gain a better appreciation of opposing viewpoints.
To sum up
The interactional view of communication tells us that relationships are central to successful communication. Although traditional project communication tools and techniques can help with the semantic and syntactical elements of communication, the relational aspects can only be addressed by strengthening relationships between stakeholders and using techniques that foster open dialogue.
Eight to Late 