Archive for the ‘Organizations’ Category
The elusive notion of context
It would have been my second or third day in this country. There was a lot going in my head at the time: the worry of a small bank balance, the tension of finding a job, the assorted insecurities of finding one’s way in a strange land. I was walking back to my motel in the evening, lost in thought when, quite unexpectedly, I caught the unmistakable aroma of eucalyptus and wattle, a scent I had grown up with in my childhood years in the Nilgiri Hills.
It was strangely comforting. A place that felt unfamiliar moments earlier seemed much less so, a small step towards a sense of belonging…
–x–
A few years later, I was working on a research problem relating to a phenomenon that is easily demonstrated in a kitchen sink. If you are interested, make your way to the nearest sink and do the following:
Turn the tap on slowly until water starts to flow out as a cylindrical jet. You will notice that the jet breaks up into near spherical droplets a short distance from the mouth of the tap.
This phenomenon is called jet breakup. Instead of describing it further, I will follow the advice that a picture is worth several words.
The problem of jet breakup was first studied by Lord Rayleigh in 1878 but had enjoyed a renaissance in the 1970s and 80s due to applications such as inkjet printing. My project was to apply the principle to the production of shot from jets of molten metal. I was not too enthused by the problem as it was, at best, a minor variation of a theme that had already been done to death. However, I was being paid to do research so I worked on it as diligently as I could, making desultory progress.
One October evening, about two years into my project, I was washing up after dinner when I noticed a curious wave-like structure on the thin jet that emerged from the kitchen sink tap and fell onto a plate an inch or two below the tap (the dishes had piled up that day). The wave pattern was stationary and rather striking.
The phenomenon is one that countless folks have seen. Indeed, I had noticed it before but never paid it much attention until that October evening when I saw the phenomenon with new eyes. Being familiar with the work of Rayleigh and others, I realised, at once, that the pattern had the same underlying cause as jet breakup. Wondering if anyone had published papers on it, I dashed off to the library to do a literature search (Google Scholar and decent search engines were still a few years away). Within a few hours I realised I’d stumbled on a new context that would change the direction of my research.
–x–
Years and a couple of career changes later, I was working in a large multinational. Through sheer luck of being at the right place at the right time, I got to do some well-regarded work in business intelligence and analytics. One of the things I realized during these projects is that the technical aspects of the work are the easiest. The hardest part is figuring out what to do…and this is hard because of people, not technology.
It was my first encounter with problem wickedness, the notion that problems involving diverse groups of individuals are socially complex. I started reading and thinking about practical ways to deal with social complexity in organizations, an effort that led to my collaborative work with Paul Culmsee. The key point, elaborated at length in our work, is that progress on tackling such problems depends critically on building a common context, one that all stakeholder groups can relate to.
–x–
There is a lot of published nonsense about building productive teams, much of it circulating on the internet. I will refrain from singling out specific articles because there are so many, but you can find them for yourself using search phrases such as “building teams”. The fact of the matter is that there are no special levers one can pull to make good teams. As JR Hackman, a well-known researcher on team behaviour stated in a review article entitled, From causes to conditions in group research:
Influences on group behavior and performance do not come in separate, distinguishable packages. They come, instead, in complex tangles of redundant features and forces. To try to partial out and assess the causal effects of each component can be an exercise in frustration because each ingredient of what may be a spicy stew loses its zest when studied separately from the others.
Hackman urges researchers (and by implication, practitioners) to forget about causes and focus on creating the right conditions for a team to do good work. The real work of a leader is to set the right context and get out of the way. As David Snowden tells us in this talk, “Leadership is a property of the way you interact with people, not a property of status or training…[it] is about creating the right context for good things to happen.”
If you are interested in finding out more about building context, check out David Snowden’s writings on the Cognitive Edge blog or the books I have linked to above. For a much longer and very enlightening series of discourses on the importance of context, I highly recommend Gregory Bateson’s magnum opus, Steps to an Ecology of Mind. The work is a collection of writings that deal with things far removed from management and organisations, but you will slowly start to see its relevance as you digest his many-splendoured essays on anthropology, evolution, animal behaviour, art, psychology, information, communication and more.
–x–
David Snowden’s Cynefin Framework offers a useful way to think about organizational problems. As an example, I have found the complicated / complex distinction useful in triaging problems frequently encountered in organizational settings. In brief, complex problems tend to have a much higher degree of social complexity compared to complicated problems. They are, in other words, wicked problems. Most organizational problems that matter are complex, not complicated.
Quite apart from the utility of the framework, I find the origin of the name Cynefin enlightening. The term is commonly (and poorly) translated as habitat. A better translation is the phrase, a place of belonging. This resonates with the following passage excerpted from a paper Snowden wrote about the origins of the term:
For Christmas 1998 my parents followed a long tradition of giving me a book related to Welsh culture. That year it was The Land and the Sea, a collection of the works of Kyffin Williams, published to celebrate his eightieth year. I have admired his work from when I first saw a portrait painted by him, of a much-loved headmaster a quarter of a century earlier. He is an artist who has a profound understanding of place, both in his landscape work and his formal and informal portraits of the people of his native land, the Brythonic Kingdom of Gwynedd which dominated what is now Wales from the fifth to the thirteenth centuries. The Romans knew it as Venedotia, or the land of Venus, and I had the privilege of growing up there, walking its hills and sailing its coast. To sit on the summit of Tryfan as the sun descends over the Glyder ridge and Y Garn, casting shadows on to the precipitous slopes of Pen yr Ole Wen is a profound experience, not just of the aesthetic beauty of the landscape, but of one’s place, one’s identity, one’s place of belonging. There is a welsh word, Cynefin, which means all of that and more and has no equivalent in the English language, where it is crudely translated as habitat or place. In his preface to Kyffin Williams’book, Nicholas Sinclair connects the word Cynefin to the interaction between human beings and their environment that is the essence of the author’s work. I took that word as the name for this model created to understand the different types of system within which we operate: ordered, complex and chaotic.
Snowden’s words resonate with my professional experiences in a wide variety of organisations as well as my personal experiences of dealing with the unfamiliar. Until I feel I belong and can relate to others in that environment, I cannot make any meaningful progress. This is a prerequisite to initiating and achieving successful change of any kind, be it an organizational restructure, a large-scale IT project or even settling into a new country.
The sense of belonging is particularly important for work that involves different groups of stakeholders, particularly those that have conflicting interests and values. In such cases it is critical to achieve a shared understanding of the problem one is trying to solve before attempting to solve it. Such an understanding is often assumed at the start but is then found to be illusory when the “solution” is implemented which, of course, is much too late. The work of building a shared understanding is essentially an act of collaborative problem formulation or sensemaking. It is, in other words, the creation of a common context.
–x–
Years later still, I was asked to move to a regional role in the same multinational. My role there was to look after IT development activities in the region. Since much of this was driven from the corporate head office, much of my work was about resolving the gaps between what various offices scattered across the region needed and what the corporate-mandated solution offered. My experiences in sensemaking helped, but the official aspects of the role were – not to put too fine a point on it – excruciatingly boring.
There was, however, a saving grace. When I was hired, my boss told me that he wanted me to explore the possibility of setting up a regional development centre for analytics that could serve the entire organization. There was a clear cost argument in favour of such centre. However, given the sharply divided opinions around offshoring, the public airing of such a proposal would cause all kinds of reactions, many of which would be negative. The first problem was to address those upfront.
Oh, and I forgot to mention, my boss told me I had zero budget to do this.
I talked to the usual suspects, a few big outsourcers and consultancies, but soon realized that their aims were not congruent with mine. Everything they told me pointed to high costs and potential conflicts down the line, not the least being vendor lock in and expensive contract variations. As I have pointed out elsewhere, the hidden costs of outsourcing are much too high.
Around that time, I came across the notion of emergent design. The essential idea is to start from where people are and take small steps, each of which lead to demonstrable improvement. This generally requires some trial and error, but since the investment at each step is small and the benefit demonstrable, it is not hard to convince the folks who sign cheques. Moreover, this enables one to continually adjust one’s approach based on feedback from the previous step, much like nature does in the process of evolution. Actions are based on a given context, but the context itself changes because of the action and thus necessitates recalibrating subsequent actions.
To begin with, I needed to develop a small use case that would clearly demonstrate the benefit of hiring developers in my region to do work for people located in the head office, but in a way that alleviated concerns about job losses. As it happened, I had a colleague in head office who led a database team that was struggling to keep up with their workload. They simply did not have enough people and lacked the budget to hire contractors locally. I had a phone conversation with my colleague and suggested hiring someone who would sit in my office and work for him remotely. He was enthusiastic once he heard the costs (much lower than local consultants!) and committed funding for year.
The fledgling centre thus made its first hire. A few months forward, my colleague was happy with the result. Consequently, he did a lot of evangelizing on my behalf about how this arrangement did not take away jobs, but augmented capacity.
Requests started trickling in; a few short months later they were pouring in. Three years later, there were over fifty developers in the centre. It had become a thing.
–x–
In recent years, advances in computing have made it possible to process vast volumes of data. It is now possible for data scientists to apply machine learning techniques to large datasets containing detailed information about individuals. There seems to be a widespread belief that, given the right kind of data, it is possible to abdicate decision-making on social matters to machine learning models. Of course, there are the warnings of data ethicists and others about the dark side of data science, but these tend to be swamped by the voices of data optimists. Data is so revered that large datasets tend to take on a life of their own, stripped of the context in which they were originally gathered. There are enough examples that illustrate why this is problematic. If one could save not just a dataset but also details on how it was collected and why, the limitations of the data would be more apparent thus making it is less likely that it would be used in inappropriate ways. The proposal on datasheets for datasets by Timnit Gebru and collaborators is a good start towards addressing the issue of context in data science.
One must keep in mind that cause-effect relationships are not obvious in large social systems. There are myriad plausible mechanisms of change and, more often than not, several of them will interact and operate in tandem. Given this, any decision based on data alone is likely to be misleading. Instead, success is more likely to come from an emergent approach based on developing multiple perspectives on the data – i.e. by building context around it.
–x–
Building context in social systems is hard because cause-effect relationships are hard to disambiguate and pin down. Consequently, it is impossible to know, with certainty, what is relevant and what is not. Indeed, if it were possible at all, one would include all details, leaving nothing out. The model of the world would then be the world itself; the map would be the entire territory. But as Borges so eloquently noted:
… In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless…
Building context is a never-ending quest to understand the territory.
–x–
…and so I have been here for many years now. The smell of eucalyptus and wattle still evokes memories of home, but I do not know which one. What I do know now, but did not know then, is that being at home in a place or a set of ideas is the work of a lifetime.
Collaborative reasoning in the age of Covid
Ever since the start of the pandemic, there have been no end of opinions, presentations and reports on how we might navigate our way out of the crisis. Much of this takes a narrow, discipline-centric view, which is inadequate because the problem is multifaceted and thus defies traditional disciplinary boundaries. It is therefore of urgent importance to chart a course that considers all aspects of recovery, not just those relevant to specific interests. A recent report produced by the Australian Group of Eight does just that. The key points of the report are concisely described in an executive summary and snapshot, so I will cover just the main points in this article. My focus instead is on the platform used to create the report, as it offers an effective collaborative approach to tackling complex issues in a broad range of contexts.
To me the most amazing thing about the 192-page report is that it was produced by a taskforce comprised of over a hundred academics and researchers across diverse disciplines, collaborating over a three-week period. As stated in the exec summary:
To chart a Roadmap to Recovery we convened a group of over a hundred of the country’s leading epidemiologists, infectious disease consultants, public health specialists, healthcare professionals, mental health and well-being practitioners, indigenous scholars, communications and behaviour change experts, ethicists, philosophers, political scientists, economists and business scholars from the Group of Eight (Go8) universities. The group developed this Roadmap in less than three weeks, through remote meetings and a special collaborative reasoning platform, in the context of a rapidly changing pandemic.
Those who have done any collaborative work involving large groups will have stories to tell about how challenging it is to get a coherent result. This taskforce achieved this in part by working on an online collaborative reasoning platform called SWARM, described in this paper. This post is mainly about what SWARM is and how it works, but I will also describe how the Roadmap taskforce used the platform to come up with a comprehensive recovery plan and the key recommendations made therein. I’ll end with some thoughts on the use of SWARM in broader organizational and business contexts.
The SWARM platform
The platform was designed and implemented by a team led by Drs. Tim van Gelder and Richard de Rozario as part of a large Intelligence Advanced Research Projects Activity (IARPA) initiative. In essence, SWARM is a cloud collaboration environment designed to enhance evidence-based reasoning in teams. It does this by supporting an approach called contending analyses, wherein team members produce and refine multiple distinct analyses of a problem, and then select the best one as their collective response.
On SWARM, team members create artefacts that represent their reasoning. Additionally, they can rate, comment on and contribute to artefacts created by others through the course of the challenge. This enables a “best response” to emerge through an iterative process of discussion, refinement and evaluation.
To understand how it works, it is necessary to briefly describe the various ways in which users can interact and contribute to solving the problem with each other in SWARM. The user interface of the SWARM platform consists of three panes (Figure 1).
Figure 1: SWARM user interface
The left pane contains the problem description and links to related documents. In the centre pane, users can post and update responses. A response may be a Resource (e.g. a link to an external article, a visualisation or an analysis) that contributes to understanding or solving the problem, or it may be a Report, which is a draft candidate for the team’s final output. Users can then comment on and rate others’ responses and comments. The most highly rated Report at the conclusion of the problem is submitted as the result of the group’s collaborative reasoning.
The right pane is a streaming chat window through which users can interact in real-time. To summarise, SWARM users can:
- converse with team members via the chat feed.
- post or update a Resource or a Report
- comment on a Resource or a Report, or
- rate a Resource, Report or Comment.
By design, SWARM does not prescribe (or proscribe) any particular analytical process. As van Gelder, de Rozario and Sinnott (2018, pp. 22-34) note, contending analyses:
…promotes engagement by providing the opportunity for any participant to contribute their own thinking (autonomy), to think in a manner matching their natural expertise (mastery or competence), and to earn the respect of others by drafting a well-regarded response (relatedness)’ – thus meeting each of the three psychological needs identified by self-determination theory.
The idea is that teams should be free to work in ways that suit them collectively, with individuals given the choice to contribute as and when they please. That said, SWARM, via its Lens Kit (https://lenskit.atlassian.net/wiki/spaces/LK/overview), offers participants a compendium of structured analytical techniques and other “logical lenses” that may be useful in analysing complex and uncertain scenarios in which the available information is scarce or ambiguous.
The Roadmap to Recovery project
The Roadmap project involved over a hundred academics from the Group of Eight – a coalition of the oldest, largest and most research-intensive Australian universities. Over three weeks in April 2020, the team worked on developing scenarios for national recovery from the COVID crisis. Their recommendations are available in a comprehensive report. The report is unique in that it synthesises the knowledge of a range of experts and takes a systemic, evidence-based view of the problem. In the words of the co-chairs of the project:
How this document differs from the hundreds of articles and opinion pieces on this issue is that this report specifies the evidence on which it is based, it is produced by researchers who are experts and leaders in their area, and it engages the broadest range of disciplines – from mathematicians, to virologists, to philosophers.
Over a three-week period, this taskforce has debated and discussed, disagreed, and agreed, edited and revised its work over weekdays and holidays, Good Friday and Easter. All remotely. All with social distancing…
…It is research collaboration in action – a collective expression of a belief that expert research can help Government plot the best path forward…
Given the wide geographical distribution of the team and the requirement for social distancing, it was clear that the team needed an online collaboration platform that enabled collective deliberation. Traditional online methods would not have worked for a group this large. As noted in the report:
Standard remote collaboration methods, such as circulating drafts by email, have many drawbacks such as the difficulty of keeping track of document versions, integrating edits and comments on many different versions, and ensuring that everyone can see the latest version. It seemed clear this approach would struggle with an expert group as large as the Roadmap Task Force.
The steering committee therefore decided to give SWARM a go.
As noted in the previous section, SWARM works on the principle that a group should canvas multiple approaches and then collectively settle on the best one, a principle summarised by the term contending analyses. The benefit of such an approach is evident in the report in that it outlines two distinct strategies for recovery:
- Elimination: as the term suggests, this strategy aims at eliminating the virus within the country. This is the lowest risk approach and is technically feasible for a relatively isolated country like Australia. However, the cost in terms of time, effort and money is substantial. Moreover, a strict implementation of this approach would bar international travel for an unrealistically long duration.
- Controlled Adaptation: this involves controlling the infection within the country to a level that does not overwhelm the healthcare system. This is less expensive in terms of time, effort and money, but the outcome is also less certain. However, as the taskforce points out, this could lead to restrictions being eased as early as May 15th, a choice that the government had made before the report was released. This decision is understandable given the cost of extended restrictions; however, it isn’t clear at all how they will handle the inevitable resurgence of the disease down the line. The report considers how things could develop as a result of this decision.
The report aims to provide a balanced case for the two options, and also emphasises that in terms of implementation, the options have considerable overlap. For instance, there are three requirements for the success of either:
- Early detection and supported isolation
- Travel and border restrictions.
- Public trust, transparency and civic engagement.
It should be clear that all three require massive government involvement and support. To this end, the taskforce has formulated an ethical framework that should guide government decision-making and policy. The framework comprises of the following six principles:
- Democratic accountability and the protection of civil liberties.
- Equal access to healthcare and social welfare.
- Shared economic sacrifice.
- Attentiveness to the distinctive patterns of disadvantage.
- Enhancing social well-being and mental health.
- Partnership and shared responsibility
An ethical framework should serve as a check on policy-making that might disadvantage specific groups. If followed, the six principles listed above will ensure that policies are fair to all sections of the community, both in terms of burdens and benefits This is perhaps the trickiest part of policy-making.
Finally, the taskforce has formulated six imperatives (essential rules) that should guide the actual implementation of a recovery. They are:
- The health of our healthcare system and its workers.
- Preparing for relaxation of social distancing.
- Mental health and wellbeing for all.
- The care of indigenous Australians.
- Equity of access and outcomes in health support.
- Clarity of communication.
Each of the above requirements, ethical principles and rules for action are unpacked in detail in the full report and summarised in the executive brief.
How the project unfolded
The Roadmap process was a bold experiment. The Group of Eight had never attempted to pull together such a large report, with so many participants and diverse perspectives, in such a short time, and where no face-to-face meeting was possible. The SWARM platform, still a research prototype, had never previously been used to address a real problem, let alone a problem of this scale and importance.
The project had a steering committee consisting of the project chairs, Professor Shitij Kapur and Go8 CEO Vicki Thomson, and two reasoning experts from the Hunt Lab, Drs. Tim van Gelder and Richard de Rozario. The committee proposed a project design which would involve two weeks working on the SWARM platform, followed by a week of off-platform final report drafting by a small group from the Go8. The two weeks on SWARM would involve the panel of experts working on 9 major topics, corresponding to the anticipated major sections of the final report, such as “How and when to relax social distancing.” It was expected that the experts would distribute themselves across the topics, with “emergent teams” coalescing to work on producing a draft report for each section. Week 1 on SWARM would be mostly “exploratory” thinking, with panelists mostly posting Resources, comments and chat. Week 2 would be mostly “synthetic” thinking, with emergent teams posting early draft Reports for each topic, and collaboratively refining the most promising drafts. In Week 3, these draft section reports would be integrated into a single overall final report.
The steering committee planned to closely monitor progress over the first two weeks and, if/as necessary, modify the process. The project did unfold largely as planned, but the steering committee had to intervene mid-late in the second week when it was apparent that some topics lacked emergent teams with “critical mass,” and in some cases even where critical mass had developed, the teams needed some guidance and prodding to deliver an adequate section report. At this point, the committee, and in particular one of the Chairs, Shitij Kapur, convened a series of zoom meetings meetings the emergent teams, and developed with them a plan for finalising their section reports. From that point on, most work on the draft section reports was done, over just a few days, using more traditional collaboration techniques, such as as circulating a Word document and communicating by email.
Thus, as things turned out, the process was a novel hybrid of a pure SWARM platform-based approach, and more standard methods. The steering committee were committed from the outset to expediency in getting the intended result (a high-quality final report) rather than being “purist” about the approach being used. The use of more traditional collaboration tools and methods later in the process, was driven by a number factors, including some limitations in the SWARM platform (most importantly, the lack of a “track changes” function in the platform’s document editor), and the natural tendency for people to revert to habits and reflexive behaviours when under great pressure. It was clear, however, that the SWARM platform played a crucial role in the first half, allowing participants range across all topics, share lots of ideas and discussion, form emergent teams, and at least start drafting reports.
Whither collaborative reasoning?
The Roadmap project highlights the value of collaborative reasoning platforms like SWARM. It is therefore appropriate to close with a few thoughts on how such platforms can help organisations build internal capability to deal with complex issues that they confront – for example, developing a strategy in an uncertain environment (such as the one we are in currently).
The first point to note is that such problems require stakeholders with diverse viewpoints and skills to work collaboratively to craft a solution. Long-time readers of this blog will know that I advocate tools like Issue-Based Information System (IBIS) to help such groups reach a consensus on problem definition, and thus settle issues around “Are we solving the right problem?” or “How should we approach this issue?” However, once the problem is defined by consensus, the group needs to solve it. This is where platforms like SWARM are particularly useful.
Although SWARM was designed for the intelligence community, the Roadmap project shows that it can be used in other settings. As another example, Tim van Gelder notes that citizen intelligence (where ordinary citizens collaborate on solving intelligence problems) is becoming a thing, but lacks a marketplace. As a possible solution, he envisages the creation of a Kaggle-like platform for complex problems (rather than data problems). He notes that there are challenges around setting up such platforms, but there is interest from large private (non-intelligence) organisations. New deployments of the platform are already underway.
The problems organisations confront in the post-Covid world will be more complex than ever before. There are those who believe such problems will yield to computational approaches that rely primarily on vast quantities of data. However, complex situations cannot be characterised by data alone, so computational approaches will need to be augmented by human sensemaking and reasoning. The success of the Roadmap to Recovery project demonstrates that platforms like SWARM can help organisations tackle such problems by harnessing the power of collaborative reasoning.
Note: For more information on SWARM, please visit the Hunt Lab for Intelligence Research.
Acknowledgement: My thanks to Dr. Tim van Gelder for reviewing a draft version of this article and for contributing the section on how the project unfolded.
The worth of an education – a metalogue
“To be prepared against surprise is to be trained. To be prepared for surprise is to be educated.” – James Carse in Finite and Infinite Games.
Prospective student: How much do your students earn after they graduate?
Teacher: I’m sorry, I don’t know.
Prospective student: What do you mean? Don’t you keep track of what students do after they complete the program?
Teacher: Yes, we do, but we don’t ask them how much they earn. It is not the kind of question one asks….
Prospective student: I know that, but shouldn’t you be gathering hard data on outcomes?
Teacher: Of course, and we gather a range of data including the roles they get on graduation and how they progress in their careers.
Prospective student: OK, but that doesn’t help me. How do I know that your degree is worth the hefty fees you charge? If I’m investing that much, I must be sure of a decent return.
Teacher: One cannot be certain about anything in life, there are never any guarantees. However, it’s perfectly reasonable to expect value from your education, so let me ask you – what would you consider a decent return?
Prospective student: Hmm, a good job, I guess.
Teacher: Well, you heard from some of our alumni this evening. They are currently employed as professionals in the field, and most of them got their jobs while studying or soon after completing their degree.
Prospective student: Yes, but you are unlikely to invite those who would say negative things about the program or those who have failed to get jobs.
Teacher: Fair enough, there are a few of those.
Prospective student: Well, that’s just my point. How do I know it will work for me?
Teacher: You don’t! Getting a job is an indirect effect of a good education.
Prospective student: I’m not sure I follow.
Teacher: An analogy might help clarify what I mean by indirect, Bill Gates did not become a multi-billionaire by setting out to become one. He became one by following his interests, his passion.
Prospective student: [a tad irritated] I still don’t get it.
Teacher: The objective of education is not to train you for a vocation…although, if you do things right, you are almost certain to get a job, and a good one at that. The aim of education is personal transformation, to broaden your perspective and thus enable you to look anew at the things you do at work and, possibly, even in life. Another aim is to prepare you to become that buzzword: a lifelong learner. No university can teach you those things, but they can help you learn them.
Prospective student: “personal transformation” sounds wonderful but terribly vague. Could you give me an example?
Teacher: Well, you’ve just heard from a few of our alumni and I could tell you many more stories. The thing is, I don’t think it is helpful to hear second-hand stories – their journeys are theirs, not yours. The story I’m interested in right now is yours: what you’re thinking, what you do and where you want to go.
Prospective student: OK. I’m a financial analyst [Editor’s note: feel free to substitute your current profession here] and I want to be a data scientist. Will this course enable me to become one?
Teacher: It could, but whether or not it actually does depends largely on you.
Prospective student: That’s not an answer.
Teacher: It is, and it’s an honest one. No course will make you a data scientist. And if any university tells you they can, they’re lying. What a good university course will do is help you learn the technical and non-technical skills that will enable you to become a data scientist. Whether you learn or not depends on you. The responsibility for your personal transformation lies largely with you. All we can do is show you the way.
Prospective student: so, do you cover … [student recites a litany of data science languages and techniques].
Teacher: Yes, we cover them.
Prospective student: Won’t doing those make me a data scientist?
Teacher: No. If tech skills are all you are after, I’d strongly suggest you don’t join our program…or any other university program for that matter. Instead, head off to one of the good online data science education providers and save yourself a whole lot of money.
Prospective student: Huh?
Teacher: A good face-to-face program at a university covers a whole lot more than tech. For example, there are certain tacit skills and dispositions that are critical to becoming a good data scientist. These skills have to do with problem finding rather than technical adeptness or problem solving. A good university course will give you opportunities to gain experience in doing that.
Prospective student; Problem finding? What’s that?
Teacher: In university assignments you’re given readymade problems that you can go off and solve. In real life, however, you are rarely given a problem. More often, you are presented with a situation from which you must extract or formulate a problem before you solve it. That’s not always a straightforward process because every situation is unique in its details.
Prospective student: If every situation is unique then there is no formula to deal with it.
Teacher: Exactly! These skills have to be taught indirectly – by putting students in safe-to-fail situations in which they can learn how to deal with the ambiguity inherent in them
Prospective student: But won’t that be throwing students into a situation they’re unprepared for.
Teacher: Although they may not admit it, most consultants – even experienced ones – rarely feel totally in control when dealing with new clients. It’s good to experience that kind of ambiguity early in one’s career, even if it is a second career. Every consulting engagement is a learning experience. This ties in with what I mentioned earlier – becoming a lifelong learner.
Prospective student: So how do you prepare students to deal with these types of scenarios?
Teacher: Through carefully crafted technical and non-technical subjects, with assignments that make them think rather than just do. To do well in the assignments you will have to think things through, try different approaches and even make judgement calls.
Prospective student: Judgement calls?
Teacher: Yes, that’s right. You will find that the biggest issues when doing data science in the real world are not technical, rather they are about dealing with ambiguous situations in which you don’t have a well-defined problem or adequate data. Then there are ethical issues that are becoming ever more important today. There are big corporations that completely ignore the ethical implications of what they do. Just because you can do something, it doesn’t mean you should. All these issues involve judgement calls in which data is of little or no help.
Prospective student: Hmm, I didn’t realise there were so many facets to being a data scientist. Thanks, you’ve given something to think about.
Teacher: No worries…and good luck, I hope you find what you’re looking for.
Afterword:
HR gurus and consultants continually pontificate about the future of work. The ground reality for many mid-career professionals is that the future of their work is highly uncertain, much of the uncertainty being fuelled by the perception that data-related technologies are going to “disrupt” established industries. Among other things, this has led to an unprecedented demand for courses that teach data-related skills.
What is often left unsaid, however, is the transition to data science – or any profession for that matter – involves more than just picking up technical skills. The biggest missing piece (in my opinion) is the ability to make sense of ambiguous situations. This is a tacit skill that is difficult, if not impossible, to teach but can be learnt given the right environment and attitude. The university ought to provide the environment, the student the attitude.
Note: A metalogue is a dialogue that unfolds in such a way that the structure of the conversation turns out to be illustrative of the issue being discussed. The anthropologist Gregory Bateson coined the term. Here is a metalogue written by him.
Eight to Late 