Why Complex Projects Fail – the Cynefin Framework explains why!

Why is Crossrail so late? Why is HS2 twice the cost? Why is there a £42bn spending gap in the MOD?

On our travels, we have seen organisations wrestle with Portfolios of large Projects, using Simple Domain or, at best, Complicated Domain techniques and tools. And get things like forecasts, budgets, and time-scales hopelessly wrong!

So instead they should treat the Portfolio as a Complex Domain. This Domain has an Environment (such as Requirements, Technology, Regulatory Environment, Stakeholders, etc.) that can change (sometimes significantly, and several times over) over the life of one or more Projects. The Domain has a System for delivering Projects – as opposed to treating each Project separately, it is the Project Delivery System that should be examined, rather than each Project, if any serious and lasting improvement is to be made.

The Cynefin Framework helps explain why …

In our last 2 blogs on this subject we reviewed the Simple Domain and the Complicated Domain. We looked at examples of these Domains and the Tools and Techniques that work in them. For example, Best Practice, when applied correctly, works well in the Simple Domain, but not in any other. Management Action is centred on Sensing, Categorising and then Responding. In the Complicated Domain, Cause & Effect Analyses work well, but possibly less well in the Complex Domain. Action is centred around Sensing, Analysing (in context) and Responding.

In this blog we move on to the Complex Domain. In the Complex Domain, Categorising and Analysing are of limited benefit. An understanding of the whole System and Synthesising its components is required, and not the analysis of individual detail.

Complex Domain (Volatility = High; Uncertainty = High; Complexity = High; Ambiguity = High): This Domain is where Complex Systems, Processes and Stages will be performed to deliver Products or Outcomes. Let’s take an example of an investigation into a murder, or the process of developing new drugs. Both of these Environments are Complex – there is no obvious linear Flow through a given set of parties carrying out a specific set of actions in a specific order. There are usually large numbers of elements that makes understanding either the Environment or the System almost impossible. Both the Environment and the System may be “open” – i.e. it’s difficult to work out where the start and the end is without the picture becoming monumentally huge and intractable. Parts of the Environment (and hence the Systems to handle) can be ignorant of the behaviour of the Environment (or System) as a whole, responding only to what is available to them locally. Interactions can be “non-linear” – e.g. small changes can have large effects. To cope with the Environment, the System will need to operate far from equilibrium – there will be no place of rest – it will always be consuming energy in order to maintain some sense of understanding and some semblance of order in the Environment.

Furthermore, the Environments (and hence Systems that operate in such) evolve – they have a history – what was “good” a year ago, may no longer be “good” now.

Therefore many techniques and tools designed for Simple and Complicated Domains will almost certainly not work, or deliver poor results in a Complex Domain. So-called Best Practice will not work. The Logical Thinking Process and Cause & Effect may work, but any proposed solutions may only be valid for a limited period of time. Analysis is not what is required. Synthesis is required and needs to be repeated at the frequency of change. And in a Portfolio of large Projects context, use of Time-series Reference Class Forecasting would be refreshing!

And Complex Domains can slip into Disorder as well – the problem may be in having the right measurement system in place to be able to recognise the difference! A simple thing that any of us who drive will have observed: Individuals travelling along the motorway, will have experienced sudden bunching of vehicles (followed by rapid braking, switching of lanes, stop-start, etc.), and then, just as suddenly it all clears with no obvious cause. Generally speaking, this is a Complex Domain – a Complex Environment, with Complex-Adaptive Systems (i.e. the drivers, infrastructure and all the rest) operating processes to get from A to B. The bunching is the Environment and the System falling over into temporary Disorder, and then recovering from it.

For some, the Complex Domain might be too difficult to manage – in which case, call Dilbert!

…. Next time, we’ll look at the fourth and final Domain – Chaotic – and consider methods to use in this context.