Short works

Books : reviews

Robert Axelrod, Michael D. Cohen.
Harnessing Complexity: organizational implications of a scientific frontier.
Basic Books. 2000

rating : 2.5 : great stuff
review : 18 March 2003

This is a very nice little book. The authors build a simple conceptual framework capturing the main components of Complex Adaptive Systems (CAS), showing how the concepts link together and behave. They do this in order to show how the ideas might be used to develop better businesses and organisations, but the framework is much more generally applicable.

Brutally summarising: the CAS framework comprises populations of agents that strongly interact with one another and their environment via strategies to fulfil some sort of purpose; the environment contains artefacts, that have affordances (features that evoke behaviours of the agents) rather than purposes; the agents and strategies have variation, which lead to different measures of success, used to guide the selection of new agents and strategies, which can lead to adaption; a designer introduces new artefacts, strategies and agents into the world; a policy maker alters behaviour by changing the measures of success (of course, the trick is to predict how to set the measures to get the desired behaviours!).

Each of these concepts is accompanied by a succinct and clear definition.

Our rough criterion for the boundaries of a population will be that two agents are in the same population if one agent could employ a strategy used by another.

The presence of "designer" and "policy maker", and "measures of success" that are not simply survival, show how the full CAS framework is more general that its biological inspiration:

Our approach is not just an extended "evolutionary metaphor" ... We view processes of biological change as wonderful examples in the larger set of Complex Adaptive Systems. However, they have special kinds of agents, particular sorts of strategies, distinctive patterns of interaction, and their own special processes of selection. The patterns one sees in biology are not always found in other Complex Adaptive Systems. ... Variation, interaction, and selection are at work in a population of business strategies, but the detailed mechanisms are often distinctly unbiological. To harness complexity effectively, many kinds of Complex Adaptive Systems must be considered.

For example, natural biological selection copies complete agents, whereas a general CAS can also copy strategies, which can be more efficient, or cheaper. Copies of agents, whether biological organisms or physical organisations, usually require matter, and so are expensive. (But not always: copies of "computer viruses" and other virtual agents are cheap.) Copies of strategies, on the other hand, usually require the copying of abstract information, and so are generally much cheaper. So, as the authors point out, although the study of CAS is inspired by biology, it covers a much wider field.

The authors' aim is to use the concepts of CAS to suggest what might be good strategies for adapting and surviving in the complex world of business. These essentially have to be meta-strategies, because a complex world exhibits constant novelty, and detailed prediction is impossible.

When multiple populations of agents are adapting to each other, the result is a coevolutionary process. ... efforts to adapt may change the context ... This can lead to perpetual novelty for both sides. The system may never settle down.

The complex systems world is a world of avalanches, of "founder effects" ..., of self-restoring patterns ..., of apparently stable regimes that suddenly collapse. It is a world of punctuated equilibria ..., and butterfly effects .... It is a world where change can keep recurring in a fixed pattern, where rapid and irreversible change can occur when a certain threshold of effect is reached, and where great variety can exist at a large scale, even though small patches have very little variety. These are not completely disorderly worlds, so turbulent that useful lessons can never be learned. They have structure, and beneficial adaptation can sometimes occur. But prediction and choice of the conventional kind are not very reliable.

complexity indicates that the system consists of parts which interact in ways that heavily influence the probabilities of later events. Complexity often results in features, called emergent properties, which are properties of the system that the separate parts do not have. ... a system should be called complex when it is hard to predict not because it is random but rather because the regularities it does have cannot be briefly described.

The main meta-strategy is recognising the necessity of a tradeoff between between exploration (too much of which leads to "eternal boiling", a system in chaos where new ideas can never be developed to fruition) and exploitation (too much of which leads to "premature convergence", where focus on an initial success cuts off the possibility of future improvements).

The authors discuss various mechanisms that can be used to encourage exploration (such as making interactions more diffuse) or exploitation (such as erecting barriers, or creating neighbourhoods). They emphasise that there can be conflicts between short term goals and long term behaviour, between what the agents want and what the policy makers want, and between the information agents have to act on and the real state of the system. Although the meta-strategies can be used to encourage appropriate amounts of novelty and exploration, there are no guarantees about what will actually be produced or discovered.

Another meta-strategy is to recognise the fundamental difference between classical continuous models, and discrete models where quantities can go to zero, where agents can "become extinct". The former are the kind of models often used for prediction and forecasting. The latter are essential for understanding CAS, and have very different behaviours and properties.

conventional theories ... are often based on assumptions of continuous variables. ... a tiny fraction of each animal type is always around, so that no matter how severe the starvation, the predator population will rebound as soon as prey return. There is no complete extinction in such models. A nano-fox is always lurking in the shadows.
     But in real populations the difference between having a few animals and zero animals is usually not just a little extra waiting time. Recreating a lost type is very unlikely, and occupation of the vacant ecological niche by another species is far more to be expected. Because Complex Adaptive Systems researchers are especially interested in variety, they often use modeling tools that allow genuine extinction.
     This is much more than a minor difference about the technical tools of systems modeling. The tools embody widespread habits of thinking about variation. A habit of ignoring the sharp effect of an extinction is inconsistent with many important social and policy settings. .... These "zero points" ... correspond to sharp changes in the later dynamics. ...
     A related notion in continuous modeling traditions is that all possible types already exist in tiny quantities. .... the Complex Adaptive Systems view ... suggests that a new idea may not simply be waiting in the wings for the circumstances that will bring it rapidly to prominence. It matters enormously whether the number of people who have thought of it is one or zero. .... "counting on the market to find a solution" can be expected to work far more rapidly and reliably in domains where several approaches have been partially worked out, as opposed to domains in which a feasible approach is yet to be conceived.
     The underlying source of this sharp effect of zero is that copying mechanisms work quite differently from mechanisms that recombine types in context. .... While reproduction can be quick, creation may require a long wait. ... the special value of rare types ... preserving plant species for future medicinal discoveries ... incubation of small businesses and preservation of skills that are vanishing in the Information Revolution.

The CAS framework can also be used to suggest strategies for structuring organisations.

[Herbert Simon] examines the tendencies of many biological and social systems to assume hierarchical ... shape. ... the upper layers of such systems typically involve processes that span longer time intervals, while the lower levels are more often involved with processes that run relatively quickly. ....
     Simon argues that this hierarchical arrangement of time scales supports effective governance in a system ... the slower activity at the upper levels establishes a stable context for faster processes running at lower levels.

Some CAS may be self-organising critical systems (SOCS). It is important to recognise these cases, because they give rise to new kinds of problems, new kinds of potential failure modes. In the critical state, into which the system self-organises, "shocks" can happen on all scales, due to build-up of stresses.

after [SOCS] build up to their critical state ... a long time without a big event does not necessarily mean that something big is due soon. There is such a complicated interdependence among all the [parts] that you cannot know whether small events are relieving or increasing the stress. ...

We use redundancy to protect against independent shocks, and diversity against correlated shocks. But SOCS have a new kind of failure shock mode.

     It is ironic that in our efforts to stabilize systems against independent or correlated failures, we often transform them into more tightly coupled systems that redistribute stress. ...
     ... stress propagation failure becomes possible when the elements interact naturally, or are designed to interact. Here the risk is that a failure in one element can cause stress in another element, leading to failure of that element as well. Eventually a cascade of failures could cause a large-scale failure. ... advances in information systems allow more and more systems of different kinds to be designed in ways that provide efficiency through a close coupling of their elements, with attendant risks of large-scale failures ....
     Unless the coupled structure of the situation is changed, interventions to stave off catastrophic releases can only be expected to be briefly effective. ... For systems in the critical state, an event from some quarter will eventually trigger a huge chain of effects. .... The treatment of self-organized criticality does not argue that there is no postponement, only that a local intervention will provide no relief in the long run.

Another meta-strategy is the choice of success measures. We all know examples of how a change to the "reward" system, introduced to encourage a particular behaviour, can sometimes lead instead to undesired behaviour (such as measuring departments on their "bottom line" to promote efficiency, only to promote damaging rivalries; or raising the interest rate to reduce another parameter, only to watch it continue to rise, or to see some third parameter change for the worse). Part of the problem is that the credit attribution is not always perfectly correlated with the desired property. We need better strategies for assigning credit. There are some things we can do.

we look for ways that the inevitable mistakes of credit attribution can provide opportunities to harness complexity. .... Surprises are actions that came out better, or worse, than expected. Either kind can fuel improvement. The essential thing is to see what factors were observable or predictable in the short run that were correlated with the surprise. ...
      ... The search is not for what predicted the outcome but for what predicted the surprise, the deviation of your expectations from what occurred. Those are the factors to which you should give increasing credit if you want to speed the process of learning which factors to credit.

All in all, there is much fascinating food for thought in here. The book is slim, but well written with no wasted words or needless repetition. The arguments are put cogently. It will not please everybody -- there is no silver bullet, no magic set of steps to follow that guarantees success in a complex world. But the CAS framework here gives a structure for thinking about that complexity, and the meta-strategies give some guidelines for ways to adapt better. Recommended.