CHAOS AND COMPLEXITY
Chaos and Complexity
inipfertä« taw
By the middle of the 20th century, mathematics was undergoing a rapid phase of growth, stimulated by its widespread applications and powerful new methods. A comprehensive history of the modern period of mathematics would occupy at least as much space as a treatment of everything that led up to that period. The best we can manage is a few representative samples, to demonstrate that originality and creativity in mathematics are alive and well. One such topic, which achieved public prominence in the 1970s and 1980s, is chaos theory, the media's name for nonlinear dynamics. This topic evolved naturally from traditional models using calculus. Another is complex systems, which employs less orthodox ways of thinking, and is stimulating new mathematics as well as new science.
Chaos
Before the 1960s, the word chaos had only one meaning: formless disorder, But since that time, fundamental discoveries in science and mathematics have endowed it with a second, more subtle, meaning - one that combines aspects of disorder with
356
aspects of form. Newton's Mathematical Principles of Natural Philosophy had reduced the system of the world to differential equations, and these are deterministic. That is, once the initial state of the system is known, its future is determined uniquely for all time. Newton's vision is that of a clockwork universe, set in motion by the hand of the creator but thereafter pursuing a single inevitable path. It is a vision that seems to leave no scope for free will, and this may well be one of the early sources of the belief that science is cold and inhuman. It is also a vision that has served us well, giving us radio, television, radar, mobile phones, commercial aircraft, communications satellites, man-made fibres, plastics and computers.
The growth of scientific determinism was also accompanied by a vague but deep-seated belief in the conservation of complexity. This is the assumption that simple causes must produce simple effects, implying that complex effects must have complex causes. This belief causes us to look at a complex object or system, and wonder where the complexity comes from. Where, for example, did the complexity of life come from, given that it must have originated on a lifeless planet? It seldom occurs to us that complexity might appear of its own accord, but that is what the latest mathematical techniques indicate.
The determinacy of the laws of physics follows from a simple mathematical fact: there is at most one solution to a differential equation with given initial conditions. In Douglas Adams's The Hitchhiker's Guide to the Galaxy the supercomputer Deep Thought embarked on a five million year quest for the answer to the great question of life, the universe and everything, and famously derived the answer, 42. This incident is a parody of a famous statement in which Laplace summed up the mathematical view of determinism:
357
taming THE INFINITE
I II A11 It ANH I ..Ml i I >I|V
'An intellect which at any given moment knew all the forces that animate nature and the mutual positions of the beings that comprise it, if this intellect were vast enough to submit its data to analysis, it could condense into a single formula the movement of the greatest bodies of the universe and that of the lightest atom: for such an intellect nothing could be uncertain, and the future just like the past would be present before its eyes.'
He then brought his readers down to earth with a bump, by adding:
'Human mind offers a feeble sketch of this intelligence in the perfection which it has been able to give to astronomy'.
Ironically, it was in celestial mechanics, that most evidently deterministic part of physics, that Laplacian determinism would meet a sticky end. In 1886 King Oscar II of Sweden (who also ruled Norway) offered a prize for solving the problem of the stability of the solar system. Would our own litde corner of the clockwork universe keep ticking forever, or might a planet crash into the Sun or escape into interstellar space? Remarkably, the physical laws of conservation of energy and momentum do not forbid either eventuality - but could the detailed dynamics of the solar system shed further light?
Poincare was determined to win die prize, and he warmed up by taking a close look at a simpler problem, a system of three celestial bodies. The equations for three bodies are not much worse than those for two, and have much the same general form. But Poincare s three-body warm-up turned out to be surprisingly hard, and he discovered something disturbing. The solutions of those equations were totally different from those of the two-body case. In fact, the solutions were so complicated that they could not be written down
as a mathematical formula. Worse, he could understand.......
the geometry - more precisely, the topology - of tin- sol...........
prove, beyond any shadow of doubt, that the motions rcpii .•.ciiinl by those solutions could sometimes be highly disordered and irregular. 'One is struck,' Poincare wrote, 'with the complexity of tins figure that I am not even attempting to draw. Nothing can give us a better idea of the complexity of the three-body problem.' This complexity is now seen as a classic example of chaos.
His entry won King Oscar If s prize, even though it did not fully solve the problem posed. Some 60 years later, it triggered a revolution in how we view the universe and its relation to mathematics.
In 1926-7 the Dutch engineer Balthazar van der Pol constructed an electronic circuit to simulate a mathematical model of the heart, and discovered diat under certain conditions the resulting oscillation is not periodic, like a normal heartbeat, but irregular. His work was given a solid mathematical basis during the Second World War by John Littlewood and Mary Cartwright, in a study that originated in the electronics of radar. It took more than 40 years for the wider significance of their work to become apparent.
Nonlinear dynamics
In the early 1960s the American mathematician Stephen Smale ushered in the modern era of dynamical systems theory by asking for a complete classification of the typical types of behaviour of electronic circuits. Originally expecting the answer to be combinations of periodic motions, he quickly realized that much more complicated behaviour is possible. In particular he developed Poincare's discovery of complex motion in the restricted three-body problem, simplifying the geometry to yield a system known as 'Smale's horsehoe'. He proved that the horsehoe system, although deterministic, has some random features. Other examples of such phenomena were developed by the American and Russian schools of dynamics, with especially notable contributions by Oleksandr
358
359
TAMING THE INFINITE
CHAOS AND COMPLEXITY
Sharkovskii and Vladimir Arnold, and a general theory began to emerge.The term 'chaos' was introduced by James Yorke andTien-Yien Li in 1975, in a brief paper that simplified one of the results of the Russian school: Sharkovskii's Theorem of 1964, which described a curious pattern in the periodic solutions of a discrete dynamical system - one in which time runs in integer steps instead of being continuous..
Meanwhile, chaotic systems were appearing sporadically in the applied literature - again, largely unappreciated by the wider scientific community. The best known of these was introduced by the meteorologist Edward Lorenz in 1963. Lorenz set out to model'
Poincare's Blunder
June Barrow-Green, delving into the archives of the Mittag-Leffler Institute in Stockholm, recently discovered an embarrassing tale that had previously been kept quiet. The work that had won Poincare the prize contained a serious mistake. Far from discovering chaos, as had been supposed, he had claimed to prove that it did not occur. The original submission proved that all motions in the three-body problem are regular and well-behaved.
After the award of the prize, Poincare belatedly spotted an error, and quickly discovered that it demolished his proof completely. But the prize-winning memoir had already been published as an issue of the Institute's journal. The journal was withdrawn, and Poincare" paid for a complete reprint, which included his discovery of homoclinic tangles and what we now call chaos. This cost him significantly more than the money he had won with his flawed memoir. Almost all of the copies of the incorrect version were successfully reclaimed and destroyed, but one, preserved in the archives of the Institute, slipped through the net.
atmospheric convection, approximating the very complex equations for this phenomenon by much simpler equations with three variables. Solving them numerically on a computer, he discovered that the solution oscillated in an irregular, almost random maimer. He also discovered that if the same equations are solved using initial values of the variables that differ only very slightly from each other, then the differences become amplified until the new solution differs completely from the original one. His description of this phenomenon in subsequent lectures led to the currendy popular term, butterfly effect, in which the flapping of a butterfly's wings leads, a month later, to a hurricane on the far side of the globe.
This weird scenario is a genuine one, but in a rather subtle sense. Suppose you could run the world's weather twice: once with the butterfly and once without. Then you would indeed find major differences, quite possibly including a hurricane on one run but no hurricane on the other. Exactly this effect arises in computer simulations of the equations normally used to predict the weather, and the effect causes big problems in weather forecasting. But it would be a mistake to conclude diat die butterfly caused the hurricane; In the real world, weather is influenced not by one butterfly but by the statistical features of trillions of butterflies and other tiny disturbances. Collectively, these have a definite influence on where and when hurricanes form, and where they subsequendy go.
Using topological methods, Smale, Arnold and their coworkers proved that the bizarre solutions observed by Poincare were the inevitable consequence of strange attractors in the equations. A strange attractor is a complex motion that the system inevitably homes in on. It can be visualized as a shape in the state-space formed by the variables that describe the system. The Lorenz attractor, which describes Lorenz's equations in this manner, looks a bit like the Lone Ranger's mask, but each apparent surface has infinitely many layers.
The structure of attractors explains a curious feature of chaotic systems: they can be predicted in the short term (unlike, say, rolls
360
361
TAMING THE INFINITE
of a die) but not in the long term. Why cannot several short-term predictions be strung together to create a long-term prediction? Because the accuracy with which we can describe a chaotic system deteriorates over time, at an ever-growing rate, so there is a prediction horizon beyond which we cannot penetrate. Nevertheless, the system remains on the same strange attractor - but its path over the attractor changes significantly.
This modifies our view of the butterfly effect. All the butterflies can do is push the weather around on the same strange attractor -so it always looks like perfectly plausible weather. It's just a bit different from what it would have been without all diose butterflies.
1900-98
Mary Cartwright graduated from Oxford University in 1923, one of only five women studying mathematics in the university. After a short period as a teacher, she took a doctorate at Cambridge, nominally under Godfrey Hardy but actually under Edward Titchmarsh because Hardy was in Princeton. Her thesis topic was complex analysis. In 1934 she was appointed assistant lecturer at Cambridge, and in 1936 she was made director of studies at Girton College.
In 1938, in collaboration with John Littlewood, she undertook research for the Department of Scientific and Industrial Research on differential equations related to radar. They discovered that these equations had highly complicated solutions, an early anticipation of the phenomenon of chaos. For this work she became the first woman mathematician to be elected a Fellow of the Royal Society, in 1947. In 1948 she was made Mistress of Girton, and from 1959 to 1968 she was a reader at the University of Cambridge. She received many honours, and was made Dame Commander of the British Empire in 1969.
lillAdS ANII I.DMI'I I "II.
David Ruelle and Floris Takens quickly found a poiniii.il application of strange attractors in physics: the baffling problem of turbulent flow in a fluid. The standard equations for fluid flow, called the Navier—Stokes equations, are partial differential equations, and as such they are deterministic. A common type of fluid flow, laminar flow, is smooth and regular, just what you would expect from a deterministic theory. But another type, turbulent flow, is frothy and irregular, almost random. Previous theories either claimed that turbulence was an extremely complicated combination of patterns that individually were very simple and regular, or that the Navier—Stokes equations broke down in the turbulent regime. But Ruelle and Takens had a third theory. They suggested that turbulence is a physical instance of a strange attractor.
Initially this theory was received with some scepticism, but we now know that it was correct in spirit, even if some of the details were rather questionable. Other successful applications followed, and the word chaos was enlisted as a convenient name for all such behaviour.
Theoretical monsters
A second theme now enters our tale. Between 1870 and 1930 a diverse collection of maverick mathematicians invented a series of bizarre shapes the sole purpose of which was to show up the limitations of classical analysis. During the early development of calculus, mathematicians had assumed that any continuously varying quantity must possess a well-defined rate of change almost everywhere. For example, an object that is moving through space continuously has a well-defined speed, except at relatively few instants when its speed changes abruptly. However, in 1872 Weierstrass showed that this long-standing assumption is wrong. An object can move in a continuous fashion, but in such an irregular manner that - in effect - its speed is changing abruptly at every moment of time. This means that it doesn't actually have a sensible speed at all.
taming THE INFINITE
Other contributions to this strange zoo of anomalies included a curve that fills an entire region of space (one was found by Peano in 1890, another by Hilbert in 1891), a curve that crosses itself at every point (discovered by Waclaw Sierpinski in 1915) and a curve of infinite length that encloses a finite area. This last example of geometric weirdness, invented by Helge von Koch in 1906, is the snowflake curve, and its construction goes like this: Begin with an equilateral triangle, and add triangular promontories to the middle of each side to create a six-pointed star. Then add smaller promontories to the middle of the star's twelve sides, and keep going forever. Because of its sixfold symmetry, the results look like an intricate snowflake. Real snowflakes grow by other rules, but that's a different story.
The mainstream of mathematics promptly denounced these oddities as 'pathological' and a 'gallery of monsters', but as the years went by several embarrassing fiascos emphasized the need for care, and the mavericks' viewpoint gained ground. The logic behind analysis is so subtle that leaping to plausible conclusions is
Stages in the construction of Hilbert's space-filling curve and the Sierpinski gasket
IIIIAIi:, ANIl I.IIMI'I i x11 v
?
The snowflake curve
I 5
dangerous: monsters warn us about what can go wrong. So, by the turn of the century, mathematicians had become comfortable with the newfangled goods in the mavericks' curiosity shop - they kept the theory straight without having any serious impact on applications. Indeed by 1900 Hilbert could refer to the whole area as a paradise without causing ructions.
In the 1960s, against all expectations, the gallery of theoretical monsters was given an unexpected boost in the direction of applied science. Benoit Mandelbrot realized that these monstrous curves are clues to a far-reaching theory of irregularities in nature. He renamed them fractals. Until then, science had been happy to stick to traditional geometric forms like rectangles and spheres, but Mandelbrot insisted that this approach was far too restrictive. The natural world is littered with complex and irregular structures -coastlines, mountains, clouds, trees, glaciers, river systems, ocean waves, craters, cauliflowers - about which traditional geometry remains mute. A new geometry of nature is needed.
Today, scientists have absorbed fractals into their normal ways of thinking, just as their predecessors did at the end of the 19th century with those maverick mathematical monstrosities. The second half of Lewis Fry Richardson's 1926 paper 'Atmospheric
364
365
TAMING THE INFINITE
UlAli:. AND CDMI'I I X 11 Y
diffusion shown on a distance-neighbour graph' bears the tide 'Does the wind have a velocity?'. This is now seen as an entirely reasonable question. Atmospheric flow is turbulent, turbulence is fractal and fractals can behave like Weierstrass's monstrous function — moving continuously but having no well defined speed. Mandelbrot found examples of fractals in many areas both in and outside science - the shape of a tree, the branching pattern of a river, the movements of the stock market.
Chaos everywhere!
The mathematicians' strange attractors, when viewed geometrically, turned out to be fractals, and the two strands of thought became intertwined in what is now popularly known as chaos theory.
Chaos can be found in virtually every area of science. Jack Wisdom and Jacques Laskar have found that the dynamics of the solar system is chaotic. We know all the equations, masses and speeds that are required to predict the future motion forever, but there is a prediction horizon of around ten million years because of dynamical chaos. So if you want to know what side of the Sun Pluto will be in ad 10,000,000 - forget it.These astronomers have also shown that the Moon's tides stabilize the Earth against influences that would otherwise lead to chaotic motion, causing rapid shifts of climate from warm periods to ice ages and back again; so chaos theory demonstrates that without the Moon, the Earth would be a pretty unpleasant place to live.
Chaos arises in nearly all mathematical models of biological populations, and recent experiments (letting beetles breed under controlled conditions) indicate that it arises in real biological populations as well. Ecosystems do not normally setde down to some kind of static balance of nature: instead they wander around on strange attractors, usually looking fairly similar, but always changing. The failure to understand the subtle dynamics of ecosystems is one reason why the world's fisheries are close to disaster,
From chaos, we turn to complexity. Many of the problems lacing today's science are extremely complicated. To manage a coral reef, a forest or a fishery it is necessary to understand a highly complex ecosystem, in which apparently harmless changes can trigger unexpected problems. The real world is so complicated, and can be so difficult to measure, that conventional modelling methods are hard to set up and even harder to verify. In response to these challenges, a growing number of scientists have come to believe that fundamental changes are needed in the way we model our world.
In the early 1980s George Cowan, formerly head of research at Los Alamos, decided that one way forward lay in the newly developed theories of nonlinear dynamics. Here small causes can create huge effects, rigid rides can lead to anarchy and the whole often has capabilities that do not exist, even in rudimentary form, in its components. In general terms, these are exactly the features observed in the real world. But does the similarity run deeper — deep enough to provide genuine understanding?
Cowan conceived the idea of a new research institute devoted to the interdisciplinary applications and development of nonlinear dynamics. He was joined by Murray Gell-Mann, the Nobel-winning particle physicist, and in 1984 they created what was then called the Rio Grande Institute. Today it is the Santa Fe Institute, an international centre for the study of complex systems. Complexity theory has contributed novel mathematical methods and viewpoints, exploiting computers to create digital models of nature. It exploits the power of the computer to analyse those models and deduce tantalizing features of complex systems. And it uses nonlinear dynamics and other areas of mathematics to understand what the computers reveal.
TAMING THE INFINITE
CHAOS ANO I DM........
In one type of new mathematical model, known as a cellular automaton, such things as trees, birds and squirrels are incarnated as tiny coloured squares. They compete with their neighbours in a mathematical computer game. The simplicity is deceptive - these games lie at the cutting edge of modern science.
Cellular automata came to prominence in the 1950s, when John von Neumann was trying to understand life's ability to copy itself. Stanislaw Ulam suggested using a system introduced by the computer pioneer Konrad Zuse in the 1940s. Imagine a universe composed of a large grid of squares, called cells, like a giant chessboard. At any moment, a given square can exist in some state. This chessboard universe is equipped with its own laws of nature, describing how each cell's state must change as time clicks on to the next instant. It is useful to represent that state by colours.Then the rules would be statements like: 'If a cell is red and has two blue cells next to it, it must turn yellow.' Any such system is called a cellular automaton - cellular because of the grid, automaton because it blindly obeys whatever rules are listed.
To model the most fundamental feature of living creatures, von Neumann created a configuration of cells that could replicate - make copies of itself. It had 200,000 cells and employed 29 different colours to carry around a coded description of itself. This description could be copied blindly, and used as a blueprint for building further configurations of the same kind. Von Neumann did not publish his work until 1966, by which time Crick and Watson had discovered the structure of DNA and it had become clear how life really does perform its replication trick. Cellular automata were ignored for another 30 years.
By the 1980s, however, there was a growing interest in systems composed of large numbers of simple parts, which interact to produce a complicated whole. Traditionally, the best way to model a system mathematically is to include as much detail as possible: the closer the model is to the real thing, the better. But tins high-detail approach fails
368
What Nonlinear dynamics did for them_
Until nonlinear dynamics became a major issue in scientific modelling, its role was mainly theoretical. The most profound work was that of Poincare on the three-body problem in celestial mechanics. This predicted the existence of highly complex orbits, but gave little idea of what they looked like. The main point of the work was to demonstrate that simple equations may not have simple solutions - that complexity is not conserved, but can have simpler origins.
Modern computers can calculate complicated orbits in the three-body problem
for very complex systems. Suppose, for instance, that you want to understand the growth of a population of rabbits. You don't need to model the length of the rabbits' fur, how long their ears are or how dieir immune systems work. You need only a few basic facts about each rabbit: how old it is, what sex it is, whether it is pregnant.Then you can focus your computer resources on what really matters.
For this kind of system, cellular automata are very effective. They make it possible to ignore unnecessary detail about the individual components, and instead to focus on how these components interrelate. This turns out to be an excellent way to work out which factors are important, and to uncover general insights into why complex systems do what they do.
369
taming THE INFINITE
A complex system that defies analysis by traditional modelling techniques is the formation of river basins and deltas. Peter Burrough has used cellular automata to explain why these natural features adopt the shapes that they do. The automaton models the interactions between water, land and sediment. The results explain how different rates of soil erosion affect the shapes of rivers, and how rivers carry soil away important questions for river engineering and management. The ideas are also of interest to oil companies, because oil and gas are often found in geological strata that were originally laid down as sediment.
Another beautiful application of cellular automata occurs in biology. Hans Meinhardt has used cellular automata to model the formation of patterns on animals, from seashells to zebras. Key factors are concentrations of chemicals. Interactions are reactions within a given cell and diffusion between neighbouring cells. The two types of interaction combine to give the actual, rules for the next state. The results provide useful insights into the patterns of activation and inhibition that switch pigment-making genes on and off dynamically during the animal's growth.
Stuart Kauffman has applied a variety of complexity-theoretic techniques to delve into another major puzzle in biology: the development of organic form. The growth and development of an organism must involve a great deal of dynamics, and it cannot be just a matter of translating into organic form the information held in DNA. A promising way forward is to formulate development as the dynamics of a complex nonlinear system.
Cellular automata have now come full circle and given us a new perspective on the origins of life. Von Neumann's self-replicating automaton is enormously special, carefully tailored to make copies of one highly complex initial configuration. Is this typical of self-replicating automata, or can we obtain replication without starting from a very special configuration? In 199 3 Hui-Hsien Chou and
370
chads and udmi'i i «11»
James Reggia developed a cellular automaton with 29 states loi which a randomly chosen initial state, or primordial soup, leads to self-replicating structures more than 98 per cent of the time. In this automaton, self-replicating entities are a virtual certainty.
Complex systems support the view that on a lifeless planet with sufficiently complex chemistry, life is likely to arise spontaneously and to organize itself into ever more complex and sophisticated forms. What remains to be understood is what kinds of rule lead to the spontaneous emergence of self-replicating configurations in our own universe - in short, what kind of physical laws make this first crucial step towards life not only possible, but inevitable.
The story of mathematics is long and convoluted. As well as making remarkable breakthroughs, the pioneers of mathematics headed off down blind alleys, sometimes for centuries. But this is the way of the pioneer. If it is obvious where to go next, anyone can do it. And so, over some four millennia, the elaborate, elegant structure that we call mathematics came into being. It arose in fits and starts, with wild bursts of activity followed by periods of stagnation; the centre of activity moved around the globe following the rise and fall of human culture. Sometimes it grew according to the practical needs of that culture; sometimes the subject took off in its own direction, as its practitioners played what seemed to everyone else to be mere intellectual games. And surprisingly often, those games eventually paid off in the real world, by stimulating the development of new techniques, new points of view and new understanding.
Mathematics has not stopped. New applications demand new mathematics, and mathematicians are responding. Biology, especially, poses new challenges to mathematical modelling and understanding. The internal requirements of mathematics continue to stimulate new ideas, new theories. Many important conjectures remain unsolved, but mathematicians are working on them.
371
tam
INK THE INFINITE
chad:, and uimm him
What Nonlinear dynamics does for us
It might seem that chaos has no practical applications, being irregular, unpredictable and highly sensitive to small disturbances. However, because chaos is based on deterministic laws, it turns out to be useful precisely because of these features.
One of the potentially most important applications is chaotic control. Around 1950 the mathematician John von Neumann suggested that the instability of weather might one day be turned to advantage, because it implies that a large desired effect can be generated by a very small disturbance. In 1979 Edward Belbruno realized that this effect could be used in astronautics to move spacecraft through large distances with very little expenditure of fuel. However, the resulting orbits take a long time -two years from the Earth to the Moon, for instance - and NASA lost interest in the idea.
In 1990 Japan launched a small Moon probe, Hagoromo, which separated from a larger probe Hiten that stayed in Earth orbit. But the radio on Hagoromo failed, leaving Hiten with no role to play. Japan wanted to salvage something from the mission, but Hiten had only 10% of the fuel required to get it to the Moon using a conventional orbit. An engineer on the project remembered Belbruno's idea, and asked him to help. Within ten months Hiten was on its way to the Moon and beyond, seeking trapped particles of interstellar dust - with half of its remaining fuel unused. The technique has been used repeatedly since this first success, in particular for the Genesis probe to sample the solar wind, and ESA's SMARTONE mission.
The technique applies on Earth as well as in space. In 1990 Celso . Grebogi, Edward Ott and James Yorke published a general theoretical scheme to exploit the butterfly effect in the control of chaotic systems. The method has been used to synchronize a bank of lasers; to control
heartbeat irregularities, opening up the possibility of an intelligent pacemaker; to control electrical waves in the brain, which might help to suppress epileptic attacks; and to smooth the motion of a turbulent fluid, which could in future make aircraft more fuel-efficient.
Throughout its lengthy history, mathematics has taken its inspiration from these two sources - the real world and the world of human imagination. Which is most important? Neither. What matters is the combination.The historical method makes it plain that mathematics draws its power, and its beauty, from both. The time of the ancient Greeks is often seen as a historical Golden Age, as logic, mathematics, and philosophy were brought to bear on the human condition. But the advances made by the Greeks are just part of an ongoing story. Mathematics has never been so active, it has never been so diverse, and it has never been so vital to our society.
Welcome to the Golden Age of mathematics.