The Physicists Who Stopped Predicting
How complexity science changed the questions we ask about markets, crises and survival
In 2009, as the financial system was still absorbing the shock of the previous year, a question was gaining urgency among economists and scientists. What happens when the people inside a model change their behaviour because of what they see around them?
The Institute for New Economic Thinking was founded that year with a $50 million gift from George Soros. Its creation reflected a growing appetite for economic research that could explain instability as something a financial system might produce itself, rather than something imposed from outside. At Oxford, a related research centre would open in 2012. Among its leading figures was J. Doyne Farmer, a physicist whose route into economics had passed through a roulette wheel.
As a graduate student in the 1970s, Farmer helped build a concealed wearable computer that estimated where a roulette ball would land. The point was not that every spin could be known in advance. A small physical advantage, applied repeatedly, could be useful even in a game designed to look random. He later helped found the Prediction Company, a systematic trading firm established in 1991 and sold to UBS in 2006.
Farmer’s career is a reminder that the physicists in this story did not abandon prediction. They asked where prediction was possible, what assumptions it required, and how those assumptions might fail once people reacted to one another. The question becomes harder when the ball is replaced by a market full of participants who learn.
A market made of people
The Santa Fe Institute, founded in 1984, became an important meeting place for scientists studying complex systems. Economists joined the conversation, especially after a 1987 workshop brought researchers from economics and the physical sciences together. Among them was W. Brian Arthur, whose work on increasing returns showed how an early lead can reinforce itself. A technology may become dominant partly because people have already adopted it, rather than because it was destined to be the best choice.
Arthur offered a smaller, memorable example. Imagine a bar that is enjoyable only when fewer than sixty people turn up. Each potential visitor must guess what the others will do before deciding whether to go. If everyone expects a crowd, they stay away. If everyone expects an empty room, they go. Their forecasts affect the outcome they are trying to forecast.
In Arthur’s El Farol model, people use different forecasting rules and switch among them as experience accumulates. Attendance can settle into a pattern around the comfortable level, even though no single forecasting rule solves the problem for everyone. The lesson is about adaptation. A belief that worked last week may change this week’s behaviour and lose its usefulness.
Financial markets are considerably harder than the bar. Investors have different objectives, funding needs and time horizons. Some must sell when their losses grow. Others may be able to buy, unless the same shock has constrained their balance sheets. Their decisions alter prices, and those prices alter the decisions that follow.
A model that treats the market as the sum of separate, fixed responses can miss what happens between participants. For that reason, researchers developed agent-based models: computer simulations that give different participants rules for acting, then observe what their interactions produce. The rules are simplifications. The value lies in asking whether a mechanism can create a pattern that would otherwise look surprising.
How a loss becomes a cascade
Consider an investor who buys an asset with borrowed money. A fall in its price reduces the investor’s equity. If a lender demands more collateral or the investor has a leverage limit, the investor may have to sell. That sale puts further pressure on the price and can force other investors to sell. Each is responding to a local constraint; together they can produce a much larger movement.
Farmer, Stefan Thurner and John Geanakoplos used a simple model of leveraged investors and margin calls to show how this mechanism can generate unusually large price changes and periods of turbulent trading. Subsequent models have explored how risk rules based on recent prices can contribute to leverage cycles. These are explanations of possible mechanisms, not detailed reconstructions of the 2008 crisis or reliable clocks for the next one.
That distinction matters. The crisis involved weak mortgage lending, securities whose risks were hard to assess, concentrated exposures, short-term funding and failures of oversight. No single simulation contains that entire history. A model that captures forced sales can nonetheless reveal a relationship hidden by a portfolio calculation: my risk depends in part on whether everyone else needs liquidity at the same time.
This is where complexity science meets the history told in the preceding articles. The mortgage securities had been priced and rated. Institutions had measured exposures. Yet their shared dependence on funding and collateral meant that a loss in one part of the system could change the conditions facing another. The map described positions. It did not fully describe how those positions would behave together under pressure.
What the tail can tell us
Benoit Mandelbrot had long warned that large market moves occurred more often than a simple bell-shaped model would suggest. Market returns also show periods in which volatility clusters: a turbulent day is more likely to be followed by further turbulence than a model of independent, identical draws would imply.
Agent-based models offer several ways such patterns might arise. Leverage, imitation, shifting liquidity and feedback between prices and risk limits can each matter. They do not establish a single cause for every crash. Nor do they make all statistical risk measures useless. A measure can answer a well-defined question about a portfolio under stated assumptions and still leave the holder exposed to a crisis that changes those assumptions.
Value at risk, for example, estimates a loss threshold over a chosen horizon and confidence level using a specified method. It does not, by itself, tell a firm whether counterparties will withdraw funding, whether other firms will make the same trade, or whether a market will remain deep enough to exit a position. Those questions call for stress tests, funding analysis and an examination of connections across the system. Network and agent-based models can help examine them, though their results depend on the behaviour and links the modeller has chosen to represent.
There is no view from outside the system that makes uncertainty disappear. The practical gain is to notice which risks a particular tool sees, and which ones it leaves to judgement.
An ecology of strategies
Markets also contain traders who interpret the same movement differently. A trend follower may buy after a sustained rise. A value investor may see the higher price as a reason to sell. A dealer may take the other side temporarily to provide liquidity. Their strategies interact, and the profitability of each depends partly on what the others do.
An attractive trade draws attention and capital. Its returns may weaken as more people pursue it, or its crowded exit may become dangerous. Yet it does not follow that every useful strategy must disappear. Investors face different constraints, respond to information at different speeds and pursue different ends. Those differences can allow persistent price moves to develop.
For a trend follower, the implication is promising but conditional. A systematic programme can participate in a sustained move without knowing its final destination. Across many markets, that gives it opportunities to capture some large moves when they occur. It also brings false starts, reversals, trading costs and long periods when trends are scarce. Diversification and position sizing matter because the strategy cannot choose the moment when an outlier arrives.
Complexity science does not prove a permanent trend-following edge. It gives us reasons to expect markets to keep changing and to take seriously the possibility that adaptive behaviour can generate large, persistent moves. Whether a particular programme benefits depends on its rules, costs and ability to stay in the game through losses.
A different question about risk
The older approaches in this series were not simply steps on a road to failure. Marine insurance spread the losses of dangerous voyages. Actuarial tables made long-term promises possible. Option-pricing theory clarified how particular risks could be hedged under particular conditions. Each achievement remains useful where its conditions hold.
The trouble begins when success inside those conditions is mistaken for a complete description of the world. A model can be precise about a small part of a system while the participants, their funding and their incentives are changing around it. Complexity researchers have made that interaction easier to study, though their own models have boundaries too.
A trader or risk manager does not have to choose between calculation and humility. Both are necessary. Measure what can be measured. Ask what happens if others react to the same signal, if liquidity vanishes, if a position must be closed at the wrong moment. Then build enough room to survive answers the model did not supply.
The next article brings the series back to its oldest question. We have learned much more about risk than the merchants of Venice knew. What, after all that learning, is the honest way to live with what we cannot foresee?
Next: Article 8, The Only Honest Answer
From the merchants of Venice to the researchers studying financial networks, each generation found better ways to understand risk. Each also had to confront what its methods could not see. The final article asks what those five centuries of progress mean for someone who must make decisions without knowing what comes next.
Richard Brennan writes on systematic trading, complex adaptive markets, and the philosophical foundations of trend following at atstradingsolutions.com. His books include The Fractals of Finance, Complex Adaptive Markets, Carved by Impossibility and The Aussie Turtles Trend Following Guide.
Want to explore why structure exists at all?
Carved by Impossibility: What Remains When Everything Else Is Eliminated
The book explores the architecture of constraint, emergence, and reality itself, and what it means for how we understand markets, life, and the universe.
Available now on Amazon in paperback, hardcover, and Kindle.
Want the theoretical foundation for why trend following works?
The Fractals of Finance: Determinism, Adaptation and the Geometry of Markets
The book explores the full architecture of feedback, fat tails, and fractal structure in financial markets, and what it means for how we trade, invest, and understand risk.
Available now on Amazon in paperback, hardcover, and Kindle.
Want a practical field manual for trading trends and capturing outliers?
The Aussie Turtles Trend Following Guide: A Field Manual for Hunting Outliers adapts the timeless principles of the original Turtle traders into a systematic, rules-based approach for modern markets. Co-authored with Adam Havryliv.
Available now on Amazon in paperback, hardcover, and Kindle.