The Physicists Who Stopped Predicting
How complexity science changed the questions we ask about markets, crises and survival
In 2009, the financial system was still absorbing the shock of the previous year. Banks had failed, markets had frozen and governments had been forced to intervene on a scale few had imagined. The crisis also exposed a problem in the way economics described the system itself.
Many financial models treated markets as if they were moving around a stable centre. Shocks might knock prices away from equilibrium, but competition and rational behaviour would eventually draw them back. The events of 2008 suggested something more troubling. The actions taken by participants to protect themselves could deepen the disturbance. A falling price could force a sale. The sale could drive the price lower. That decline could trigger another institution’s risk limit, margin call or funding problem.
The market was responding to itself.
The Institute for New Economic Thinking was founded in 2009 with a $50 million gift from George Soros. Its creation reflected a growing appetite for economic research that took instability seriously. An associated research centre opened at Oxford in 2012, with J. Doyne Farmer among its leading figures.
Farmer had arrived in economics by an unusual route. As a physics graduate student in the 1970s, he helped build a concealed wearable computer designed to estimate where a roulette ball would land. The machine could not predict every spin. It only needed to find a small advantage often enough to matter.
He later helped establish the Prediction Company, a systematic trading firm founded in 1991 and sold to UBS in 2006. By then, Farmer’s attention had widened from finding patterns in markets to understanding the kind of system that produced them.
Roulette is difficult, but the wheel does not study the players. It does not change its behaviour because someone has discovered a profitable method. Markets do.
A market made of people
The Santa Fe Institute had been exploring problems of this kind since the 1980s. It brought together physicists, biologists, computer scientists and economists who were interested in systems whose collective behaviour could not be understood by examining each component in isolation.
W. Brian Arthur was one of the economists drawn into that work. His research on increasing returns showed how an early advantage can reinforce itself. A technology may attract users because other people have already adopted it. Their adoption draws investment, skills and supporting infrastructure, which makes the technology more attractive again. History matters because each step changes the choices that remain.
Arthur illustrated a related problem with a bar in Santa Fe called El Farol. The bar is enjoyable when fewer than sixty people attend and unpleasant when it is crowded. Anyone deciding whether to go must form a view about what everyone else will do.
If most people expect a crowd, they stay home and the bar is quiet. If most expect an empty room, they go and create the crowd they hoped to avoid. The forecast helps produce the outcome, which may then discredit the forecast.
In Arthur’s model, people try different forecasting rules and favour those that have worked recently. Attendance tends to move around the comfortable level, although no rule remains best for everyone. Once enough people adopt a successful rule, their combined behaviour changes the environment in which it succeeded.
That small example captures something familiar in markets. A profitable strategy attracts capital. The additional trading may weaken the opportunity or crowd the exit. A risk measure changes position sizes. Those changed positions affect prices, correlations and liquidity. Participants are not standing outside the market and observing it. Their decisions are part of what the market becomes.
Building the participants into the model
Traditional economic models often simplify a population into a representative agent with consistent preferences and expectations. Agent-based models take another route. They create many participants, give them different rules and constraints, and allow them to interact inside a computer simulation.
The participants may include banks, investors, households or market makers. One may borrow to buy assets. Another may reduce exposure after a loss. A lender may demand more collateral when prices fall. The modeller then watches what their combined behaviour produces.
The rules are simplified, sometimes drastically. An agent-based model is no more a complete market than a weather model is the atmosphere. Its usefulness lies in showing how a proposed mechanism might work through the system.
Consider a leveraged investor. A fall in the price of an asset reduces the investor’s equity. If the lender requires more collateral, the investor may have to sell. That sale adds pressure to the price and weakens the balance sheets of other investors holding the same asset. Some of them must now sell as well.
No participant needs to intend a crash. Each may be following a sensible rule designed to protect an individual portfolio. The cascade emerges from their interaction.
Farmer, Stefan Thurner and John Geanakoplos used a model of leveraged investors and margin calls to study this process. It generated large price movements and clustered periods of turbulent trading. Later work examined how leverage and risk controls based on recent market behaviour could create cycles of apparent safety followed by instability.
These models did not recreate every institution, security and policy failure involved in 2008. The crisis grew from weak mortgage lending, opaque securities, concentrated exposures, short-term funding and failures of oversight. A compact simulation cannot contain that history. What it can do is isolate one mechanism and show why it deserves attention.
A portfolio may appear manageable while every holder can choose when to trade. It becomes a different portfolio when many of them need cash at once.
Risk that lives between positions
The mortgage securities at the centre of the financial crisis had been priced and rated. Banks had calculated exposures and arranged hedges. Those measurements did not fully capture the connections created by common lenders, collateral requirements and dependence on short-term funding.
When one institution sold, its transaction affected the price used to value similar assets elsewhere. Lower prices produced losses on other balance sheets. Those losses led to demands for collateral, tighter lending and further sales. Risk that had appeared to belong to separate portfolios travelled through the links between them.
This helps explain why market returns behave differently from a sequence of independent draws. Benoit Mandelbrot had shown decades earlier that large price movements occur more often than a simple bell-shaped model would suggest. Volatility also clusters. Turbulent periods tend to be followed by further turbulence.
Complexity models offer possible mechanisms for those observations. Leverage can amplify a move. Imitation can align positions. Funding pressure can turn patient investors into urgent sellers. Shifting liquidity can make a modest order move the market much further than it would in normal conditions.
None of this makes statistical measurement pointless. Value at risk, for example, can estimate a loss threshold over a chosen period and confidence level using a stated method. That can be useful. It does not answer whether a counterparty will withdraw funding, whether several firms will attempt the same trade or whether a position can be closed near its quoted price.
Those questions require stress tests, funding analysis and some understanding of the network in which the portfolio sits. Agent-based and network models can assist, although their conclusions still depend on the participants, rules and connections chosen by the modeller.
The gain is not certainty. It is a wider field of view.
An ecology of strategies
Markets contain participants with different jobs, horizons and constraints. A trend follower may buy into a sustained rise. A value investor may regard the same price as expensive. A dealer may take the other side for a few minutes to supply liquidity. An index fund may trade because money has entered or left, regardless of its view of value.
These strategies coexist. Their profitability depends partly on the behaviour of the others.
When a trade performs well, capital and imitators follow. The opportunity may narrow. If too many investors build similar positions, the eventual exit can become dangerous. Capital then moves elsewhere, participants adapt and the market changes again.
For a trend follower, this ecology offers a plausible reason why persistent price movements continue to appear. Information does not reach everyone at the same time or carry the same meaning for every participant. Some investors adjust quickly. Others move only after a risk limit, committee decision, valuation change or funding constraint forces them to act. Price can therefore travel for longer than a single reaction.
That explanation does not prove that trend following has a permanent edge. A systematic programme still has to identify a move after it has begun. It will encounter false starts, reversals, transaction costs and long periods in which useful trends are scarce. Its outcome depends on the markets it trades, the speed of its rules, its position sizes and its ability to continue through losses.
What complexity science contributes is a reason to doubt that markets will settle into one final, efficient arrangement in which every participant uses the same successful method. The participants keep adapting, and their adaptations keep altering the opportunity.
What changes for the risk manager
The earlier advances in this series remain valuable. Marine insurance spread the cost of lost voyages. Actuarial tables made long-term promises more workable. Option-pricing theory clarified how particular exposures could be valued and hedged under specified conditions. Each solved real problems.
Complexity science adds another layer. It asks what happens when the use of a model changes the behaviour of the system being modelled. It asks whether an action that protects one institution can make the wider market less stable. It asks what connects positions that appear unrelated during calm periods.
For a trader or risk manager, the practical response is straightforward. Use the calculation, but identify what it assumes. Examine the funding behind the position and the liquidity needed to leave it. Consider who else may be responding to the same signal. Ask what happens if relationships that usually diversify the portfolio begin moving together.
The model still has a role. It simply does not get the last word.
The final article returns to the problem that has followed this series from Venice to modern markets. We have learned far more about risk, yet every decision still reaches beyond what can be known in advance. The remaining question is how to act when measurement ends and uncertainty remains.
Next: Article 8, Built for What We Cannot Foresee
From the merchants who divided the risk of a voyage to the researchers tracing financial cascades, each generation expanded what could be understood and managed. The final article asks what those advances mean for someone who must commit capital before the outcome is known.
Richard Brennan writes on systematic trading, complex adaptive markets, and the philosophical foundations of trend following at atstradingsolutions.com. His books include The Fractals of Finance, Complex Adaptive Markets, Carved by Impossibility and The Aussie Turtles Trend Following Guide.
Want to explore why structure exists at all?
Carved by Impossibility: What Remains When Everything Else Is Eliminated
The book explores the architecture of constraint, emergence, and reality itself, and what it means for how we understand markets, life, and the universe.
Available now on Amazon in paperback, hardcover, and Kindle.
Want the theoretical foundation for why trend following works?
The Fractals of Finance: Determinism, Adaptation and the Geometry of Markets
The book explores the full architecture of feedback, fat tails, and fractal structure in financial markets, and what it means for how we trade, invest, and understand risk.
Available now on Amazon in paperback, hardcover, and Kindle.
Want a practical field manual for trading trends and capturing outliers?
The Aussie Turtles Trend Following Guide: A Field Manual for Hunting Outliers adapts the timeless principles of the original Turtle traders into a systematic, rules-based approach for modern markets. Co-authored with Adam Havryliv.
Available now on Amazon in paperback, hardcover, and Kindle.