Markets do not usually fail because participants behave irrationally.
They fail because many participants behave intelligently in the same way.
When intelligence converges, resilience disappears and the system becomes vulnerable to its own sophistication.
This essay explores how that convergence forms, why it becomes dangerous, and what it reveals about the structure of modern finance.
In our previous piece on Brian Arthur’s El Farol problem, we saw how prediction reshapes the system being predicted.
Here we extend the idea from expectation to behaviour.
This is the optimisation trap.
Why Optimisation Spreads
Optimisation begins with good intentions.
A portfolio manager reduces noise.
A quant tightens a rule.
A risk team smooths a drawdown.
A consultant promotes a framework that backtests well.
A regulator encourages standards that are measurable and defensible.
None of these actions are unreasonable.
But they rarely remain isolated.
Finance rewards what is explainable, comparable, and benchmark aligned.
Once an optimisation becomes widely accepted, it spreads quickly.
Consultants teach it.
Allocators request it.
Vendors automate it.
Risk committees enforce it.
Regulators reinforce it.
What begins as innovation becomes convention.
What begins as independence becomes convergence.
When the Market Stops Being External
Markets are not neutral backgrounds.
They are ecosystems shaped by the behaviour of those within them.
Every successful strategy alters the conditions it tries to exploit.
Every optimisation influences the structure it attempts to understand.
Once enough participants adapt in similar ways, the environment stops acting like an independent system.
It becomes increasingly defined by the models used to navigate it.
This transition has occurred many times in history.
1987: Dynamic hedging and forced selling
Portfolio insurance models instructed funds to sell as markets fell.
When many funds followed the same rule, their selling reinforced itself and accelerated the crash.
1998: LTCM and relative value convergence
Trades that appeared independent across institutions turned out to be nearly identical.
When spreads moved, everyone required the same exit at the same time.
2018: The volatility ETN shock
Short volatility products linked to the same metric were forced to buy volatility as it rose.
Mechanical flows created a sudden spike.
In each case the issue was not irrationality.
It was uniformity.
Why Shared Intelligence Creates Multiplicative Fragility
It is natural to imagine that ten intelligent strategies create ten units of intelligence.
Adaptive markets do not behave that way.
Correlation multiplies exposure
When many portfolios rely on similar signals, their exposures do not diversify.
They merge into a single economic bet.
This is why so many professional strategies behave as pro-growth trades and fail together when liquidity tightens.
Empirical studies show that cross-strategy correlation rises sharply in stress regimes.
Independence is a calm-weather illusion.
Flows reinforce one another
Shared signals generate shared entry and exit points.
Order flow becomes self-reinforcing.
Price begins to reflect collective behaviour rather than new information.
Errors spread faster than insights
Shared models produce shared failures.
Stop-losses cluster.
Dealers hedge mechanically.
Margin requirements jump.
Deleveraging cascades.
This creates positive feedback loops that resemble physical systems pushed past their thresholds.
Institutions converge because of incentives
Benchmarks, VaR models, consultant templates, regulatory frameworks, and career risk all push large organisations toward the same narrow set of strategies.
Intelligence therefore does not add.
It compounds.
And compounded intelligence becomes fragility.
The Hidden Correlation That Appears at the Worst Time
Most strategies appear diversified in quiet markets.
In reality many depend on the same macro forces.
Value, carry, credit, vol-selling, risk parity, and many factor portfolios thrive when liquidity rises.
They fail when liquidity retreats.
During stable periods their correlation is low.
Under stress it spikes.
Diversification that appeared robust evaporates precisely when it is needed most.
What looked like many ideas was one idea expressed through many instruments.
Why Fragility Gives Way to Renewal
Uniformity does not last forever.
The same reflexivity that destroys edges also creates space for new ones.
Innovative quants, structural specialists, niche traders, and unconventional thinkers introduce diversity back into the system.
Adaptive markets cycle through discovery, imitation, convergence, and reset.
Fragility and renewal are both natural phases of evolution.
Where Simplicity Helps and Where It Fails
It is tempting to imagine simplicity as the cure for optimisation.
But simplicity can converge just as easily.
Trend following often performs well in stress because it adapts to price rather than forecasting price.
Its assumptions are modest and it adjusts automatically to new regimes.
But it is not immune to crowding.
If everyone used identical lookbacks, markets, and sizing rules, it would converge like everything else.
The advantage of a strategy lies not in simplicity or complexity.
It lies in the diversity of its assumptions and the adaptability of its design.
Resilience is a framework, not a single method.
How Convergence Turns Vulnerability Into Crisis
The mechanism of failure is concrete and observable:
- Shared models identify the same signals
- Entry and exit points cluster
- Volatility targeting cuts exposure simultaneously
- Market makers hedge mechanically as liquidity dries
- Margin calls trigger forced deleveraging
- Redemptions concentrate selling pressure
- Depth disappears because everyone requires the same liquidity
Crashes emerge not from mystery but from coordinated behaviour that overwhelms the system.
Testable Implications
A useful framework produces testable predictions.
-
When strategy convergence increases, cross-strategy correlation should rise most quickly during stress.
-
When institutions become more benchmark constrained, recovery periods after shocks should lengthen.
-
When strategy innovation rises, systemic event frequency should fall.
-
When risk models become uniform, drawdowns should cluster across funds.
These predictions separate explanation from metaphor.
They allow the idea to be challenged or confirmed.
The Real Lesson
Markets fail when many intelligent participants optimise in similar ways under similar constraints.
They recover when diversity returns through new ideas and new behaviours.
The solution is not to avoid optimisation.
It is to avoid optimising into sameness.
The greatest edge in an adaptive system is not cleverness but independence.
Not precision but adaptability.
Not forecasting but the ability to remain functional when others converge.
In a world that rewards being smart, the lasting advantage belongs to those who stay different.