THE MARKET IS NOT A MACHINE

Niels and I began this conversation in a crowded bar in Santa Fe.

Not literally. The bar was El Farol, famous locally for its Thursday-night Irish music, and the visit was a thought experiment devised by the economist Brian Arthur.

Imagine one hundred people deciding whether to go. If fewer than sixty attend, it is a pleasant evening. If more than sixty arrive, it is too crowded and the people who stayed home made the better choice.

Everyone can see the attendance history. Everyone is trying to make a sensible forecast.

But if everybody predicts a quiet evening, they all go and the bar becomes crowded. The forecast defeats itself. If everybody expects a crowd, they stay home and the bar is quiet. The forecast defeats itself again.

This sounds like a charming puzzle about a night out. It is actually one of the clearest ways I know to understand a financial market.

The people trying to predict the outcome are helping to create it.

That single idea changes almost everything.

The future is not waiting to be discovered

Arthur gave the simulated patrons different forecasting rules. One expected this week to resemble last week. Another used an average. Another followed a different pattern in the attendance history.

Each person selected whichever rule had worked recently. After Thursday night, the actual attendance became new evidence for deciding which rule to use next time.

The result was fascinating. Attendance moved around the bar’s comfortable capacity, but no forecasting rule remained permanently superior. As one rule attracted followers, their behaviour changed the environment in which the rule operated.

A fairly stable average emerged from strategies that were continually changing underneath it.

This is the first important connection to markets. A persistent statistical pattern does not necessarily reveal a fixed mechanism. The visible regularity can survive while the participants, rules and relationships producing it keep evolving.

The market is not a fixed examination paper with one correct answer. It changes partly because people keep trying to solve it.

Beliefs can manufacture their own evidence

George Soros made the idea of reflexivity famous in financial markets.

People form beliefs about a system. Those beliefs shape their actions. Their actions alter the system, and the changed system becomes the evidence from which the next beliefs are formed.

A bank run is the cleanest example. A bank may begin the week in reasonable condition. A rumour spreads and depositors withdraw money to protect themselves. The withdrawals drain the bank’s liquidity. By the end of the week, the shared fear may have helped create the failure it anticipated.

Markets contain the same loop.

A rising price improves reported performance. It can strengthen collateral, attract capital and encourage further buying. That buying lifts the price again and appears to validate the original belief. On the way down, losses can trigger stops, reduce collateral, produce margin calls and force more selling.

The price is not merely recording what participants believe. It is changing their wealth, choices and capacity to act.

The future is therefore not sitting fully formed, waiting for the cleverest observer to uncover it. Participants help write it step by step.

Why Santa Fe challenged the machine

The Santa Fe Institute was founded in 1984 and brought together extraordinary minds from physics, economics, biology and computer science. Its community included Murray Gell-Mann, Philip Anderson, Kenneth Arrow, John Holland and Brian Arthur.

They were interested in complex adaptive systems: systems made up of interacting participants that learn and change while the system itself is operating.

The traditional equilibrium picture remains useful. Imagine a fish stall. If the price is too high, fish remain unsold and the seller reduces it. If the price is too low, the fish disappear quickly and buyers compete for what remains. The imbalance meets resistance and the market moves towards a balance between supply and demand.

That describes negative feedback. A movement produces forces that push back against it.

Santa Fe widened the question. What happens when participants differ, learn from experience and react to one another? What happens when capital enters and leaves, technologies change, new instruments appear and yesterday’s successful strategy attracts new competitors?

An equilibrium model often begins by asking what the market looks like once its parts have settled into balance. The Santa Fe approach begins with participants holding different plans. They act, observe what everyone else has done, and revise those plans. The market forms through the interaction.

Brian Arthur described the economy less as a machine and more as an ecology. Strategies compete. Some grow stronger, some disappear and new ones arise.

That is much closer to the market a trader actually inhabits.

When movement begins to feed itself

Most people have heard of diminishing returns. Add more workers to the same fixed patch of land and eventually each additional worker contributes less.

Brian Arthur helped restore increasing returns to the economic conversation.

Increasing returns arise when an early advantage changes the environment in a way that makes further success more likely.

VHS and Betamax provide the classic illustration. Betamax was widely regarded as having technical advantages, but as more households bought VHS machines, video stores stocked more VHS tapes. The wider selection made VHS more attractive to the next household, which gave stores another reason to carry it.

Niels had a particularly good reason to remember this. His family acquired a Betamax machine through a connection with Sony in the United States. They owned the supposedly superior technology and then discovered that hardly any films were available for it.

Quality alone did not choose the winner. Adoption helped build an ecosystem around the winner.

The same logic appeared in personal computing. A larger installed base attracted more software developers. More software made the platform more useful, attracting more users and then still more developers.

This is movement that feeds itself.

Reflexivity and increasing returns are related, but they are not identical. Reflexivity concerns beliefs and actions changing the system being understood. Increasing returns concern adoption or success changing the environment so that further success becomes more likely.

They frequently meet in markets.

A rising price changes beliefs. Those beliefs attract capital. The extra demand raises the price and improves the apparent evidence supporting the belief. Each step changes the conditions facing the next participant.

This gives trend following a rational foundation. Some movements can persist because the movement itself recruits responses that extend it.

It does not tell us which movement will persist, or how far it will travel.

That distinction is the whole game.

A response, not a prophecy

An Outlier Hunter accepts that many trading attempts will be small and unremarkable because a few exceptional price movements can drive much of the long-term result.

The job is not to predict which market will produce the exceptional move. It is to participate broadly, keep failed attempts manageable and avoid cutting off the rare winner merely because it has travelled further than expected.

This is why I do not use fixed profit targets.

If a trend is being extended by responses that have not yet occurred, I cannot know its final size when I enter. A trailing exit allows the market to keep developing the movement. I will surrender some open profit when the trend reverses, but that giveback is the price of leaving the upside open.

This came up through an excellent listener question. Why not take some profit when a trade is already well ahead, reinvest the realised gain and produce a smoother return?

That choice can be tested historically, and it may indeed smooth the curve. But it changes the strategy.

Taking profits earlier reduces exposure to the right tail. It exchanges some positive skew for a more even distribution of realised outcomes. The difficulty is that the largest outliers are rare, their ultimate size is unknowable in advance, and a historical sample contains very few examples of the events doing most of the work.

So the apparent benefit can be measured far more confidently in the centre of the distribution than the opportunity being surrendered in its tail.

There is nothing inherently wrong with taking profits. It simply belongs to a different design objective. An Outlier Hunter is built around concentration in the exceptional event, not protection from the discomfort of giving back part of an open gain.

The artificial market that discovered technical traders

The Santa Fe researchers eventually rebuilt the logic of El Farol with money and prices attached.

Their artificial stock market contained investors using different forecasting rules. Forecasts influenced demand. Demand changed the price. The new price then determined which rules appeared successful and which were replaced.

When the agents adapted more actively, the model produced behaviour that looked surprisingly familiar: turbulent periods arrived in clusters, and price-based technical rules survived within the market ecology.

The model did not prove that a particular moving average or breakout rule must remain profitable. Its importance was deeper.

Complex market behaviour did not have to arrive solely as a reaction to news from outside. It could emerge from participants learning and responding inside the market itself. Technical behaviour was not necessarily irrational noise surrounding a fundamental equilibrium. It could be one of the processes through which the market formed.

Trend followers are inside that ecology too. Their entries can add to an existing flow and their exits can contribute to a reversal. We are participants, not observers standing outside the system.

All systems will be gamed

Brian Arthur used the provocative phrase “all systems will be gamed.”

People study rules, incentives and boundaries, then search for an advantage within them.

Goodhart’s law captures part of the problem: when a measure becomes a target, it ceases to be a good measure.

Suppose a company rewards employees for the number of new customer accounts they open. Someone discovers that opening several accounts for the same customer earns several bonuses. The account total rises, but genuine customer growth does not. Once the tactic spreads, the measure no longer represents what management thought it represented.

Markets are filled with targets and boundaries: benchmarks, volatility limits, margin rules, risk models, option strikes and levels that have held for years.

Stability encourages behaviour designed around its continuation. Leverage grows. Hedges shrink. Liquidity appears more dependable than it will be under stress. Positions accumulate around the assumption that the boundary will hold again.

Then a disturbance arrives. Stops trigger, margins rise or lenders withdraw. Each protective response moves the price and activates the next layer of responses.

The boundary can look strongest just before all the behaviour built around it helps produce the break.

This is close to Minsky’s insight that stability can breed instability. The calm period is not empty. It is where the conditions for the transition may be accumulating.

The outlier is therefore not always a larger version of an ordinary fluctuation. It can be the visible result of the relationships inside the market changing.

The roulette ball does not learn

Doyne Farmer and Norman Packard’s roulette experiment gives us the cleanest contrast.

More than a decade after Edward Thorp and Claude Shannon built an earlier wearable roulette computer, Farmer, Packard and their colleagues developed their own system. They measured the movement of the wheel and ball while betting remained open and used a small computer, eventually concealed in a shoe, to estimate a favourable region of the wheel.

They did not need to name the exact pocket. Under sufficiently stable physical conditions, better measurement could improve the odds.

But the roulette ball could not study their success.

It could not copy their method, anticipate their bet or change its behaviour to remove their advantage.

A financial market can.

Market participants learn, imitate, compete and adapt. A profitable relationship may weaken precisely because it has been discovered, funded and crowded.

Farmer carried the discipline of measurement, modelling and testing from roulette into markets. What changed was the object being studied. In a market, adaptation and competition are part of the mechanism itself.

What a backtest can honestly tell us

A backtest tells us what specified rules would have done through a known historical sequence, subject to the quality of the data and the assumptions about execution.

That is enormously useful.

It can expose excessive turnover, unbearable drawdowns, hidden concentration and dependence on very few trades. It can tell us whether the rules express the behaviour we intend.

It cannot certify that the population, incentives, costs and constraints that produced the historical sequence will recur unchanged.

A pattern that survived for forty years is evidence and deserves to be taken seriously. It is not a physical law. Success attracts capital. Capital changes competition and execution. Participants copy rules, anticipate them and probe their vulnerabilities.

El Farol gives us the subtle version of this argument. The average attendance can remain fairly stable while the strategies underneath it keep changing.

More history can improve our description of the past without turning a market relationship into a permanent constant.

The danger begins when the backtest becomes a map of the future. Parameters are tuned around turning points already known. Markets that happened to contain the largest outliers receive more capital. Quiet exposures are removed because they contributed too little in that particular history.

The finished portfolio looks wonderfully efficient because it knows where the historical outliers occurred.

Live trading begins without that knowledge.

What an Outlier Hunter actually does

On an ordinary trading morning, one market produces a valid entry. Another position stops out. A third has made a gain that already looks unreasonable, but it still satisfies the conditions for remaining open.

The programme has three jobs.

Take the valid entry.

Accept the failed attempt.

Keep the winner while the trailing exit says it remains alive.

From all the complexity, the practical response becomes surprisingly simple.

Search broadly because we do not know where the next reinforcing movement will appear.

Keep initial exposures small because most attempts will not become exceptional.

Use open-ended trailing exits because the rare movement that matters needs room to become much larger than expected.

The process responds without becoming discretionary. A trailing stop changes as price changes, while the decision rule remains consistent. Research is where we reconsider the design. Daily execution is where we follow it.

Complexity does not remove discipline. It tells us why discipline should be built around uncertainty rather than confidence in a forecast.

The bit that stayed with me

The Santa Fe view does not say that equilibrium never appears, that forecasting has no value or that history should be ignored.

It says that a market contains people who learn from history, act on forecasts and change the conditions confronting everyone else.

Three ideas follow.

Participants are inside the market, so their beliefs and actions help create the outcomes they are trying to anticipate.

Some movements feed themselves. That gives us a rational explanation for persistent trends and exceptional outliers without telling us where the next one will occur.

History is evidence rather than a promise. The practical response is to build a process that can participate while the future unfolds: search broadly, keep failures manageable and leave room for the move that becomes far larger than anyone expected.

That is why the market is not a machine.

It is an ecology of participants continually responding to a world their own actions are helping to create.

And if we have all concluded that El Farol will be quiet this Thursday, it may be wise to make other arrangements.

My thanks to Niels for another thoroughly enjoyable conversation, and particularly for the personal Betamax story that brought Brian Arthur’s increasing-returns argument to life.

For anyone who wants to explore these ideas more deeply, I have written a series on complex adaptive markets on my website. I also strongly recommend M. Mitchell Waldrop’s Complexity: The Emerging Science at the Edge of Order and Chaos. It is a wonderful account of the Santa Fe Institute, the remarkable people involved and the ideas they were trying to build.