The central conundrum to every trader is the question…..are the markets random or not The reality however is that the market is a random walk much of the time punctuated with periods of non-random directed price behaviour. The market exhibits behaviour characterised by fat tails. What this means in statistical terms is that markets carry more risk than what normal distributions characterised by random Brownian motion imply.

Real world events induce abrupt change to otherwise well-behaved markets creating extreme variation within these otherwise random conditions leading to a significant proportion of overall price variance. In mathematical terms market movements are therefore better characterised by the term Levy Flight, which is a class of random walk, in which step lengths have a probability distribution that is heavy tailed…..anyway enough with the jargon.

Market randomness in this context relates to uncertainty arising from a lack of perfect information. The same randomness exists within the motion of the water molecules of a cloud (a complex system). With access to perfect information about the actions taken by all market participants, then it theoretically would be possible to predict the exact trajectory of price from one given moment in time to another …….however for practical purposes, the reality is that it is simply insurmountable (intractable) to have access to perfect information. This therefore has implications on the legitimacy of the claims associated with the Efficient Market Hypothesis. The best that we can hope for in complex systems is to predict the overall direction of price by looking at the collective action of all its participants (over a large sample set) to gauge ‘gross statistical measures’ of market movements.

The reality appears to be this……. that market participants (humans and their associated creations [algorithms]) make ‘subjective’ decisions based on partial information that is available to them, combined with a swathe of other reasons (rational or irrational), that are specific to the participant. The participant interacts with the market (the system) at only two times…….the entry and the exit. At these two points of time, the reason for this interaction are effectively transferred into the price of the instrument. We can therefore say that the market now contains all the information pertinent to that particular participant’s behaviour. At each interaction in time (where a buyer transacts with a seller) this information is progressively transferred to the market.

With this interpretation we can therefore say that the market, at all points in time, fully reflect all the information available to the market. As a result those that agree with this statement would necessarily conclude that the ability to predict future price is futile as there is no further available information not already accessed by the market. This is a very powerful argument for those that believe in the Efficient Market Hypothesis……however it is not the full picture.

This interpretation fails to include the lurking non-random feature of a market referred to as auto correlation which results in non-random directional price moves (or Levy Flight).

Auto-correlation (also known as serial correlation) is the correlation of a signal at a particular point in time with a previous signal in a past time. This feature is what gives rise to long term market direction attributed to underlying fundamental/economic push factors which reveal themselves only over a large data sample set, or in periods of crisis, where the plethora of reasons for why a participant interacts with the market is reduced to simple fear or greed decisions which drive coordinated collective human behaviour.

In the short term timeframe (the period of time where traders tend to focus), the nature of the market is highly random. Given that the ‘tick’ is the fundamental unit upon which price action is constructed, the information gleaned from short term patterns (eg. 1 min to 5 mins) cannot with any statistical certainty (low sample size) have any predictive power as their contribution to total price action over an extended timeframe may or may not have any material impact on the longer term trajectory of price.

The market is an open nested system which has a lower limit (or asymmetrical boundary condition). What this means is that while the market has an indefinite open-ended upper bound, it has a finite closed lower bound, being the tick or the interaction between two participants (a buyer and a seller).

This ensures that what is effectively a random price fluctuation in the shortest timescale becomes progressively more non-random towards the upper limits due to repeatable participant behaviour that progressively starts dominating the noise of the market. That is why monthly charts exhibit a market profile that is usually symptomatic of the underlying driving fundamentals that drive overall ‘gross price behaviour’ , whereas on the shortest timescales, the market profile is simply a consequence of the placement and timing of trading activity which could be for a host of factors.

It is the same asymmetrical boundary condition that is thought to be applied to the structure of our universe, in that what were thought to be random quantum fluctuations at the smallest scales at the Big Bang, according to the Standard Cosmological Model, are now believed to be the formative seeds under universal expansion that now represent the grand structure of the cosmos. This is not a mystical statement. It is just a standard thermodynamic feature of a particular class of complex open system.

At any particular moment in time over a narrow sample, the trade decisions made may have no context to the overall direction that price may take based on market consensus, which is a broader longer term measure of overall price direction. The ability therefore to use short term price patterns as a predictive measure of future price movement is far more unlikely to possess the requisite statistical validity required to deliver a trading edge over the long term.

For example, have a look at the following price chart of a sample instrument and predict the future direction of the market.

Unfortunately if you had predicted that the market is bullish and is likely to continue, then if I told you that this was a totally random price series at the tick by tick level, what would you think now? It is a totally random price series by the way……..and a random price series by definition has absolutely no predictive power.

However as we increase the sample size of ticks and extend the timeframe upwards to the weekly and monthly timeframe, what previously was a random price series may ultimately manifest into a component of a non-random (auto-correlated) time series. Crazy huh? You may be familiar with the expression….”order from chaos”.

Visually, you will not be able to discriminate between a random price series and an auto-correlated one. However we have statistical tools such as auto-correlation that can distinguish the two within confidence interval limits. Remember in probability land…..nothing is certain….however statistical tools allow us to assess what is more likely to be random or non-random price behaviour.

Now have a look at the chart below which applies a slight consistent bias of 2% per period to the random data above. Here is how it now looks.

Once again we would interpret this visually to have a strong directional bias and conclude that the market was very bullish. However the degree of non-randomness in this price series is minimal. In other words the market profile in this particular example is considerably random with a slight upside non-directional bias. It is this very small non-random drift (positive or negative) where alpha lurks or the feature we use as traders to gain ‘the statistical edge’ in our systems.

This is what ultimately caps our aspirations to make obscene sums from trading…… The reality is that not much alpha (a real exploitable opportunity) actually exists in a modern mature market. As a result these mature markets with less alpha are more tailored for traders that employ mean reversion techniques, which is a convergent strategy, that assumes an extended price move from the longer term mean is an outlier that will ultimately revert back to the mean of price action as the sample size increases.

On the contrary however emerging markets appear to display persistent alpha (where strong non-random price directionality rears it’s head frequently). These markets offer significant opportunities for trend traders, which is a divergent strategy, that assumes that a price move away from the mean of longer term price action will continue on under momentum as persistent market behaviour begins to dominate market noise over an extended sample set.

Emerging markets such as relatively new indices or instruments typically commence their life with relatively lower random ‘noise’ than more mature markets. This may in part be attributed to the nature of the participants at the early stages of a markets development who are mostly the institutional players of a particular size or scale.

Arbitrage that exists in an emerging market takes some time to disappear. The composition of participants and the nature of participation appears to have a strong role to play in the way price action unfolds.

With more mature complex markets however, the nature, scale and composition of participants serves to capitalise on any arbitrage present and quickly exploit these latent opportunities. In this sense, the more mature markets tend to be more efficient leading to intense complexity in price action and a far greater extent of ‘noise’.

We must always remember that in this zero sum game (or negative sum game with frictional costs), there is always the counter party who is taking an opposite view on the way they think price will play out.

In complex markets, arbitrage opportunities are extremely quickly exploited leaving a feeding frenzy in it’s wake with a lot of wounded participants. The patterns of activity left behind from this feeding frenzy are therefore exceedingly complex and their predictive power is harder to crack. …..anyway I am getting off track.

Now compare the random and slightly non-random graphs now brought together to see what in this example a touch of positive directional bias does to an otherwise random price series.

I hope this drums home the point that without the use of statistical tools to interrogate the market, using visual clues alone to trade the market is as effective as reading tea leaves.

The ability to predict future price action with more reliability tend to emerge as the sample size is increased (eg. more time or more data). With an increased data set, we can become more and more confident that the representative directionality in price action is not just a random ephemeral feature and more a pattern of durability and persistence associated with repeated or collective participant behaviour.

This repeated or collective behaviour leads to an auto-correlated time series which is what the market statistician (or quant) is seeking to discriminate. The impact on price associated with coordinated or repeated price behaviour reveals itself as a bias in overall price direction (a gross movement)….a small non-random drift embedded in otherwise random price patterns leads to a gross statistical feature. The same can be said for clouds. The individual motions of atoms in a cloud are random in nature but the collective actions of all the atoms lead to the gross statistical features of the cloud itself such as it’s overall motion and size.

As traders, when you enter or exit the market, you will never know with certainty whether the trade result could be attributed to random or non-random price movement.

Have a laugh the next time you watch the news and the market pundits confidently state with authority that BHP enjoyed a 5% uplift due to recent merger talks………the reality is that it may simply be attributed to a cat running over the keyboard of an institutional trader from a large bank……you will never know the actual reason for the price movement be it rational, irrational, random, or non-random. The market and the information contained therein is just too complex to interpret with fidelity.

The key to finding an edge in your trading endeavors is to engage in a consistent behaviour over an extended data set where, as more trades are undertaken, the cumulative trade results become more statistically valid.

At the end of the day, your performance results from trading/investing are dominated by simple random chance, but the fine line we are seeking as traders/investors is whether our systems have a slight positive edge or not. The small non-random bias in the market from time to time is what delivers that edge and more than compensates us for the frictional costs of trading to deliver positive expectancy. Once we have a small positive expectancy, that really is the most difficult aspect of trading as we can then leverage our returns using a range of Money Management (MM) techniques (eg. position sizing)…..to aim for improved returns within managed risk constraints.

The best measure to use to assess whether you have positive expectancy (or a definitive edge) is to use a large sample of trades that span across as wide a range of market conditions as possible…….or as Nick Radge (a highly regarded portfolio manager here on the Sunshine Coast and veteran trader) states….”don’t focus on a single trade……focus on the next thousand trades”.

To be able to stay in the game for the next thousand trades in a market riddled with uncertainty requires that risk management must be your over-riding objective in order to protect your capital. So the secret sauce to successful trading lies in the application of consistent repetitive behaviour that has been rigorously tested across a broad range of market conditions….and that has a statistical positive edge……in tandem with strict risk management that allows you to stay alive long enough in an uncertain market to reap the bounty of limited alpha when it is available.

While theories such as ‘The Efficient Market Hypothesis’ (EMH) are powerful arguments for random markets they are not truly reflective of the underlying reality. These theories employ simplifying assumptions that for complex systems, fail to recognise the propensity for power laws to apply.

A power law is a relationship found in complex systems that exist between two quantities (typically a relational dependency) whereby a relative change in one quantity creates a proportional change in another quantity, and is independent of the initial sizes of these quantities (eg. varies as the power of another). Markets are incredibly complex structures and a small disturbance at one time and place can, through these nested interdependent relationships, create a cumulative large domino effect. This is what emerges from a complex system when the fat tails of Levy Drift rear their ugly head…..in a similar manner to the butterfly effect that we may see occasionally which disrupts natural complex systems.

Some of the key things to remember with this market is that all participants do not have equal position sizes or agendas. If they did, then random walk would rule. Market behaviour associated with human beings and decision making (eg. price flexibility) plus dis-proportionate position sizes and repeated behaviour leads to bias or directional price movement.

For example, in the real world let’s say an institution has the task of unloading it’s entire holding of a single instrument for hedging purposes. It obviously cannot unload it’s entire holding in a single trade and needs to progressively reduce it’s exposure over time. This committed and continued emphasis (sustained and coordinated action) against the swarm of traders taking counter positions each with a different agenda and trading timeframe amplifies price action in a specific directional movement until the process of unloading subsides. This is what creates directional price movement. This process continued at regular intervals creates ‘real’ trends whether they be sideways or up and down.

When attempting to interpret the behaviour of a complex system such as a financial market, it is useful to turn to other complex systems to identify potentially common characteristics. For example let’s assume we are proponents of EMH and conduct experiments of how a handful of sand grains (participants) thrown into a pond (the market) interact with this medium. We will note that due to the simplifying assumption applied in this modelling, namely that each participant’s market interaction (sand grain) has equal weighting, the impact of the interaction with the pond will be to create a wave disturbance that emanates from each sand grain and quickly dissipates by destructive interference. The resulting wave disturbance is ephemeral (short lived) in nature just as an efficient market under normal trading produces.

Now let’s look at the market reality, by removing this simplifying assumption. Let’s assume we have a handful of pebbles of different sizes including sand grains. This is more representative of a modern market and typifies the complex mix of institutions, investors and traders etc. and their disproportionate impact with the market. Now throw this collection into the same still pond. What we will observe is that the residual disturbance is a composite of constructive and destructive interference and is more enduring in nature than the prior example. This latter example is far more representative of a ‘real market setting’ ……but look at what arises from this market context when we relax simplifying assumptions, we get the emergence of a non-random market feature that provides the foundations upon which a trader, as opposed to a gambler, speculates.

It is this feature (which is missing in EMH) of the real market that actually is responsible for manifesting non-random market behaviour and the reason we get so many fat tails that cannot be faithfully described by a normal distribution. I hear you Nassim Taleb.

Without perfect information on the mechanics of all the constituents of this market, we simply cannot predict the exact nature of the any predictive outcome, but using the Law of Large Numbers we certainly can predict (using probabilities) the average outcome of the entire system in very broad ways such as that price will rise, track sideways or fall. That’s about all the reliable information we can extract from this market but this is enough to give the required edge to tip the results in your favour.

For example, if we heat a hot air balloon, we cannot determine the exact mechanics of all the gas molecules but we can determine with confidence that the average result of increased molecular activity will cause the walls of the balloon to expand with a little bit more force and we can describe this movement by the gas laws.

If you can accept this system complexity, then the financial markets are not so hard to understand or interpret, and you do not need to step into scientific ‘woo’ land to attempt to understand it…..but a bit of thermodynamics certainly helps. Order from disorder is a central pillar to the laws of classical physics and chemistry. It’s how any large classical and complex system works such as our weather, steam engines and our financial markets.

Just remember…….in a population of a million gamblers, where everybody risks $1 on a bet of a non-biased coin toss where the winners then play each other for round after round to a single final outcome …… randomness ensures at the end of the game that there will always be a winner who pockets $1 million dollars…..and yes….a large proportion of the general public will think they have special powers.

Don’t be fooled by Randomness.

Trade well and prosper

The ATS mob

Want to know more? Then join us at ATS Trading Solutions. We offer unique services for the retail trader wishing to learn more about the techniques applied by the systematic diversified trend followers to generate wealth building sustainable returns. Membership includes access to all Programs and Trading Tools available in our shop, systematic development and portfolio construction education and practice and participation in our Members Forum. 

For a short time only we are offering free access to the exclusive areas of our website reserved for paying members with 100 free vouchers on offer for the first month, so you can have a good look around and get a feel for what we offer. Just go to the membership page of our website and choose the Premium Membership Plan and proceed to checkout. On the Membership Checkout page just click on the discount code and apply the code VIP100 to receive your first month free. So long as you cancel your membership in your first month then you won’t be charged a penny. If you don’t cancel in the first month it will automatically go to $9.99 AUD per month following your first free month.

Looking forward to seeing you over there. So come and join us and participate in our forum with other like minded systematic diversified trend following folk.

Join us