Trend Following Primer Series – Revealing Non-Randomness through the Distribution of Market Returns – Part 4
Primer Series Contents
- An Introduction- Part 1
- Care Less about Trend Form and More about the Bias within it- Part 2
- Divergence, Convergence and Noise – Part 3
- Revealing Non-Randomness through the Market Distribution of Returns – Part 4
- Characteristics of Complex Adaptive Markets – Part 5
- The Search for Sustainable Trading Models – Part 6
- The Need for an Enduring Edge – Part 7
- Compounding, Path Dependence and Positive Skew – Part 8
- A Risk Adjusted Approach to Maximise Geometric Returns – Part 9
- Diversification is Never Enough…for Trend Followers – Part 10
- Correlation Between Return Streams – Where all the Wiggling Matters – Part 11
- The Pain Arbitrage of Trend Following – Part 12
- Building a Diversified, Systematic, Trend Following Model – Part 13
- A Systematic Workflow Process Applied to Data Mining – Part 14
- Put Your Helmets On, It’s Time to Go Mining – Part 15
- The Robustness Phase – T’is But a Scratch – Part 16
- There is no Permanence, Only Change – Part 17
- Compiling a Sub Portfolio: A First Glimpse of our Creation – Part 18
- The Court Verdict: A Lesson In Hubris – Part 19
- Conclusion: All Things Come to an End, Even Trends – Part 20
Revealing Non-Randomness through the Distribution of Market Returns
For any liquid market we can plot a distribution of returns over a large time interval which through its expansive coverage incorporates many different market regimes. What we find in this plot is that the nature of the market distribution is typically not normally distributed. What this means is that the serial correlation present in the time series, provides a non-random bias to the time series leading to the conclusion that markets are not purely efficient or purely random in nature.
If markets were 100% efficient, then this implies by definition that speculative practices are a fools game. Having no non-random bias in the price series implies for example that price at t=10 has no relationship to price at say t=1. In other words, price at each time interval is deemed to be independent in the history of price action.
So, if we interpret that each price interval in a time series is truly independent, then when plotting a random distribution of price over large data samples, we should find that the histogram of market returns plots in accordance with a normal distribution. This is a characteristic distribution that signifies that there is no edge available in this market and that participants are effectively gamblers at the casino.
This does not mean that there will not be winners and losers in this game just as we find at the casino….but what it does mean is that the outcome for a participant is solely dictated by luck or random chance. You may be lucky and have large strings of wins just as you may be unlucky and have large strings of losses….or you may breakeven.
Under a normal distribution we can describe the plot with only 2 values. The mean and the standard deviation. This assumes a stationary market where conditions can be described perfectly by these two values. Having these two values, we can then plot a histogram that adopts a bell-shaped curve symmetrically distributed about the mean. In quantitative circles we refer to this normal distribution plot as the Gaussian envelope.
Now clearly, as speculators, we want to believe that a Normal Distribution does not faithfully describe the market state over a large data sample and that we can exploit some ‘real’ underlying bias in the data. We want to trade an uneven playing field where we can exploit the edge we find in that arena and turn a game of luck into one of skill which, under the Law of Large numbers, has enduring permanence.
Fortunately, when we do this exercise and plot the market distribution of returns of large samples of real (as opposed to theoretical) data, we find that it is not quite a level playing field. We can run this exercise over any large data sample of market data for liquid markets and we inevitably find that these markets are ‘not quite efficient’.
Chart 11: Histogram of Daily Returns across a long time interval for Soybeans, Crude Oil, Spot Gold, DAX, S&P500 and EURUSD
For example, Chart 11 above demonstrates how real market data when using long term time horizons clearly have kurtosis where real market data strays outside the Gaussian envelope (normal distribution) into the peak and tails of the distribution. These areas in the plot that lie outside the normal distribution…. are the areas from a quantitative viewpoint where a real exploitable bias in the data exists.
In other words, real market data exhibits a plot that represents some other type of distribution such as a Cauchy distribution. Debating the actual form of distribution is another fierce academic exercise in quant world which we actually can avoid as we are hopefully traders as opposed to academics, but hope is offered in that real market data is clearly not purely random (or efficient).
For any speculator seeking to exploit an edge, all that matters is that the distribution is not randomly distributed. Where the distribution departs for a normal distribution is where this edge resides.
What we find when you plot the market distribution of returns over a very large data sample is that the plot of the distribution does not fit snugly within the Gaussian envelope. In fact there are two clear zones where an edge resides in the distribution. Namely the peak of the distribution and the tails of the distribution. When we examine this entire distribution closely we find that it is a composite of different distributions reflective of a complex systems emergent and adaptive nature.
A more realistic interpretation of the distribution is that it has multiple means (averages) and multiple standard deviations that reflect different market regimes. The peak of the distribution can broadly extend across a zone and may not be ‘sharp peaked’ having no centralised defined locus and the tails of the distribution can be fat or thin signifying a range of different standard deviations associated with multiple means.
These two discreet zones lying outside the Gaussian envelope reflect the zones that define the edge held by convergent methods and the edge held in divergent methods.
Chart 12: Histogram Plot of the Monthly Market returns of S&P500 over a 70 year data sample
Now Chart 12 displays the distribution of monthly returns for the S&P500 over a 70 year timeframe. Visually you can see that the shape of the distribution can significantly depart from a Normal Distribution. There is variation in this departure for every liquid market demonstrating that different markets do have slightly different ‘distribution signatures’ but the material differences between liquid markets are rarely material. This allows us to generalize and conclude that markets while ‘mostly efficient’ are not perfectly efficient.
Now the zones that lie outside the normal distribution define the zones where an edge resides and conveniently provide a basis to categorise whether you are a convergent or a divergent trader. As discussed previously both methods seek to exploit serial correlation in a price series, but at different zones within the distribution of market returns.
Convergent methods seek the edge associated with the bias in the daily returns at the peak of the distribution, and divergent methods seek the edge associated with the bias in the data towards the left and right tails of the distribution.
Convergent methods exploit the zones around a markets central tendency (equilibrium) or the mean of a historical price distribution. The frequency of this tendency towards equilibrium is large represented by the high values of the probability density function (refer to Chart 12), yet the extent of the directional movement exhibited by the daily returns is compressed within a tight zone around the peak. This leads to the characteristic high Pwin% and low Reward to Risk (R:R) of convergent approaches.
Divergent approaches exploit the zones of the tails of the distribution which are rare occurrences observed by a very low probability density function (refer to Chart 12) but have a very wide range of possible values of any arbitrary extent. This leads to the characteristic low Pwin% but high R:R of these approaches.
We also need to note that the entire distribution can shift from left to right and is not necessarily placed symmetrically around “0” which indicates a degree of skew in the overall data. For example, the long bias in equities associated with the method of index construction shifts the distribution to the right causing a skewed distribution leading to longer left tails than right tails. For markets such as currency pairs that have no preferred bias, the distribution is frequently more symmetrically distributed with equivalent left and right tails.
Having an understanding of the market distribution of returns and its variation for different liquid markets is very helpful for the speculator.
Firstly, it gives us confidence that markets while generally efficient offer opportunities for exploitation and secondly it tells us that the available edge in a liquid market is only slight. By far the majority of price action cannot be discerned as having any exploitable bias. This should therefore make us ‘wise risk managers’ as for much of our trading activities, we are going to be ‘fooled by randomness’.
Finally, it tells us what distinguishes divergent methods from convergent methods and that to participate in either form of opportunity requires totally different trading behaviours.
Convergent methods harvest the bias that concentrates around the peak of the distribution of returns whereas divergent methods harvest the bias the concentrates in the tails of the distribution.
Therefore, for convergent methods, intense effort needs to be placed on identifying the mean (or equilibrium point) around which this bias persists and requires a predictive mindset to exploit it. Having an understanding of what constitutes the equilibrium allows you take advantage of that opportunity with as many trades as possible for as long as it lasts.
With repetitive cycles comes a mindset that starts to lock in that predictability. Many convergent traders start compromising on risk management by progressively pushing there stops further out or eliminating stops altogether in preference to performance exits. While there is no question that stops do actively introduce drawdowns into system performance, the potential for large losses creates a negatively skewed obstacle for convergent traders when conditions cease. Under negative skew, the average profits are many but much smaller than the few large losses….but this only applies when conditions are favourable. When conditions become unfavourable, the large losses become a large consecutive string of large losses and the convergent strategy can plummet towards total risk of ruin.
Chart 13: RIP Long Term Capital Management
This is the real obstacle for convergent trading. Convergent techniques tend to warehouse risk (more on this term later in the series). Unfortunately, you do not find this feature stressed enough in the risk metrics adopted by industry….yet trend followers are fully aware of this little nugget of wisdom and is a major reason for why we avoid these techniques. The message is this….”stay away from strategies with negative skew”. It is akin to not having a release valve on a pressure cooker. They are risk time bombs waiting to explode.
Successful convergent techniques also typically deploy profit targets centered around this central tendency designed to exploit the short-term nature of the convergent move. The high frequency of returns based around this central tendency tells convergent traders that they will be right most of the time. This can lead to an associated bias that reinforces predictive behaviour such as a willingness to apply more leverage to the strategy using enhanced position sizing….however the associated hubris ‘of being right much of the time’ can lead to extreme losses for the confident predictive trader when they are wrong and the equilibrium dissolves as markets transition and start to diverge.
Divergent methods that harvest the bias in the tails must drop any predictive notion as these events are unpredictable in nature and very infrequent, but through systematic application of a rules-based process again and again under diversification, can take advantage of these anomalies if and when they occur. Unlike convergent methods, divergent methods that target the tails of the distribution have no profit target in mind as the uncertainty associated with the extremity of these moves cannot be anticipated. Now the penalty clause for divergence is not nearly as severe as it is for convergent methods. While markets are noisy or during convergent phases, most of the time, divergent traders simply do not participate in trade events, but if they get it wrong, which may happen many times in succession, the risk management practices used by this technique simply leads to many small losses. These can and do pile up over time, but they can be managed, and control exercised over capital preservation. Ideally, divergent traders define the Gaussian envelope itself and use this envelope as a basis to decide when to trade. As prices start to extend to the limits of the Gaussian envelope, divergent traders ‘lock and load’ in anticipation of an imminent trade event to ‘destination unknown’.
The very infrequent nature of the tail event makes the divergent trader very uncertain and forces them to diversify and adopt strict risk mitigation measures for the times that they are frequently wrong and need to preserve capital. All trades apply equal small $risk bets as there is never any guarantee if a trade will result in a possible windfall. However, the rare times we are right, the market can deliver significant wealth building returns despite the small bet sizes given the non-linear moves that are capable within the left and right tails.
The dichotomy between both approaches requires totally different methods and trading behaviours, yet both can yield alpha….however given the vast differences and its impact on the psychology, they don’t mix well together for the psyche. We strongly recommend that you pick your poison and stick with it. Playing the game from both angles inevitably leads to tears and you frequently find that one approach will detract from the other.
What needs to be noted at this stage is that assuming an edge can be found at both loci (the peak and the tails), then a failure in one approach is the gain for the alternative approach. This is why we state in this zero-sum game that trend followers operate in that zone where ‘predictive model’s break-down’.
You may be familiar with the movie “Margin Call”. In that movie, the Hedge Fund experienced a risk event where market conditions strayed outside the tolerable thresholds of their modelling resulting in the company recognizing that they needed to quickly offload the risk they were holding to avoid a corporate collapse. This style of hedge fund adopted convergent processes and warehoused risk within their models. Sitting on the other side of a trade in events like these are the trend followers who capitalise on those prey whose ‘models break down’ in these extreme market conditions.
Scene from Margin Call – The Movie (with Poetic Licence)
Apart from the difficulty faced by convergent models in managing risk another issue commonly faced in convergence is that there is considerable room at the peak of the distribution for many different central tendencies. Choosing the right equilibrium level is a task that requires considerable statistical expertise in application and there is no guarantee in how long that condition will last until a new equilibrium level is found. The markets’ ability to select numerous equilibrium levels over its lifetime requires you to stay on your game and correctly identify when conditions have changed, and it is this issue of convergence which results in many account blow-ups. These methods focus on chasing returns with little emphasis placed on risk management.
However, for Divergent traders, risk management is our primary goal. To stay alive until an exotic directional anomaly occurs whereby our systems with their rules-based approach latch onto the outlier which the market can deliver from time to time. If we can stay outside the bounds of the normal distribution and only participate at those times where we are at the edge of the distribution near the tails, then we can avoid the incessant noise and mean reverting tendency of the market. All we must do is trade both the left and right tails of the distribution with tight stops to prevent us from ever entering the dreaded fat tailed environment in the opposite direction to our chosen trade direction.
What gives us faith in our philosophy is that we are realists who accept that markets are complex adaptive systems as opposed to theoretical constructs that obey statistical principles riddled with assumptions.
Real world events induce abrupt change to otherwise well-behaved markets creating extreme variation within these otherwise mean reverting or random conditions leading to a significant proportion of overall price variance.
Stay tuned for our next instalment in this Primer Series.
Trade well and prosper
The ATS mob