In the Beginning there was Trend Following – A Primer – Part 16
Trend Following Primer Series – The Robustness Phase – T’is But a Scratch – Part 16
Primer Series Contents
- An Introduction- Part 1
- Care Less about Trend Form and More about the Bias within it- Part 2
- Divergence, Convergence and Noise – Part 3
- Revealing Non-Randomness through the Market Distribution of Returns – Part 4
- Characteristics of Complex Adaptive Markets – Part 5
- The Search for Sustainable Trading Models – Part 6
- The Need for an Enduring Edge – Part 7
- Compounding, Path Dependence and Positive Skew – Part 8
- A Risk Adjusted Approach to Maximise Geometric Returns – Part 9
- Diversification is Never Enough…for Trend Followers – Part 10
- Correlation Between Return Streams – Where all the Wiggling Matters – Part 11
- The Pain Arbitrage of Trend Following – Part 12
- Building a Diversified, Systematic, Trend Following Model – Part 13
- A Systematic Workflow Process Applied to Data Mining – Part 14
- Put Your Helmets On, It’s Time to Go Mining – Part 15
- The Robustness Phase – T’is But a Scratch – Part 16
- There is no Permanence, Only Change – Part 17
- Compiling a Sub Portfolio: A First Glimpse of our Creation – Part 18
- The Court Verdict: A Lesson In Hubris – Part 19
- Conclusion: All Things Come to an End, Even Trends – Part 20
The Robustness Phase – T’is But a Scratch
In our last Primer we revealed how our ‘design-first logic’ has avoided any propensity to produce curve fit results, when targeting the outliers using our trend following models.
We discussed why it is essential to adopt a “design-first logic” before we apply our powerful data mining processes towards the trend following quest, to avoid ‘fitting our solutions to noise’. Our logic enforced a causal relationship to unite our trading systems performance to the market behaviour (or the signals) that we want to participate in.
We also took you on a small journey where we demonstrated how it was essential to develop simple trend following systems with few variables, to avoid our propensity to be too prescriptive in our trend following designs.
…….and then we gave our engineers a number of ‘Golden Rules’, that must be observed in every simple design solution and they went ahead and developed them……thousands of them. All variations around these Golden Rules or core principles of trend following.
And here we are now. Our clever engineers have given us literally thousands of trend following systems of both ‘both breakout configuration’ and ‘other trending condition configurations’, that span a diverse array of trending form found in outliers, and may be applicable to include in our trend following portfolios generated at the end of the workflow process.
So now we can be confident that these solutions can respond to ‘Fat Tailed Environments’, and can therefore proceed to our next processing phase in our workflow to test how well these solutions have stacked up against the ‘outliers of the past’.
Once we complete this next phase of the process, then we have a higher probability that the ‘surviving candidates’ will have a good chance of navigating the ‘outliers of the future’.
Revisiting Robustness – Invert your Thinking.
In Primer 6 we came to the conclusion that robustness would be our chief determinant in our selection process for suitable return streams, and we adopted the MAR ratio as the preferred risk adjusted metric that we would use for our evaluation purposes. We need to keep this in the back of our minds when discussing robustness.
So now we have our vast array of system designs sitting in front of us delivered by our Engineering Division, and we want to now rigorously test them by putting them through the hoops with our large and diverse data stockpile, to evaluate any risk weakness that resides in them.
We already know that our system variations will respond well to trending conditions in these fat tailed environments, but we are unsure how these systems will perform over enduring periods of unfavourable market condition.
Notice what I am now doing now. I am flipping the mindset from thinking about the windfalls of outliers to thinking about risk associated with attempting to capture them.
Previously we came to the conclusion that our systems could catch the outlier, but now I am focussing on the risk of this eventuality being thwarted by noise and mean reversion. In other words I am now assessing the impact of our models when ‘normal market conditions’ interfere with our quest for outliers.
This is where risk lies for the Trend Follower. Our systems protect us from adverse left tail trade exposure, but we can be exposed to the costs of many whipsaws if we stray too far into the Gaussian envelope. This is what creates slow building drawdowns which we want to avoid, if possible.
Remember that we seek to trade the tails of the distribution of returns. What I am doing now by focussing on robustness is arbitrarily defining the dividing line between the bounds of a Gaussian distribution and the Fat Tailed conditions that extend beyond that distribution into the left and right tails.
We want our systems to not only catch the tails, which we know they will do through design logic (NOT optimisation), but we more importantly now want them to be survivors so that they can always be around to participate in fat tailed conditions IF they arise. The better survivors still catch the outlier but have lower drawdowns.
So, this is what we will be assessing in this second stage of the Workflow Process, termed the Robustness Phase. The ability of our systems to survive the hostile adverse market environment of the past. In our next Primer we will be assessing the ability of our systems to survive a possible hostile environment of the future.
But before we carry on, I would like to give you another example of how inverted logic can be applied successfully to crack puzzles wide apart.
The Curious but Clever Case of Abraham Wald
How to Assess Robustness in our Workflow Process
Now down to the nitty-gritty of this processing phase. After all, we have to do some ‘brain draining’ work sometime.
This Robustness phase will be used to assess:
- The risk adjusted returns of each systems return stream on a pre-cost basis so that we can then use visual methods or the MAR ratio to evaluate weakness across the entire return path (equity curve);
- The risk contribution that each return stream makes to a possible future portfolio. Each return stream will be non-compounded and normalised to allow for direct comparison between return streams;
- The Multi-market capability of the system which is a method we deploy to increase the low trade sample size that we achieve through our systematic methods applied to a single return stream. Multi-market capability lifts the trade sample size from a small trade sample to a large trade sample and is a principle measure we use to assess robustness; and
- The suitability of the return stream for future compounding treatment. An evaluation is made of each return streams MAR ratio on a non-compounded pre-cost basis to evaluate how well they are suited for later compounding treatment.
Before we complicate our exercise by not only looking at the trend following capability of each return stream, but also including the ‘broker specific’ cost inclusions that can have a major impact on our already weak edge that is present in our systems, we conduct this phase of the process on a pre-cost basis.
This means we exclude broker specific impacts such as spread, slippage, holding costs (or SWAP) during this phase of the process.
All we need for this phase of the process is the Open, High, Low, and Close data for any liquid market.
We will be backing in the broker specific costs in the later ‘recency phase’ of the workflow process to assess how the system can fare under realistic ‘live conditions’, but until then we don’t won’t to complicate this process.
Undertaking this assessment on a pre-cost basis therefore allows us to isolate performance in relation to the ability of our systems to capture trending environments and be influenced by noise and the mean reverting character that may lie in the market data.
Having achieved this, we can then use visual graphical methods to eyeball the equity curves produced to identify any ‘risk weakness in the signature. We can also use the MAR ratio as a method for this assessment. Given that we are not compounding the equity curve at this point we use the Return on Investment of the return series (ROI) as opposed to the Compound Annual Growth Rate (CAGR) and use a proxy for the MAR ratio as defined by ROI/Max Draw.
Determination of Risk Contribution
We normalise the return streams of each system using ATR based scaling techniques to allow for ‘apples for apples’ comparison amongst alternatives spanning different markets, timeframes and systems.
Each return stream has an equal $ risk bet applied per trade which ensures that volatility is normalised in equivalent dollars.
So for our thousands of ‘pre-cost’ return streams, we can now directly compare each and assess their overall contribution of risk to a possible future portfolio either by way of referring to their Return/Drawdown contribution or by visual assessment of their entire return stream which can be used under ‘visual comparative methods’.
Multi Market Capability
Now given that each return stream is only targeting unpredictable and rare outliers which can mean that we are holding trades for many years, the trade sample size per return stream is very low. We are trying to determine the impact of noise and mean reversion and their impact on overall robustness, and for this assessment, we need a high trade sample size.
We have a clever trick up our sleeve to achieve this given that we have now normalised each return stream from our prior processes.
We test our systems over our entire data set that comprises many markets as now each market is just different data whose independent qualities have just been eliminated by normalisation methods.
This lifts the trade sample size from say 15 trades over a 30 year plus market data sample to say 40x this level (eg. 600 trades per system) when we apply our entire data set. This significantly increases the rigour of this testing phase and ensures that the noise and mean reversion affects that contribute to whipsaws have statistical significance.
You see, each market is just data to us now offering a vast array of different possible paths upon which we can now test the efficacy of our systems under different market conditions.
We apply a filtering mechanism at this point that states that we want at least XX markets of the total YY multi-markets (say 30 markets of a total 40 in our universe) to pass this test by delivering positive expectancy.
There will be some markets over this long data history that do not possess sufficient outliers to generate a sufficient edge to achieve positive expectancy (pay for all the whipsaws), so we cannot expect our systems to achieve an edge over all markets tested on. But we do expect a high pass rate, and the better the multi-market capability of each return stream, then the more we like them.
Suitability of Each return Stream by Market for Future Compounding Treatment
Finally, after all the prior steps in this process, we now need to make a decision to further reduce the scope of the hundreds of return streams that still have survived to this point. After all, we do have finite capital restrictions that does limit our ability (through minimal allowable lot sizes applied by our Brokers), to trade all these options, even though we might not like to as we hate ‘selection bias’.
Reality unfortunately starts to take a bite, and some decisions need to be made, to select those candidates that we will be taking to the next phase of the workflow process.
For the survivors so far, that have now passed the multi-market test, we therefore now need to refer to their performance at the individual market level.
We use the proxy for our MAR ratio on a pre-cost basis as our metric for each normalised return stream for each market, to identify those return streams that offer the best reward to risk relationship on a per market basis.
We eliminate any candidate that has a negative MAR (negative expectancy), and then we rank the surviving candidates by positive MAR. Of course, this does introduce weak optimisation into our method, however we counter this by including the top 100 or so in our collection per market.
We need lots of survivors for our next step in the process and their survival is related to the MAR ratio that each normalised return stream generates when compounded.
So at the end of this entire process, we may have 40 discreet markets (defined by the extent of our universe) each with 100 strategies that have passed all phases of the historic robustness test. They all possess a weak edge, they are ideally configured for compounding, and they all have demonstrated their ability to survive the turmoil that the market has thrown at them across a vast array of different market conditions.
Chart 38 below provides an example of 40 Non-Compounded Return streams using Breakout Techniques and Various Trend Following Methods that have passed the Robustness Phase. Notice the correlated step-ups during material trending events displayed across the set, and the absence of risk weakness in the drawdown profiles during intervening periods of the data history. The process is starting to sculpt optimal equity curves for compounding treatment.
Chart 38: Example of 40 Return Streams for EURUSD which have passed the Robustness Phase
We now have a set of robust candidates ready to future treatment through the workflow.
Had enough of this robustness phase?…..I’ll bite your legs off if you get too close.
Now we are ready to move to the next phase of our workflow process, which is the Recency Phase.
This phase, like the process of natural selection, assesses how these robust candidates now fare over more recent market conditions. We inject live trading costs back into the results to simulate a live environment, and we then preferentially select those that have fitness embedded in them.
This is the adaptive phase of the process which is now required to ensure our systems can respond to dynamic market conditions.
So stay tuned to this series.
Trade well and prosper
The ATS mob