Backtesting and Diversification – The Essentials for Price Following
There is always the inevitable debate between those who think backtesting is a waste of time and those who treat the process like gold. This argument typically splits us into our two major trading philosophies we have discussed before…….. that of prediction, where the emphasis is placed on recent market conditions (hence backtesting long data sets is meaningless) and that of price following (where future conditions are unpredictable and we like to test our systems over a broad set of market conditions in the past to see how well our systems manage risk).
Let’s dig into why price followers like to use the long term as opposed to the short term in our testing assumptions.
You might be aware of the enduring feature of ALL liquid markets where academic studies have demonstrated across a vast array of different markets that there is a persistent anomaly in the data referred to as momentum. When something is persistent, it really shouldn’t be referred to as an ‘anomaly’. As an enduring feature…it really should be part of the overall description in which markets simply behave. The fact that markets move between different equilibrium means that transitions (or momentum) is simply a necessary feature of the markets for this to occur.
….and the argument that some markets don’t trend is a misnomer as already discussed. To check this for yourselves, just open up a monthly chart on any liquid market with a history of 20 years or more and you can see this for yourselves.
For example, a market that has plagued me for many years now is AUDCAD. It is tempting to assume that it just mean reverts around an equilibrium and doesn’t trend….however look at the following monthly chart.
We had a monster fast short trend in 2007 and some great easy trends in 2000 – 2003 and 2004 – 2006. Sure for the past 10 years it has been messy where each monthly bar has been wide with no clear directional movement. A trend trader dedicating his fortunes towards AUDCAD would have been whipsawed to death….. but the bottom line is this……. no-one can predict when markets simply stop trending or not. It is easy in hind-site but a different story when trading the right-edge.
Now more often than not….the throwing of the hands up in the air and declaring that “this market doesn’t trend!!!” is a granular statement based on expectations from the system you are trading….not the characteristics of the market itself. For example I am a D1 trader in general, and my medium term to longer term expectations have really been tested by AUDCAD over the last 10 years….so much so….that it is tempting to throw in the towel and simply exclude AUDCAD from my universe. An M30 or M10 swing trader might be staring at this sentence with a quizzical expression stating…”what you talkin’ bout Holmes?…..AUDCAD has been trending like the clappers on my timeframe”…….. The granularity at which you address the market is the issue here which is a timeframe diversification problem.
So you see the conundrum here. It is not that liquid markets don’t trend, in fact they have to as economies change over time and are never static…but what really is the problem is that they don’t have to trend when we want them to….and further the system we deploy to catch those trends is a huge obstacle in catching the market trend. Each variable we include in our strategy can be the snag that whipsaws us out of a generally trending market condition. We therefore need simple systems with lots of room to breathe if we want to tackle them.
Let’s look at drawdowns for a moment. Let’s assume that you commence trading AUDCAD in March 2010 where you are not diversified and you simply selected AUDCAD as a good market to trade based on its recent trending history over 2007. You would have immediately entered drawdown….and it would have kept on going. The rate of descent would be partly attributed to the market itself but also dominantly attributed to your position sizing (leverage). If your leverage was high, your descent would have been faster. Can you see now why leverage is perhaps your biggest factor to address in managing drawdowns? You can’t blame the market here….but you can blame your position sizing based on past expectations and your single system that has been configured to exploit the past historic condition.
This reliance on a single future outcome that represents an extension of the past is what kills you here. Not the market….but rather that grey squishy stuff in your skull…being your expectations about the future. The future will never be the same and therefore you need a diverse armory of different systems, different markets and different timeframes (granularity).
So what do we get when we summarise all this.
1. We need to test our philosophy over a very long term data set. This ensures that our position sizing is not too over-leveraged for the future based on what has recently occurred in the past.
2. We test over a long term horizon to challenge our biased assumptions. We need to test to remove that bias that may exist in our head that certain markets do not trend.
3. We test over a long term data set to deliberately capture ‘different market conditions’ and observe their impact on our choice of systems we deploy. The future is many possible paths….not a single path that is an extension of the past. We therefore need to test across a broad range of different past market conditions to see how your choice of systems fares with many possible market conditions in the future. You will never get an exact repeat, but in specifically looking at the worst conditions of the past, you are more robust in terms of your ability to address an uncertain future.
4. All markets trend….but they do so at times the market dictates…..not at our whim. We need to ensure we maintain a watch-list over a vast universe of markets and continuously test our systems across all of them. This does not mean that we have to trade all markets all the time….we will get to that later at the end of this post. We therefore need to maintain a diverse watch-list over an array of different liquid markets (and across asset classes if possible).
4. A single system in it’s configuration provides variables that are obstacles for trending markets. We cannot dictate how a trend will unfold. It is wiggly and does not take a trajectory of nice straight paths. Our stop and trailing stop are the major obstacles we need to address here. They must be sufficiently wide to capture the volatility inherent in a broad class of different trends. We can never select simply one of them as the best….we need a range of them to give us the greatest chance of what the market delivers to us. You need diversification of systems to achieve this.
5. The level of granularity we use to address trending markets is another major obstacle. We need diversification across time (not necessarily timeframes) to address this issue. For example using a 50 period Donchian with a 5ATR trailing stop exit as a single system addresses a particular trend duration….. using a 200 period Donchian with an 8ATR trailing stop address a different trend duration. This can all be achieved on D1 using the same system design with varying parameters without having to trade off different timeframe charts.
So what is the result of all this……well you will find that you have a vast array of different markets to choose from, a vast array of different timeframes to choose from and a vast array of different systems to choose from….talk about choices.
Now even though the Jerry Parker’s, Moritz Siebert’s and Niels Kaastrup-Larsen’s of this world insist that you can never be too widely diversified…..the reality is that in our finite small retail world your margin and capital requirements will dictate that a finite selection from this huge array of options is always necessary…..but which to pick…*scratches head*. Jerry, Moritz and Niels are right of course…they always are….but we have to accept our limitations in this small scale world.
Now this is where we are faced with the decision of a diversified portfolio. This is where you need to follow the processes outlined in prior posts on this blog to compile your robust solutions. You throw all the trade statistics of these various swathe of options into your compiler and let it do the rest of the decision making. Ensure the compilation method you deploy is quantifiable and not simply an adhoc subjective decision.
- Your compiler will address which markets to currently trade (those that have a slight profit factor over the test period (aka greatest chance of having a slim edge));
- Your compiler will risk adjust each option to ensure that leverage (position sizing) is not influencing your overall decision and that each option can be compared to each other in a standardized way;
- Your compiler will eliminate any weak deficiencies that may compromise the overall portfolio;
- Your compiler will ensure you adopt a risk target that you stick to such as a max 20% drawdown…and adjust the mix of options to achieve that result on an ongoing basis. No longer do you have to guess if your system is broke or simply exhibiting a drawdown. The compiler will take this burden from your shoulders and dynamically adjust the portfolio to achieve this going forward; and
- Your compiler will eliminate those options that do not contribute to the overall portfolio result.
Do not let that brain of yours step in the way of systematic processes. Avoid the temptation to override.
Your first portfolio is a bit like the sourdough starter of your fine bread of the future. Look after it and grow it. You never have to go back to ground zero to start again with it. You simply improve it on an ongoing basis with the rinse and repeat process we have outlined here where you options are added to the mix and your compiler discards the non-contributors objectively.
Trade well and prosper
The ATS mob