The initial idea surrounding Kantu – to use parameterless price patterns to generate strategies – allows us to come up with a lot of trading systems that may generate satisfactory historical results across several financial instruments but the truth seems to be that the scope of these patterns limits the possible trading solutions that can be found across the board. The main limitation is that the patterns only use OHLC information and derive patterns through an addition of rules of the form [1] (><) [2] where [1] and [2] are a choice of the open, high, low or close of any given candle available for selection. In order to expand the potential of the program it is necessary to allow the software to generate rules that go outside of this “box” by giving the program freedom to generate rules in another manner. This does not mean that you will go outside of the parameterless price pattern model, but that you will consider other possible choices. For example you could also potentially run comparisons between a candle’s middle value (middle distance between high and low) and the OHLC values, and in this manner you can generate other potential patterns that do not rely on OHLC values.

-

-

When this became apparent I decided to implement some additional rules, but after some coding it became obvious that I couldn’t code everything that users might want to use. Since Kantu already implements a powerful back-tester it also seemed unreasonable to limit the scope of the software, given that some users might also want to use things such as traditional indicators or some other type of pattern that requires some limited parameterization, such as the definition of VSA patterns, harmonic patterns, etc. In order to comply with this I modified Kantu to be able to use any type of custom input, so that users can add the inputs that they desire into the software so that they can generate systems that perform comparisons with whatever data they want. The new feature extension for Kantu allows a user to add any input to the software and generate systems that use rules based on these inputs to achieve the desired historical results. It is also important to note that this functionality is completely independent from the previously implemented features, the user can decide whether to use the regular price-pattern generation technique or the new custom inputs when loading data.

The process to do this is fairly simple. When creating data to import into Kantu – either through MT4 or through the F4 framework – the user can add whatever variables he wants to use for strategy creation within the file. For example if I wanted to use the middle point of a bar as an additional input I would simply add it as an additional column to the csv that contains the data. In order to make this process as error-free as possible I can simply add an additional parameter to the code that creates the csv file so that my input for each bar is also saved for the whole array. In The F4 framework the csv file is saved using the RecordBars EA, with this code that saves time and OHLCV values:

fprintf(fp,”%s, %lf, %lf, %lf, %lf\n”,timeString, open(1), high(1), low(1), close(1));

If I want to add the middle point, I would simply add an additional column to the csv:

fprintf(fp,”%s, %lf, %lf, %lf, %lf, %lf\n”,timeString, open(1), high(1), low(1), close(1), (low(1)+(high(1)-low(1))/2));

In this line the (low(1)+(high(1)-low(1))/2) defines the middle point of the bar which we are saving as an additional column. In this same manner I can save anything I would want to use, the average volume, the body as a percentage of the ATR, the candle range, etc. I can also have things such as the body/range ratio or even indicators such as the MA, MACD, etc. I could even add the OHLC of another pair to use multiple pairs for system generation! In fact there is absolutely no limit to the nature or amount of variables I can add. I could add 50 columns with 50 different inputs and the software can then use these inputs to generate strategies. As a matter of fact I could even add fixed constants if I want to make sure that I will be able to generate systems that make some fixed comparison such as RSI(20) > 50. One of the few limitations is clearly that – since Kantu doesn’t know the nature of the inputs – it cannot automatically generate code for the systems but pseudo code is generated as showed on the image above (full code generation is only possible right now for the price pattern functionality).

With this coding addition the scope of Kantu becomes limitless in the sense that the user can now use any given input to generate trading strategies, the limitations on the inputs – such as ensuring them to be parameterless – are the sole responsibility of the user and the software simply finds strategies with some requested historical performance that use the data that has been included within the loaded data file. My idea with this release will be to provide Kantu users with an increased ability to create strategies with whatever inputs they may desire to use. This may also shine some light into the differences between indicators/price action based systems and the differences between different types of indicators. I would also like to point out that all testing functionality – genetic search, walk forward analysis, fixed quota, IS/OS study, etc – is also available when using custom inputs.

This release is currently on the works and will be released before next Monday. If you would like to learn more about system generation and how you too can create trading systems using software please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

However this use of the Monte Carlo simulation is very simple and it is limited in the sense that you are only stopping strategies along one of the many possible worst case scenarios. But what do I mean by “many” worst case scenarios? Well, the scenario I described above talks about a convergent Monte Carlo simulation, where we are looking at the absolutely worst case that can arrive as the number of trades becomes large. This means that no matter how many additional trades we have, the probability of facing a deeper drawdown if the statistics of the system remain constant is close to zero. This is because the simulation already goes beyond the point of guaranteed profitability, a trade number beyond which the system with a positive expectancy will always generate a positive return with a confidence beyond 99.999%. Generating a deeper drawdown than that predicted by a 100K iteration worst case beyond this number of trades becomes extremely hard as this is actually the limit of the trading strategy.

-

-

What are the other worst cases ? The above scenario allows traders to make a very broad assumption, which is that all drawdowns are created equal. If a trader commits to the convergent Monte Carlo simulation worst case, they are effectively saying that they will wait for this worst case before stopping the strategy, an absolute proof that the strategy’s long term edge has failed. But can we find proof that the system has failed before we reach the convergent worst case ? The answer is yes and this is something that will allow you to stop a strategy way before it has to reach the much deeper level predicted within the convergent case. The trick here is that we must understand that not all drawdowns are created equal and a smaller drawdown period can constitute a worst case provided that it happens within certain conditions. This means that if your convergent worst case is a 30% drawdown, you might be able to say with a 99.999% that your system has failed at a 15% drawdown, provided that this drawdown happens within some predefined conditions.

How is this so? They key is in the number of trades that it takes you to reach a certain drawdown level. When you carry out a Monte Carlo simulation across a variety of trade numbers that are smaller than the convergent case, you can find out the level of confidence with which you can discard a system after only a few dozen trades. The above image shows you an example of a system where I have carried out simulations across a wide variety of trade numbers and you can see that the Monte Carlo worst cases are different for each case. It is also important to look at the number of iterations below the target drawdown figure that you want to use for discarding (above it’s been calculated 8%) as this will tell you how confident you can be about discarding your strategy. This is extremely important because a 2 trade MC simulation would tell you that the worst case across 100K simulations is 2 consecutive loses but this doesn’t mean that the system should be discarded because the probability to achieve this result across the 100K iterations is probably around 50% (meaning that this is just a highly probable behaviour across this trade number).

The analysis for the sample strategy showed above allows me to make predictions for the 20 trade case that show me when the strategy can be discarded. For example if I reach a drawdown of 8% after 20 trades I know that I can discard the strategy with a confidence of 99.9% because the probability that my system would reach a 8% drawdown so quickly are close to zero (0.1%). However note that as the number of trades goes up the probability to reach a 9% drawdown also increases, meaning that if this drawdown is reached after 80 trades I can no longer make this decision because the probability that I am making a wrong call is much higher. In this case I wouldn’t want to wait till the convergent worst case (around 26-30%) if I can discard my strategy after an only 8% drawdown that happens under the right circumstances. If for my distribution of returns the probability to reach an 8% drawdown after 20 trades already allows me to discard a strategy with a 99.9% confidence, there is no reason to wait for the wider worst case.

-

-

In summary I want to show you that you can better manage your trading systems if you not only pay attention to drawdown depths but also to the amount of trades it takes to reach a certain drawdown level. If you run Monte Carlo simulations for different trade numbers you can come up with worst case scenarios that will allow you to discard strategies when they “nose dive” into drawdown levels that would be close to impossible if their long term distribution of returns was to be preserved. I also want to call some attention to the results of the same system with a game theory money management module (second image), which shows how advanced money management affect the strategies overall results. When using game theory we could discard with smaller worst case scenarios, especially in the short term (20-40 trades).

The above analysis features – as well as several other enhancements – are currently being implemented on the Asirikuy Monte Carlo simulation which Asirikuy members will be able to use soon. If you would like to learn more about Monte Carlo simualtions and system analysis please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

What is the problem with the creation of portfolios ? Let us suppose you have two systems, A and B. When you simulate a run of A+B you get much better results than when you trade A or B on their own but you are unsure if you should put them together for live trading. However you decide to go forward and trade them live but suddenly you hit a drawdown period where both A and B tank and you are left with a reaching of your statistically determined worst case scenario, without knowing what went wrong. Why was your decision to put A and B together wrong if your back-testing results show you that it made perfect sense, the results were in fact better – and the risk was supposedly less – than when you traded them apart! In order to understand why things went wrong and why the choice to run A+B together wasn’t justified we first need to understand what makes systems give better results when simulated as a portfolio under historical testing.

-

-

If A and B are both profitable during your back-testing results, then it is obvious that the profits of A and B will be additive since both of them are – in the end – going to generate profits. However the drawdown periods from A and B will most likely not perfectly match and therefore you will see that your maximum drawdown periods will – with a very high probability – drop to a smaller value. So you have added a lot of compounding power (additive profits) but your drawdown has dropped due to the misalignment of drawdown periods. Results for portfolios of historically profitable systems will always be better than the individual systems due to the above reasons. You need to look for something beyond the simple improvement in trading statistics to be able to judge if A and B should belong together or if they should stay apart. But what makes this difference?

One possible answer: study system correlations.  Correlations measure the way in which the results of a strategy are associated with the results of another, whether a 2% profit on system A is likely to cause a 2% profit/loss on system B. If system A makes 1% and system B always also makes 1% then the correlation coefficient between the two is 1 while if the opposite happens, A makes 1% and B loses 1%, then the correlation coefficient is -1. However you shouldn’t study the correlations of system results as a whole but you should calculate the moving window correlation of some return figure (correlation for the past X monthly returns) and see how it evolves through a long term (10+ year test). This will give you an idea of how stable the relationship between the strategies is and therefore how likely it might be in the future for the strategies to correlate and drag you into a deep drawdown. If your systems are always negatively correlated then there is a big chance that there is a causal link that prevents drawdown alignment and therefore it would be a safer bet to trade A+B. Note that correlations between systems will never be negative “all the time” because they sometimes align in times of profit! It is also important to see when positive correlations have happened (whether they align with loses or profits).

So what should you look for? The images in this post show you several examples of system correlations. In all cases A+B give better results than either A or B traded alone but in some cases it is clear that the correlations imply that the systems should not be traded together. The first image shows the 48 moving monthly return Pearson correlation coefficient for two trend following strategies on the EUR/USD, since both strategies tackle similar market phenomena they show a very high positive correlation through the entire test (they only have a two month period of slightly negative correlation!). Although the results of A+B are better than A or B we should not trade these two systems together because it is obvious that they have a tendency to work in the same way and therefore the probability that they will align and kill the account when they fail is quite high. It is evident that systems that are designed with poor regard to correlation tend to have worse robustness because the strategies are hard-wired to tackle similar market phenomena.

-

-

Take a look at the second image I have posted. This image shows you a counter-trending strategy coupled with a trend-following strategy on the EUR/USD. The strategies have a much better hedging relationship and you can see that correlations are largely negative through a very big portion of the test. The positive portions you see correspond to profitable periods and – even then – the magnitude of these correlations is quite small (less than 0.2). You could therefore infer that the probability that these two systems will align within a significant drawdown phase is much smaller because the very nature of the strategies suggests that they trade in different ways that generate return structures that tend to be negatively correlated. This obviously doesn’t mean that the systems cannot become heavily and positively correlated in the future but it does mean that the chance of this happening becomes much less significant. The idea here is to pair strategies that are fundamentally different so that “trading in the same way” becomes much harder between them.

It is also worth noting that the Pearson correlation coefficient is not very robust to outliers (assumes a normal distribution) and therefore it’s not the best idea to use the simple monthly returns to calculate it. It is therefore useful to use variants of the monthly return that can be treated via logarithmic transformations (as Fd has suggested within our community) to generate normal distributions. When working with the regular monthly return (non-normal) the Spearman correlation coefficient might be more accurate (as it is more robust to outliers and makes no assumption about normality).

Last but not least, I would like to thank Fabio – an Asirikuy member – who pointed me in the direction of some material that suggested the use of correlation windows as a tool to enhance the building of trading portfolios (so this is definitely not a new thing). This is certainly a useful tool that makes the building of portfolios that are “very likely to fail” much less likely. Of course, if you would like to learn more about trading systems and portfolio building  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

When you develop a machine learning technique for trading you are generally searching for a method that allows you to predict a given outcome that translates directly into an inefficiency (a net positive, long term monetary gain for you). This means that you will generally attempt to predict both the future direction of price and its magnitude, so that you can get a good idea of how much money you will be able to make if your prediction is right. My approach has generally been to attempt to predict the direction/magnitude of the next bar in a price series and this can be quite successful for the higher time frames with the appropriate input/output structures. The reason is simply that the daily time frame bars are large and therefore the movements within them are easier to predict than the movements within a lower time frame, which are inherently more random. Applying this simple methodology – attempt to predict the next bar(s) – leads to huge failure along the lower time frames and therefore this technique is limited to only higher time frames (but even then it’s likely to be sub-optimal).

-

-

Looking at the way in which regular trading systems are developed – through mathematical expectancy analysis – it soon becomes clear that when you develop a system using machine learning you should aim to look for trade entries that have a high chance of success towards a predefined profit target and position holding time. This means that you shouldn’t train a system in order to learn if the next hourly bar (or the next ten or twenty or fifty for that matter) will be bullish or bearish but you will simply train it to give you an idea of whether – if you had entered a trade on that bar – after X time you would have eventually reached your profit target before your loss target (this takes into account the whole price range (high/low) movement). This means that you train your system to distinguish circumstances that lead to favourable movements  within a given predefined maximum holding time, without constraining your strategy to trying to predict the outcome of a particular time period. The inputs used to define the machine learning method are bound to be similar, so the outputs are marking the main difference between this approach and the more basic “predict if the outcome is bearish or bullish for the next bar(s)”.

It is no secret that up until now my success in the development of machine learning techniques for the lower time frames has been very limited but I believe that this is precisely due to the above issue dealing with the way in which I have been designing my predictions. Attempting to predict if the net movement for the next bar(s) will be positive or negative leads to very frequent trade entries – as your machine learning technique tends to predict one or the other – while having an output that attempts to forecast the success of an actual trade entry (whether you will reach the profit target before the loss target) is something much more in line with what we want as we are directly training the system to reach a predefined trading goal. With this type of output you can trade a clearer probability for trade success or failure while the “next bar(s)” prediction does not allow you to take into account the total price excursions but merely a very limited view of the open/close differences of the selected period.

-

-

This is perfectly in line with the work of Belo and Barbosa and other academics who have postulated that higher time frames are more profitable for machine learning because they seem to be more predictable (have less noise). The issue, as explained above, is most likely related with the way in which they are trying to predict things. Certainly it is extremely difficult to predict the outcome of the next 15 minute or 1 hour bar while predicting the outcome of the next daily bar is much easier, the reason is that the daily bar is simply much bigger and therefore its movement represents overall market phenomena much more clearly. The problem here is that you’re trying to predict the wrong thing. I believe there is no reason why you cannot achieve a profitable 1H system using machine learning, but it seems naive to believe that you can do this by predicting the next 1H bar (which is very difficult) but you should instead predict your chances of success if you entered a trade now and you want to hold it for a maximum of, for example, 10 hours.

My next goal in the development of machine learning strategies will be to tackle the issue of lower time frame machine learning systems using the above mentioned approach. I will first try simple linear/kernel  classifier models that are bound to be much computationally lighter and I will then switch to more advanced models such as neural networks and support vector machines. I have yet to see if the above mentioned technique will be successful in machine learning but my first experiments point to some preliminary success. I also want to know if I can also achieve success without the use of binary classifiers but simply by predicting directly the mathematical expectancy values of a trade taken at a given point in time with a defined holding time. Do you have any ideas to develop better machine learning systems for the lower time frames ? Do you have any opinion about my proposed design ? Leave  a comment with any questions or suggestions!

If you would like to learn more about machine learning techniques and how you too can use libraries such as FANN and shark to design trading strategies  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

When I first started my journey in machine learning (ML) – with neural network strategies – my first approach was to use data from a given symbol in order to predict its outcome in the future. I was quite successful with this approach on the EUR/USD where I was able to quickly come up with profitable neural network strategies (Sunqu, Paqarin and Tapuy) even being able to create NN strategies that used images in order to learn from the market. However it soon became clear to me that this strategy wasn’t going to be successful on other pairs. Pairs like the USD/JPY and the GBP/USD didn’t seem to be predictable merely from the OHLC information from their own time series. By looking at the academic literature on the matter – although often limited to the accuracy of the ML method and not the actual profit, which is what matters – I was able to conclude that people who made machine learning approaches for these symbols had often used much more complicated ensembles than simple inputs such as the previous day’s bullish/bearish bias.

-

-

My view is that a machine learning method created from an over complicated set of expressions derived from a time series, such as a bunch of different indicators, is much less preferable than a method that is able to predict based on much simpler inputs. My experience tells me that these methods are weak and in fact they have never been able to survive the constant re-training tests to which I submit my ML implementations when back/live trading. Of course, this might be something I’m doing wrong but – in any case – I prefer to focus on methods that use simple inputs to make predictions. So why is it relatively easy to achieve success with a symbol such as the EUR/USD while for the USD/JPY and the GBP/USD it is so difficult? This is a question that I have been asking myself for a while as for me it is fundamental to be able to create ML strategies for as many instruments as I want/need.

My first guess, is that we lack enough information. While the EUR/USD daily rates seem to provide enough information as to make successful predictions about the future, the USD/JPY and the GBP/USD do not contain enough information within their own data to predict their future with a decent accuracy. In this case the PCA analysis reveals that the OHLC variables that lead to higher predictability on the EUR/USD do not work very well on these symbols, pointing to the fact that we are missing the data that explains the variability within these pairs. Obviously it is clear that the movements on a single symbol might be related to a bunch of other symbols and fundamentals that represent wider economic conditions and therefore we might find success in the prediction of these instruments by using data that is not their own OHLC.

In order to bypass the limitations of the MT4 back-tester, I have decided to create a module for the F4 framework that will be in charge of acquiring historical data from an external source. By using data from an online source I am able to get at least EOD information from a wide variety of symbols up to the point where it is needed. Since obtaining data outside of the back-tester obviously opens up the possibility for data-snooping (as you could get data for future dates relative to your currently back-tested candle) the data is always queried from the online source such that only data up to the last closed date of the back-test is used (therefore avoiding any snooping by design). By implementing this library we are able to get EOD data from any requested symbol (gold, silver, S&P500, Dow, etc) for the date needed within our back-test. When live-trading the library uses the same procedure in order to obtain data up to the last candle close, giving us the possibility to use the exact same mechanism on both live and back-testing.

-

-

The possibilities that open up are endless. With the ability to query data for a wide variety of instruments we are able to test machine learning approaches that correlate a wide variety of instruments while still using any front-end we want (MT4, MT5, etc). This means that I can now test whether there is a way to predict the USD/CAD using the EOD of oil futures and I can also look at data from the USD index or the EUR index in order to attempt to predict other pairs. The next step is to also include fundamental data – by parsing an online source as well – in order to be able to back-test using correlations between things such as trade balance, interest rate parity, etc. Certainly a true quantitative approach to trading would wrap a whole view of the market, rather than limited predictions based on a single pair’s own EOD data. This module will open up the possibility to get us exactly there, to enable us to make ML based predictions using a really global ensemble of market data. Image a system – as showed above – where the machine learning algorithm is mutated on every bar to actually adapt to the symbols that yielded the most profitable results within the past X bars.

If you would like to learn more about the creation of machine learning strategies and how you too can use the F4 framework to design strategies that use these techniques  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Definitely one of the biggest problems we currently have in trading is the design of a trading methodology.  The problem involves everything from deciding how to design a strategy, to optimizing a system once it is fully coded and then deciding for how long it should be traded and when it should be redesigned or re-optimized. The main problem is that this whole process requires the intervention of a human trader in order to design models, test models and choose how models are being used. Ideally we would want an algorithm that could do everything by itself, an algorithm that could have many models to choose from, select models in a way that works historically and then change the way in which it adapts and trades according to its historical results. We are talking here about a trading implementation that is autonomous and can function without any human intervention, evolving in a way that does its best to guarantee profits under unknown market conditions.

-

-

Is there such a thing? Is there a way to create a true trading robot? The answer seems to be that there is … at least someone trying :o). Rui Pedro and Orlando Belo have done some extensive research in this area and their approach to the solution of the problem is very interesting. These two guys build their autonomous robots using 3 different pieces that handle different parts of the trading process. The idea is that these parts are able to fully take care of the whole strategy design, decision making and money management aspects of the trading strategy, also evolving trading rules as the market changes and the system needs to adapt.

So how do they do it? The first part of their system is an Ensemble Model that uses several classifiers and regression models in order to draw predictions from the market data. This was very interesting for me because I had never thought about this solution when working with neural networks in the past. Instead of using a single predictive model (for example a neural network) you can use several different types of models that all have different advantages and disadvantages. Within the Ensemble model these guys always include logistic regression models, decision trees, neural networks, etc. In total they have over 10 models within their Ensemble model that are used by the autonomous system to have a variety of “choices” to trade the market. How do we go then from these models to making actual trading decisions?

The case-based reasoning system, which is the second layer of the 3 layered model, handles this part of the puzzle. This part of the model takes a look at what the system has experienced in the past according to previous model results and judges what should be traded and what should not be traded. For example if the system sees that in the past when the random forest and RIPPER rule learner predictors agreed and the system followed there was a profit, then it will bet that there will be a profit again this time if the case repeats itself. The case-based reasoning system analyses all the cases that the system has experienced and their outcomes, with the idea to adapt the system as the number of cases evolves. The quality of the predictors might change over time and the case-based reasoning system handles the relevance of the predictors in a way that can derive the best historical results.

The last part of their puzzle is actually a rule-based section where the system can be told things that it isn’t capable of learning through trading. For example it can be told that the first Friday month is the NFP release or that the trading week should be cut to certain hours (not in hind-sight but to reduce broker dependency for example), etc. This part of the trading model is the least complex as it only includes a bunch of hard-coded rules (that shouldn’t change) which allow the system to improve its results as this information cannot be derived through the trading data.

The most interesting thing about the above is that these guys built a model for the USD/JPY in 2007, using 2003-2007 data (there is no apparent optimization but this was the data they used to test the model) and then traded this same system live from 2007-2008 achieving results that were comparable with those of the in-sample period (they were less profitable but overall acceptable). Through their 2008 paper they also show some of the ways in which the predictors adjusted to new market conditions, which were not available within the period they used for the design of the autonomous implementation. Interestingly enough the results of the financial crisis and the periods that followed were never published in any paper but during the year 2010 they did release a new paper that described a whole multi-system approach that involved several autonomous systems, each trading on its own currency pair.

-

-

This  paper was less convincing as it involved different ensembles for the different currency pairs and the inputs of the regression models also involved indicators (which obviously need shift information that is subject to optimization). It is also important to mention that the models were fitted to maximize their individual predictive profits during the in-sample period, showing that the adaptive process was used to yield the best results across a certain set of market conditions. The method used by Rui and Belo therefore has the same disadvantages as a regular NN based system in the sense that the model conditions need to be defined and tested across some given market period. The models can still adapt as market conditions change and the way in which they adapt can in a certain degree be controlled by the case-based model that assigns their lot sizes. They therefore have a fundamentally first degree adaptive technique with some flexibility given by the second model used. The last paper these guys published on the subject – which talks about an Autonomous hedge fund – takes the concept even further, developing autonomous systems that trade FX, futures, stocks, etc.

One of the things that Rui and Belo achieved – which I haven’t been able to – is to produce adaptive models that give significantly profitable historical results in currencies besides the EUR/USD. Implementing some of their ideas and improving on several of them will also be part of my check-list for 2013. In order to do this I plan to use the C++ shark open source machine learning library and some imagination :o). If you would like to learn more about the NN systems we have developed and how you too can use F4 to implement your own neural network ideas  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

So how do we find out what the historically optimum in-sample size for system creation has been? The first step is to generate a large amount of profitable and symmetric in-sample systems using different sizes for the in-sample period and then compare their success rate across a fixed out of sample period size. It is also important to distribute this in-sample/out-of-sample pairs randomly across the whole back-testing data in order to avoid the fitting of the results to some particular set of market conditions. For example in order to evaluate the success rate of a 500 day in-sample period I generate 1000 profitable and symmetric systems using randomly selected 500 in sample periods followed by a 365 day out of sample. One test might be from 1992 to 1994 with an out of sample from 1994 to 1995 while another might be from 2005-2007 and then the out of sample from 2007 to 2008. I performed this analysis using in-sample sizes from 100 to 6500 days with a constant out of sample of 365 days. I also performed this analysis on EUR/USD data from 1986 to 2012 (before 2000 using DEM/USD data).

-

-

The first result we can analyse from the above experiment is the difference between the number of profitable and unprofitable OS periods. Intuitively we would think that the number of systems profitable in the OS would increase as a function of the in-sample size, since more market conditions would allow us to better accommodate the results of the OS period. This happens to a certain extent but it is also true that after a given point the increase of the in-sample size does not lead to a significantly better edge. As you can see in the image above, the number of systems profitable in the OS increases logarithmically as a function of the in-sample see (logarithmic fit in red), this predicts that the increase in your edge as the in-sample size increases will be less significant as your in-sample becomes larger and larger. This hints that the number of market conditions might be convergent to a certain extent, meaning that after a certain point you cannot “better adjust” to new market conditions based on past ones because you have already taken into account all the behaviour which is relevant. Note that the error lines show the variability I experienced after doing 10 different runs to account for the random nature of the system generation procedure used by Kantu.

These results also show us that there is a minimum number of in-sample days which is required to reach acceptable results. Systems created using data from less than 500 days are especially unreliable (out of sample edge is almost completely lost) while systems created with more than 3000 days of in-sample data tend to reach similar results. This shows us that there is a bare minimum of information that is needed to gain an edge in the OS while introducing much larger amounts of data only improves this edge slightly. In general this also supports the fact that 3000 days (about 8 years) is the best starting point for system generation. Using more years can give you a further advantage but it doesn’t seem to be fundamental for success in the OS. This supports our development methodology at Asirikuy, where we have used 9 year periods for system generation and optimization. From this it also seems important to note that there is in fact an improvement of the OS success with the in-sample size so using the largest possible in-sample size you can will confer you an additional advantage – at least in the case of the EUR/USD –  even if this is not too great.

-

-

Another interesting thing to analyse is how the OS profitability varies according to the size of the in-sample period. Obviously we want to have OS periods that are just as or more profitable than the  in-sample (measured as (absolute profit)/(total trades)) so that we can have a smaller probability of being disappointed by our OS performance. The graph above shows you the number of systems with OS profitability higher or equal than the in-sample per trade profitability. As you can see the results are fairly similar to the initial results we discussed – where we analysed only the net number of OS profitable systems – showing that the probability to have a profitability higher than the one of the in-sample period is also logarithmic in nature. The probability to have a better OS period – from a profitability perspective – also increase with the size of the in-sample period, the probability increases very fast at first, then increasing more linearly as we reach larger in-sample sizes. In contrast with the previous case, the increase of probability here is far more pronounced, hinting that – although the number of systems profitable in the OS might not increase very much with larger in-sample sizes – the increase in the probability of having a more profitable OS period – relative to the in-sample – is higher.

It seems that the above quantitative evidence supports the design paradigm that suggests the inclusion of as much data as possible in the system generation process. Having more data seems to lead to a higher probability of out of sample success, both from a probability of net profit and a probability to have profits higher than those of the in-sample period. Therefore it would be advisable to always include as much data as possible when generating a strategy because this generates an additional probability to be successful under unseen market conditions.  However it is also worth pointing out that the nature of both studied behaviours is logarithmic, meaning that we should expect convergence of these values towards some tangent as the in-sample size grows much larger. From the derived logarithmic fits it seems to be that in-sample sizes above 8000 days would give practically no improvement in the expected OS performance.

However the are still a lot of questions to answer about the above. Is this process the same across other pairs? Is this process the same across other markets? How does the out-of-sample size affect these results? These are all very interesting questions that I will certainly continue to research within the next few months :o). If you would like to learn more about system generation and how you too can perform your own research  please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

I have to say that through the lifetime of Asirikuy I have always received emails from time to time from people who are interested in using our tools without joining our community. People often wrote to me about the APDAT or the Monte Carlo simulator and it seemed to me that joining the community was essential to using this tools as I perceived as irresponsible behaviour to make tools available to people who might not have the knowledge necessary to take advantage of them properly. However during the past few months I received emails from several professional and regular retail traders who made a very good case for the external selling of tools and I have therefore decided to implement their advice and start this externalisation procedure.

-

-

One of the best arguments made, which I have come to realise in the past year, is that it is not my responsibility to educate those that do not want to be educated and therefore I should not be overly concerned with the way in which tools are used provided that I voice my concerns about their use properly. Another very important aspect is that making the tools available externally keeps away people who want to buy the tools but who are not interested in learning or in our trading implementations. Keeping access to our trading strategies and frameworks restricted to those who are interested in learning and contributing in a community environment is fundamental as those people interested in just access to tools might not take advantage of all the other aspects of our community.

In particular the most important reason why this makes sense is because by keeping everything bundled I feel that I am often making people pay more for the whole package while they could be paying less for just the tools if this is the only thing that they want to use. By joining Asirikuy I provide a learning experience that allows users to become more involved in the learning process and truly understand everything that is going on but I shouldn’t seek to provide this service to people who are simply not interested in this part of our community. Another important argument for sharing some tools is that I want to start sharing revenue with people who have contributed with the coding of some of these implementations within our community. 

If the programmers involved with each tool are in agreement with the external selling I will make them available through this website through a simple (one time) paypal link. Some tools might be more expensive than others but our analysis tools will most likely be in the 20-150 USD price range. The first tool that will be available through an external purchasing option will be Kantu – our parameterless price action based system generator – which will be available for 120-130 USD. There will be a demo version available for this program (everything but code exporting will be available). It is worth noting that programs with comparable functionality – such as adaptrade or priceaction lab – are retailing for a price well above the 1000 USD. Another advantage is that Kantu will be available for Windows, Linux and MacOS (unlike these other programs).

Purchasers of our tools will also have all minor version updates available for free (bug fixes, etc) and updates to new major releases will be available under a small fee (whether to update is obviously optional).  Obviously all updates are available to Asirikuy members for no additional charge and any manual or document developed for externally available tools will also be available to members of our community. Other aspects of our website – such as our Alpari data – might also be available for purchase (although I have to consult my source first). The Kantu demo and purchase option will be available from this week so stay tuned to learn more about this release :o)

If you have any concerns, questions or suggestions regarding this change please feel free to post within the comments, I am certainly eager to read what Asirikuy users and programmers think about this new direction  as well as people who are interested in our tools but not in joining our community :o)

If you would like to learn more about trading and how you too can become a better trader through a community approach please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Let’s suppose you are a fund manager in the year 1999 and you have been asked by your group of select clients to find a single parameter-less price action based daily system that the fund will be trading during the next 12 years on the “new and shiny” EUR/USD currency. Your clients want to have a very high probability for profit and they don’t want any surprises (meaning that they want something for which there is a high chance of past performance matching future results). Scratching your head, you walk into the office and wonder how you can actually select a strategy that is robust – having a high chance of future positive performance – with the minimum possible downside risk. Since you refused to buy that magic crystal ball the year before, you have nothing but your analytical skills to get you out of this problem. What is the best way to make this selection? Hint, the most intuitive things are the most prone to failure.

-

-

By using the DEM/USD data from 1986-1999 and Kantu – our price action based strategy generator at Asirikuy – I have created thousands of in-sample profitable and symmetric systems and analysed the in-sample statistical characteristics that profitable systems in the out-of-sample part (2000-2012) actually have in common. To my surprise, if you had made the decision to trade the strategies with the highest trading frequency or the highest profitable to drawdown ratios you would have made a big mistake as your probability of system failure during the next 12 years would have been very high. Seemingly contrary to what happens when you have small out-of-sample periods, when the intention is to have a long out-of-sample, trading frequency is not a determinant factor and in fact higher frequencies are detrimental to overall out-of-sample profitability. So focusing on this part of the spectrum is really not a good thing because neither higher frequencies nor better profit to drawdown statistics generated good results in the out of sample.

What works tremendously well (over 80% success rate) is actually a not so forward selection criteria. If you ignore everything else and only select systems that have very low maximum drawdown period length – for example less than 400 days – you end up with a bunch of systems that have a very high probability to succeed. These systems also have other things in common, such as relatively low trading frequencies (10-20 trades/year) and high stabilities (they are very linear in nature within the in-sample phase). Surprisingly if you had simply generated 10K systems and selected those with the lowest maximum drawdown period lengths you would have had a good chance of coming profitable in the next 12 years of out of sample testing. Statistics would have only been equal or better than in-sample on a limited number of cases (probably less than 10%) but in all cases you would be above your statistical worst case scenario and – in +80% of cases – you would be above a neutral balance change. Interestingly the worst cases here are not the highest frequency traders so although higher trading frequency does not lead to predictions of good long term out-of-sample performance, a high trading frequency on its own is also not a good criterion for discarding strategies. However if you attempt to restrict your maximum drawdown length search only to strategies that trade more frequently, you will get a much worse overall result.

As you can see, in the past we had a simple criteria related with the maximum drawdown period length that was more than enough to select a good trading strategy that would maintain its performance (or at least preserve our capital) for at least the next decade. The reason why this variable has such a strong predictive power can perhaps be related with the fact that it implicitly optimizes several other variables that are also correlated with system robustness. For example a short maximum drawdown length implies that the system has been able to tackle a significant portion of previous market conditions, hinting at a more stable system that is more linear and less prone to fall into losing territory. If a system has a maximum drawdown length of 100 days during a 10 year period it implies that it could handle a wider variety of market conditions in a positive manner than a system that had a maximum drawdown length of 200 days.

-

-

Another main reason why I believe this criterion works so well is because it is not related to performance. Whenever you try to optimize a strategy based on a measure of profitability (such as the Sharpe ratio or the AAR to maximum drawdown ratio) you are never able to achieve a positive long term out-of-sample with a good statistical edge (at least under my testing conditions). The market seems to punish those who attempt to follow the track of the best performing strategies (from a return perspective) while a reward is given to those who focus on the stability of returns. This makes sense as highly profitable in-sample strategies can happen due to many reasons, for example due to excessive profitability during a very reduced percentage of market conditions – while a very stable system loses in-sample profitability at the expense of a higher stability of returns. Fd has also suggested the use of the kurtosis and skewness of returns as predictive variables for out-of-sample performance and we will also see in future posts that these variables can also play an important role. I believe that the predictive power of these variables is also related to the fact that they are not related to performance but to the stability of the trading strategy.

In essence the above research gives some hints into what might work when designing systems for long term profitable out-of-sample performance. Optimizing systems to give the highest return or lowest drawdown levels hasn’t worked in the past but using non-profit related variables that relate to system stability has yielded the best results. Choosing systems that have performed the most evenly across a large set of market conditions seems to be a key factor in the building of strategies that are bound to last for long periods of time. It is also interesting to note that these results hold when expanding the in-sample period, something interesting that supports the use of all in-sample data to build long-term profitable systems. Right now it seems to me that using 1986-2012 data to look for strategies that have extremely high stability would be a very good way to build strategies with a low probability for failure and – even if they fail – with the possibility of detecting failure in a very fast manner.  Trading a portfolio of the top performers – from a stability perspective – seems to be the best idea.

-

-

If you would like to learn more Kantu and how you too can build your own price action based trading strategies please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

The question of whether it is best to design systems for one pair or another is not easy to answer. Many people will say that the ability to develop systems on one pair rather than another stems from personal bias and skills rather than from an innate difference between the pairs that makes system development “easier or harder”. In order to void the human development factor and put the question in quantitative terms I want to find out whether in-sample profitable and symmetric strategies generated automatically on one pair have a higher probability of being profitable in out-of-sample than strategies generated on another pair. The procedure to get this information is pretty straightforward with Kantu; I first generated 20,000 historically profitable and symmetric systems on each different pair with random 9/3 in/out of sample period going from 1987 to 2012 and I then compared the results with another pair.  So one of the systems may have had an in sample period from 01.Jan.2000-01.Jan.2009 with an out of sample from 01.Jan.2009-01.Jan.2012 while another system might have gone from 1996 to 2005 in sample and then from 2005 to 2008 in out of sample. The randomization of the in/out of sample periods is fundamental to get a “full picture” of the currency pairs. Simulations included spreads as well as random slippage.

-

-

After we get the results we can then proceed to draw a distribution of out of sample returns that shows us the frequency (amount of systems) for each out of sample return class. As you can see in the image above (EUR/USD results) the out of sample returns tend to form a bell-shaped curve with the interesting fact that the curve is skewed towards positive territory. This means that if we generate a profitable and symmetric strategy on the EUR/USD we have a more than random chance of being profitable in out of sample regardless of the in-sample statistics. In the case of this pair it is enough to have something that is historically reasonable (profitable and symmetric through in-sample trading) to have an above chance expectation of positive performance for the next 3 years. In other words the EUR/USD is inefficient because we can get profitable out of sample results based on historical in-sample performance. This definitely explains why I always found it easier to optimize and create new systems on the EUR/USD and it also explains why neural networks tend to do so much better on this pair than on others. Note that the edge we have above is both regarding the number of profitable systems in out of sample (About 600 more than predicted if we had no edge) and total net addition of all profits and losses (meaning that the sum of all the out of sample results is positive). 

What happens when we look at another pair, such as the USD/CAD (which I have always had problems with). What we find is that the distribution of returns has changed and is now centred around negative territory. This means that profitable symmetric in-sample results were not enough to give you an edge on this pair during the past 25 years. The USD/CAD is therefore less inefficient than the EUR/USD because designing systems that give a positive expectation for out of sample profitability is simply harder as there are far more good in-sample strategies that fail in the out of sample when compared to the EUR/USD. This is a quantitative comparison of currency pair efficiency based on parameterless price action based pattern generation, giving us a very important clue about why designing systems for minors such as the USD/CAD might not be the easiest or most profitable idea. In fact if you try a system generation procedure where you select the most highly correlated in-sample performance variable with out of sample performance as your selection criteria you will see that results are always profitable for the EUR/USD while they are always losing for the USD/CAD (more on this methodology on a future post).

-

-

Why is there such a big difference between these pairs? My main explanation for this is a complete negation of the efficient market hypothesis (which we know is wrong, particularly since the strong failure of random walk based volatility prediction models during the financial crisis). My vision is that traders are not rational profit maximizers – as the efficient market hypothesis says – but that traders are in fact emotional and irrational and therefore higher volume leads to higher inefficiencies because there is a higher manifestation of the underlying human nature that makes the markets predictable to a certain degree. This means that an instrument will be easier to exploit if it has a higher trading volume, something that is confirmed when you analyse the other majors and minors which shows you a direct relationship between average yearly trading volume and the skewness of the out of sample distribution of returns derived by Kantu.

The conclusion is that it is far easier and potentially far more profitable to trade highly liquid pairs (at least algorithmically) than to trade minors or exotics. Although this doesn’t mean that profitable trading on these pairs is impossible – I’ll show you a way to do it on a future post – it is indeed telling us that there are pairs where it is easier and more practical to develop trading methodologies. This also has consequences for automatic system generation because it means that although we might be able to find profitable in-sample results for all pairs, only on some pairs will these systems tend to give profitable out of sample performance. Therefore the previous analysis of a pair’s efficiency is fundamental to determine whether algorithmic system generation is the correct approach or what modifications might be necessary to make it so.

If you would like to learn more about Kantu and how you too can develop algorithmic systems automatically please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)