tag:blogger.com,1999:blog-13460453131959613522023-03-16T03:29:31.681-07:00MatlabTradingBlog for MATLAB® users interested in algorithmic trading strategies, backtesting, pairs trading, statistical arbitrage etc.Igorhttp://www.blogger.com/profile/10662229147968940872noreply@blogger.comBlogger53125tag:blogger.com,1999:blog-1346045313195961352.post-17632730396685588492016-12-07T23:12:00.000-08:002017-05-10T19:44:09.561-07:00Testing and Analysis of Algorithmic Trading Strategies in MATLAB (Part 4) – Genetic Algorithms<i>This post is about how important is to use different types of optimisation methods such as genetic algorithms and parallelisation to get results faster.</i><br /><br /><h2>Genetic Algorithms Optimisation</h2><br />Despite the fact that the <i>genetic (evolutionary) algorithm</i> principle is very well explained in the MathWorks webinars, in the examples, however, it is used only for optimisation of the choice of a strategy group from a set. This is a good example of the use of these algorithms, however, it happens that there is a need to set many variables with significant intervals for one strategy, you don't get by with one iteration and the <i>parallelisation</i> of processes – calculations can take several days. Certainly, there are strategies in the final stage of <i>optimisation</i>, when we almost surely know the <i>trading strategy</i> is successful, we can wait for several days as well or rent the whole cluster - the result might be worth it. However, if we need to "estimate" the results of a "bulky" strategy and decide if it is worth it to spend the time, then genetic algorithms may be perfectly suitable.<br /><br />We provide the possibility to use three methods to optimise the strategy in <a href="http://wfatoolbox.com/?utm_source=post4genetic&utm_medium=link&utm_campaign=Blog">WFAToolbox</a>:<br /><br /><ul><li><b>Linear method</b> – it is a usual mode of sorting in which you will see all intermediate (suboptimal) results. It gives maximum accuracy.</li><li><b>Parallel method</b> – all kernels of your CPU will be used. It does not allow to see intermediate results, but significantly speeds up the operation. It gives maximum accuracy during increase of computation speed.</li><li><b>Genetic method</b> – it uses the evolutionary optimisation algorithm. It allows to see suboptimal values, but gives the result close to the best. It's not a very accurate method, but it's precise enough for the initial "run" of the strategy. Very fast.</li></ul><br /><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><img alt="genetic algorithms with MATLAB" border="0" height="326" src="https://3.bp.blogspot.com/-n8hMozffIDE/WEkGi4-szKI/AAAAAAAAAOE/wRS1IQUU9PY79IlmJu9dHA0U51oU-ZOIQCLcB/s640/Genetic.png" title="genetic algorithms with MATLAB" width="582" /></td></tr><tr><td class="tr-caption" style="text-align: center;">Genetic Algorithms</td></tr></tbody></table><br /><br />We are often asked if <a href="http://wfatoolbox.com/?utm_source=post4genetic&utm_medium=link&utm_campaign=Blog">WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB</a> has the ability to use the GPU in calculations. Unfortunately, GPU is not suitable for all tasks and its use is very specific. In order to use it, you need to adjust the logic and the code of each strategy for graphic cores testing. Unfortunately, due to such non-universality of the method one cannot use GPU in <a href="http://wfatoolbox.com/?utm_source=post4genetic&utm_medium=link&utm_campaign=Blog">WFAToolbox</a>.Anonymoushttp://www.blogger.com/profile/15451548338600496196noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-43450919987668936472016-12-05T01:36:00.001-08:002017-05-10T19:43:49.400-07:00Testing and Analysis of Algorithmic Trading Strategies in MATLAB (Part 3) – Visualisation of Process<i>Continuing <a href="http://www.matlabtrading.com/2016/11/testing-and-analysis-of-algorithmic-trading-strategies-in-MATLAB-part2-easy-to-use-gui.html">Part 2</a> of the discussion of problems and solutions in testing and analysis of algorithmic trading strategy in MATLAB, I invite you to read this post about problem of unavailability of visualisation of the processes in modern software solutions for testing trading systems.</i><br /><br /><h2>Visualisation of Testing Process</h2><br />In my work experience, I often analysed other popular platforms for <i>trading strategy testing</i>, such as <i>TradeStation</i>, <i>MetaStock</i>, <i>Multicharts</i> etc. and I was always surprised at how little attention was paid to visualisation of the testing process. The thing is that when we don't see the results of the intermediate, sub-optimal values of optimised parameters, we often throw away gold along with the dirt. The matter is because of an overly broad sampling, the <i>strategy</i> adjusts the parameters the way we either see a "<i>perfect strategy</i>" which fails in real life or see one or two deals, which are supposedly the best because it was selected such time interval data where the best <i>trading strategy</i> would be "buy-and-hold", but why are then other <i>strategies</i> necessary for?<br /><br /><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/-KlKdplxqiH4/WEUvtDKHiFI/AAAAAAAAANQ/wbTYeE4TPIQxteIEJsA2WO5jSjLXkkVzQCLcB/s1600/iso-surface.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img alt="Visualisation of trading strategy testing process in MATLAB (proposed in webinar)" border="0" height="345" src="https://2.bp.blogspot.com/-KlKdplxqiH4/WEUvtDKHiFI/AAAAAAAAANQ/wbTYeE4TPIQxteIEJsA2WO5jSjLXkkVzQCLcB/s640/iso-surface.png" title="Visualisation of trading strategy testing process in MATLAB (proposed in webinar)" width="582" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Visualisation of trading strategy testing process in MATLAB (proposed in webinar)</td></tr></tbody></table><br /><br />As a result, without seeing intermediate results, we need to «blindly» change the parameters to try to get the better data; or watch it in some 3D or 4D (colour is the 4th dimension), as proposed in webinars. The analysis of values in the N-dimensional spaces can definitely be an alternative, but has several limitations:<br /><br /><ul><li>What if there are more than 4 dimensions?</li><li>When you see what signals and at what frequency they appear in the price range, you have almost all the necessary visual representation of your strategy: the frequency of the transactions, their profitability (income curve), the accuracy of opening, the similarity with other suboptimal values, etc.; that cannot be said about the performance in N-dimensional space where all useful information is, in fact, that the optimum value is not only one but there is a whole range of suboptimal values in one or more areas.</li></ul><br />While optimising a strategy in <a href="http://wfatoolbox.com/?utm_source=post3visualisation &utm_medium=link&utm_campaign=Blog">WFAToolbox – Walk-Forward Analysis Toolbox for MATLAB®</a>, as a new optimum value is found, the <i>trading strategy</i> signals in the period <i>in-sample</i> and <i>out-of-sample</i> immediately appear on the chart, so you can always control what range of options you should assign, and also you can pause the optimisation without waiting for the end of test, as it becomes clear that something went wrong or everything is fine.Anonymoushttp://www.blogger.com/profile/15451548338600496196noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-23924942092988527942016-11-30T18:07:00.000-08:002017-05-10T19:48:57.339-07:00Testing and Analysis of Algorithmic Trading Strategies in MATLAB (Part 2) – Easy-to-use GUI <i>In this post, in continuation of <a href="http://www.matlabtrading.com/2016/11/testing-and-analysis-of-algorithmic.html">Part 1</a>, I will try to describe the most common problems which occur while testing algorithmic trading strategies in MATLAB when using one's own groundwork or the code from the automated trading webinars.</i><br /><br /><br /><h2>Easy-to-use GUI</h2><br />Let's start with the fact that there is no graphical interface because if we presume that almost the whole process of testing and analysing <i>trading strategies</i> is standardised (it is 99%), you would like to have the interface which helps you call up the necessary data and start the testing process with one click.<br /><br />Someone (experienced users of MATLAB, in particular), might argue that the use of ready-made functions is not any worse and actually is even sometimes better and more convenient that the static GUI. It's possible, but a GUI however has a number of advantages:<br /><br /><ol><li>For new (and not only) users of MATLAB it is much more convenient to use a GUI with buttons and entry fields than to search in the code; therefore, there is a GUI even in the MathWorks Toolboxes in most cases because it is more convenient. </li><li>It allows focusing only on the code of your strategy because use of a GUI does not at all imply that it somehow limits your ability to write a strategy.</li></ol><br />Thus, in <a href="http://wfatoolbox.com/?utm_source=post2backtestingproblems&utm_medium=link&utm_campaign=Blog">WFAToolbox</a>, we created a possibility to write any codes for your strategy, <i>using any of MATLAB toolboxes</i> and working with <i>multiple assets</i> for the strategies such as pairs trading, basket trading or triplet arbitrage, etc.; but at the same time this code is easily integrated in the GUI by use of patterns, which are simple enough to apply in the code and they do not limit opportunities.<br /><br />In order to easily master the patterns of code to create your strategies, not only we created detailed <a href="http://wfatoolbox.com/documentation/">WFAToolbox Documentation</a>, but also <a href="http://wfatoolbox.com/video-manual-demo/">WFAToolbox Video Tutorial</a>, which provides an opportunity to a full-scale work with the app in a few minutes.<br /><br /><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><a href="http://wfatoolbox.com/?utm_source=post2backtestingproblems&utm_medium=image&utm_campaign=Blog" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img alt="Easy to use GUI of WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB" border="0" height="392" src="https://1.bp.blogspot.com/-R5AY8RdQJt8/WD-EUAYq8BI/AAAAAAAAAMI/z6qk_kku1z00qGTZwVgKROaHbNh2I-jqQCLcB/s640/image1.gif" title="Easy to use GUI of WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB" width="580" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Easy-to-use GUI of <a href="http://wfatoolbox.com/?utm_source=post2backtestingproblems&utm_medium=link&utm_campaign=Blog">WFAToolbox</a></td></tr></tbody></table><br /><div class="separator" style="clear: both; text-align: center;"></div><br /><br />Anonymoushttp://www.blogger.com/profile/15451548338600496196noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-66695634837908873942016-11-29T00:49:00.000-08:002017-05-10T20:00:39.196-07:00Testing and Analysis of Algorithmic Trading Strategies in MATLAB (Part 1) - Introduction<i>Hello, my name is <a href="https://plus.google.com/105748621316676349879">Igor Volkov</a>, I have been developing algorithmic trading strategies since 2006 and have worked in several hedge funds. In this article, I would like to discuss difficulties arising on the way of MATLAB trading strategies developer during testing and analysis, as well as to offer possible solutions.</i><br /><br />I have been using <a href="https://www.mathworks.com/products/matlab/" rel="nofollow" target="_blank">MATLAB</a> for testing of algorithm strategies since 2007 and I have come to conclusion that this is not only the most convenient research tool, but also the most powerful one because it makes possible using of complex statistical and econometric models, neural networks, machine learning, digital filters, fuzzy logic, etc by adding toolbox. The MATLAB language is quite simple and well documented, so even a non-programmer (like me) can master it.<br /><br /><h2>How It All Started...</h2><br />It was 2008 (if I am not mistaken) when the first webinar on algorithmic trading in MATLAB with Ali Kazaam was released, covering the topic of optimising simple strategies based on technical indicators, etc. in spite of a rather “chaotic” code, tools were interesting enough to use. They served as a starting point for research and enhancement of a testing and analysis model which would allow to use all the power of toolboxes and freedom of MATLAB actions during creation of one's own trade strategies, at the same time it would allow to control the process of testing and the obtained data and their subsequent analysis would choose effective portfolio of robust trading systems.<br /><br />Subsequently, <a href="https://www.mathworks.com/videos/search.html?suggestion=&c%5B%5D=videos_all&q=trading" rel="nofollow" target="_blank">Mathworks webinars</a> have been updated every year and gradually introduced more and more interesting elements. Thus, the first webinar on pairs trading (statistical arbitrage) using the <a href="https://www.mathworks.com/products/econometrics/" rel="nofollow" target="_blank">Econometric Toolbox</a> was held in 2010, although the Toolbox of testing and analysis remained the same.<br /><br />In 2013, <a href="https://www.mathworks.com/products/trading/" rel="nofollow" target="_blank">Trading Toolbox</a> from Mathworks appeared which allowed to connect MATLAB to different brokers for execution of their applications. Although there were automatic solutions for execution of the transactions, from that point MATLAB could be considered a system for developing trading strategies with a full cycle: from data loading to the execution of automated trading strategies.<br /><br /><h2>Why Should Every Algotrader Reinvent the Wheel?</h2><br />However, Mathworks has not offered a complete solution for testing and analysis of the strategies – those codes that you could get out of webinars were the only "elements" of a full system test, and it was necessary to modify them, customise them, and add them to the GUI for ease of use. It was very time consuming, thus posing a question: whatever the strategy was, it must go through the same process of testing and analysis, which would allow it to be classified as stable and usable – <b>so why should every algotrader reinvent the wheel and write his/her own code for proper testing strategies in MATLAB?</b><br /><br /><i>So the decision was made to create a product that would allow to perform the whole process associated with the testing and analysis of algorithmic trading strategies using a simple and user-friendly interface. </i><br /><table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><a href="https://4.bp.blogspot.com/-kwIrm4Q4120/WD0XJbohbUI/AAAAAAAAAJY/9oy0cZEdQhouPUNHxVtgGtZ7MNeXOd7sQCLcB/s1600/GIF-Article-Opt3.gif" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="126" src="https://4.bp.blogspot.com/-kwIrm4Q4120/WD0XJbohbUI/AAAAAAAAAJY/9oy0cZEdQhouPUNHxVtgGtZ7MNeXOd7sQCLcB/s640/GIF-Article-Opt3.gif" width="580" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;margin-top:10px;margin-bottom:10px;"><a href="http://wfatoolbox.com?utm_source=post1intro&utm_medium=link&utm_campaign=Blog">WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB®</a></td></tr></tbody></table><br />We decided to call the solution <a href="http://wfatoolbox.com?utm_source=post1intro&utm_medium=link&utm_campaign=Blog">WFAToolbox - Walk-Forward Analysis Toolbox</a> which demo version has been available on <a href="http://wfatoolbox.com?utm_source=post1intro&utm_medium=link&utm_campaign=Blog">http://wfatoolbox.com</a> since 2013. Anonymoushttp://www.blogger.com/profile/15451548338600496196noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-82313663131977593292016-11-07T17:15:00.000-08:002017-05-10T20:03:28.431-07:00Whoa?! What happened with the blog?Hi,<br /><br />My name is <a href="https://plus.google.com/u/0/105748621316676349879" target="_blank">Igor Volkov</a> and I'm the founder and developer of <a href="http://wfatoolbox.com/?utm_source=post0welcome&utm_medium=link&utm_campaign=Blog">WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB</a>. WFAToolbox is an add-on that is aimed to ease the life of MATLAB users, who develop their own algorithmic trading strategies. This add-on has lots of benefits that are too many to enumerate here, but I will suggest you try it now by downloading its demo version from our official website: <a href="http://wfatoolbox.com/?utm_source=post0welcome&utm_medium=link&utm_campaign=Blog">http://wfatoolbox.com</a>.<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><span style="margin-left: auto; margin-right: auto;"><a href="http://wfatoolbox.com/?utm_source=post0welcome&utm_medium=image&utm_campaign=Blog"><img alt="WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB (Algorithmic Trading)" border="0" height="395" src="https://3.bp.blogspot.com/-Z8t4RGWf9oA/WCEpVGa35RI/AAAAAAAAAGg/L1PVl5ibXbcBrfN1zm8KgSFGmgtfJlZgACLcB/s400/Screen%2BShot%2B2016-11-03%2Bat%2B19.22.05.png" title="WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB (Algorithmic Trading)" width="580" /></a></span></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="http://wfatoolbox.com/?utm_source=post0welcome&utm_medium=link&utm_campaign=Blog">WFAToolbox - Walk-Forward Analysis Toolbox for MATLAB</a></td></tr></tbody></table><br /><br />First of all, I would like to answer the following questions:<br /><br /><h2>What happened with the blog?</h2><br /><h3>1. Jev Kuznetsov is not the owner anymore</h3><br /><div>The blog was purchased from our friend, Jev Kuznetsov, who has moved to his other blog <a href="http://tradingwithpython.blogspot.com/" rel="nofollow" target="_blank">http://tradingwithpython.blogspot.com</a>. He concluded that Python is better than MATLAB for trading, which I considered to be false. MATLAB remains one of the best software in the world for algorithmic trading purposes IMHO (I have some facts about this though for future discussion).</div><br /><h3>2. We have changed the brand</h3><br /><div>From this moment the blog will be called MatlabTrading, which is much more understandable regarding the topics it will include. Furthermore, the domain name has been changed to <a href="http://www.matlabtrading.com/">www.matlabtrading.com</a> instead of the initial <a href="http://matlab-trading.blogspot.com/">matlab-trading.blogspot.com</a>, although the old domain is still working redirecting from the primary domain name.</div><br /><h2>What will happen to the blog?</h2><br /><h3>1. More posts and articles</h3><br />We hope to bring life to this blog by posting relevant contents once or twice a week. In the first few months, we will post mostly those articles and videos that we already have to make it easier for our dear readers to search for information on one resource and have crosslink on them.<br /><br />Then we have plans to write posts about practical aspects of algorithmic trading in MATLAB. How to create modern automatic trading strategies such as:<br /><br /><ul><li>Statistical arbitrage pairs trading / mean reversion / market neutral trading strategies based on cointegration / bollinger bands / kalman filter etc for commodities, stocks and Forex.</li><li>Trend following strategies with <a href="http://www.jurikres.com/catalog1/ms_ama.htm" rel="nofollow" target="_blank">Jurik Moving Average</a> and other sophisticated digital filters;</li><li>Forecasting strategies with machine learning (Support Vector Machines) and other methods;</li><li>Creating robust trading strategies using visual walk-forward testing money management for reinvesting your capital (science on how to get $1M from $10K in a year with maximum, but estimated risk and sweat rewards). Maybe after reading this you've thought this is going to be another dumb article for those poor guys seeking how to become rich through "trading on forex" and all that. Well, that is totally false! We are working in MATLAB, and majority of us are scientists and experts in that aspect so everything is serious.</li></ul><br /><h3>2. More interactivity</h3><br /><div>I will be happy if we can all relate through comments in posts. Subscribe to our news to get alerted about the newest posts and events. Later on, we have plans to make Google Hangouts webinars. <span style="color: red;">Don't miss it, click on "Follow" button at the upper right corner to join our community</span>.<br /><span style="background-color: yellow;"><br />What would you like to read in our blog posts? What topics can you suggest? Please write here in comments.</span></div>Anonymoushttp://www.blogger.com/profile/15451548338600496196noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-82370917142255352782013-01-01T08:21:00.002-08:002013-01-01T09:35:04.727-08:00Intraday mean reversion<div dir="ltr" style="text-align: left;" trbidi="on">In my previous post I came to a conclusion that close-to-close pairs trading is not as profitable today as it used to be before 2010. A reader pointed out that it could be that mean-reverting nature of spreads just shifted towards shorter timescales. I happen to share the same idea, so I decided to test this hypothesis.<br /><br />This time only one pair is tested: 100$ SPY vs -80$ IWM. Backtest is performed on 30-second bar data from 11.2011 to 12.2012.<br />The rules are simple and similar to strategy I tested in the last post:<br /><i>if bar return of the pair exceeds 1 on z-score, trade the next bar</i>.<br />The result looks very pretty:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-32UMbB804eg/UOMeM2HKfXI/AAAAAAAADqQ/bFY-oGYiVC4/s1600/intraday_mr_spy_iwm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="247" src="http://1.bp.blogspot.com/-32UMbB804eg/UOMeM2HKfXI/AAAAAAAADqQ/bFY-oGYiVC4/s320/intraday_mr_spy_iwm.png" width="320" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"></div>I would consider this to be enough proof that there is still plenty of mean-reversion on 30-second scale.<br />If you think that this chart is too good to be true, that is unfortunately indeed the case. No transaction costs or bid-ask spread were taken into account. In fact, I would doubt that there would be any profit left after subtracting all trading costs.<br />Still, this kind of charts is the carrot dangling in front of my nose, keeping me going...</div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com6tag:blogger.com,1999:blog-1346045313195961352.post-31923916791797329492012-12-30T16:31:00.000-08:002013-01-01T08:22:28.940-08:00Is pairs trading dead?<div dir="ltr" style="text-align: left;" trbidi="on">Bad news everybody, according to my calculations, ( which I sincerely hope are incorrect) the classical pairs trading is dead. Some people would strongly disagree, but here is what I found:<br /><br />Let's take a hypothetical strategy that works on a basket of etfs:<br />['SPY','XLY','XLE','XLF','XLI','XLB','XLK','IWM','QQQ','DIA']<br />From these etfs 90 unique pairs can be made. Each pair is constructed as a market-neutral spread.<br /><br />Strategy rules:<br />On each day, for each pair, calculate z-score based on 25-day standard deviation.<br />If z-score > threshold, go short, close next day<br />If z-score < -threshold go long, close next day<br /><br />To keep it all simple, the calculation is done without any capital management (one can have up to 90 pairs in portfolio on each day) . Transaction costs are not taken into account either.<br /><br />To put it simply, this strategy tracks one-day mean reverting nature of market neutral spreads.<br />Here are the results simulated for several thresholds:<br /><br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-Nt7KcYHfvaw/UODaLfzk9kI/AAAAAAAADpQ/8N0PW8055aQ/s1600/mr_death.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="237" src="http://3.bp.blogspot.com/-Nt7KcYHfvaw/UODaLfzk9kI/AAAAAAAADpQ/8N0PW8055aQ/s320/mr_death.png" width="320" /></a></div>No matter what threshold is used, the strategy is highly profitable in 2008, pretty good throuh 2009 and completely worthless from early 2010.<br />This is not the first time I came across this change in mean-reverting behavior in etfs. No matter what I've tried, I had no luck in finding a pairs trading strategy that would work on ETFs past 2010. My conclusion is that these types of simple stat-arb models just don't cut it any more.<!---threshold--><!---threshold--></div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com11tag:blogger.com,1999:blog-1346045313195961352.post-31184623463026623552012-12-30T10:54:00.000-08:002012-12-30T10:54:41.979-08:00Pca - how it really works<div dir="ltr" style="text-align: left;" trbidi="on">I suppose that my previous post did not provide insights on how PCA really works. Here is another try at the subject, using a simple pair as an example.<br />Let's take SPY and IWM, which are highly correlated. If daily returns of IWM are plotted against daily returns of SPY, the relationship is highly linear (see left chart).<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-A_DIb0_NxUU/UOCKSVgXEOI/AAAAAAAADpA/EvklvEnvUG8/s1600/spy_iwm_pca.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="182" src="http://1.bp.blogspot.com/-A_DIb0_NxUU/UOCKSVgXEOI/AAAAAAAADpA/EvklvEnvUG8/s320/spy_iwm_pca.png" width="320" /></a></div>Applying PCA on this data gives two principal component vectors, plotted in red (first) and green (second). These two vectors are orhogonal, with the first one pointing in the direction of highest variance. Transformed data is nothing more than the original data projected on the new coordinate axis formed by these two vectors. The transformed data is shown in the right chart. As you can clearly see, all points are still there, but the dataset is rotated.<br />The second vector is in this case -0.78 SPY + 0.62 IWM which produces a market-neutral spread. Of course the same result would be achieved by using the beta of IWM.<br />The fun thing about PCA is that it is useful in building three- and more legged spreads. The procedure is exactly the same as above, but the transformation is done in a higer dimensional space. </div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com6tag:blogger.com,1999:blog-1346045313195961352.post-39173770176674022522012-12-03T13:51:00.000-08:002012-12-03T13:51:30.332-08:00Using PCA for spread trading<div dir="ltr" style="text-align: left;" trbidi="on">Classical pairs trading usually involves building a pair consisting of two legs, which ideally should be market-neutral or in other words, pair returns should have zero correlation with market returns. The process of building a 'good' pair is pretty standard. A typical way of building a pair (spread) involve choosing two correlated securities and forming a market-neutral pair using stock betas.<br /><br />Multi-legged spreads are more advanced and very difficult to build using the traditional method.<br />However, there is a mathematical method called <a href="http://en.wikipedia.org/wiki/Principal_component_analysis">Principal Component Analysis</a> that can be easily used to create stable (=tradeable?) spreads. All the linear algebra is luckily hidden inside the <i>princomp</i> function, but if you'd like to understand how PCA really works, take a look at this <a href="http://www.cs.bham.ac.uk/~pxt/IDA/PCA.tutorial.pdf">tutorial</a>. The transformed data can be described as : 1-st component: 'max volatility portfolio', which is usually very highly correlated with the market. 2-nd component: 'market-neutral' portfolio, having maximum variance. 3-d and further components have decreasing degrees of variance. Note that by design, PCA produces orthogonal components, meaning that all portfolios are not correlated to each other. So 2nd and further portfolios are market-neutral.<br /><br />Here is an example of applying PCA on some correlated etfs in the energy sector:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-kRujUhINIZU/UL0bSxEddlI/AAAAAAAADj8/GzEYNqC3aaI/s1600/energy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://1.bp.blogspot.com/-kRujUhINIZU/UL0bSxEddlI/AAAAAAAADj8/GzEYNqC3aaI/s320/energy.png" width="320" /></a></div>The upper chart shows raw prices, the lower char are the cumulative returns of principal components. To compute the principal components I only used first 250 days of data. It seems that the principal components, which are linear combinations of each security returns are quite stable out-of-sample, which is a pleasant surprise. First (blue) component has most of the variance, and it is clearly correlated to the movement of the prices in the upper chart.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-EziqDUQ7cKY/UL0d5kThjAI/AAAAAAAADkM/XnqmWTiLof0/s1600/energy_zoom.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://4.bp.blogspot.com/-EziqDUQ7cKY/UL0d5kThjAI/AAAAAAAADkM/XnqmWTiLof0/s320/energy_zoom.png" width="320" /></a></div>Let's take a closer look at the last two components: these seem to be quite stable and tradeable even far out-of-sample.<br /><br /><br /></div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com8tag:blogger.com,1999:blog-1346045313195961352.post-19744980591676671812012-09-27T14:31:00.001-07:002012-09-27T14:31:24.770-07:00Gap strategy with intraday data<div dir="ltr" style="text-align: left;" trbidi="on">The gap fading strategy from previous posts looked all right, but my worry is that Yahoo data does not provide accurate quotes. To check the strategy performance, I've generated a new OHLC dataset based on the Weighted Average Price (wap) of 30-second intraday data. So the opening quote is the wap of first 30 seconds of trading and close is the last 30-second wap. To make sure that my dataset is correct, I have compared it to the yahoo quotes. As shown in the chart below, the difference between the two quotes is ~5ct which seems very reasonable.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-bX9Un1NkXMw/UGTEVdlUzRI/AAAAAAAADZs/ewOTHbRQIQQ/s1600/price_difference.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="237" src="http://4.bp.blogspot.com/-bX9Un1NkXMw/UGTEVdlUzRI/AAAAAAAADZs/ewOTHbRQIQQ/s320/price_difference.png" width="320" /></a></div>Now, testing the gap fade strategy on the OHLC data that I generated myself produces much less favorable result:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-ZaMntVwBEgA/UGTEUpmNZFI/AAAAAAAADZo/p4ifGAcSKcA/s1600/backtest_intraday_data.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="237" src="http://2.bp.blogspot.com/-ZaMntVwBEgA/UGTEUpmNZFI/AAAAAAAADZo/p4ifGAcSKcA/s320/backtest_intraday_data.png" width="320" /></a></div>One look at the pnl chart is enough to say that this strategy would be rubbish.<br />This brings me to a conclusion that I already was aware of: Yahoo opening quotes are not suitable for strategy backtesting.</div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com3tag:blogger.com,1999:blog-1346045313195961352.post-21530300895315560232012-09-20T14:29:00.001-07:002012-09-20T14:29:13.776-07:00Gap strategy revisited<div dir="ltr" style="text-align: left;" trbidi="on">In the beginning of 2011 I've <a href="http://matlab-trading.blogspot.nl/2011/02/closing-gap-strategy.html">backtested a fade gap strategy</a>. There seemed to be an edge to fading gaps, so let's take a look how this strategy performed since then. Once again strategy rules:<br /><div style="text-align: left;"></div><ul style="text-align: left;"><li>Trade only gaps larger than 0.1 %</li><li>Enter on the open (short for Up gap and long for Down gap). Profit target is set at previous day close.</li><li>If profit target was not reached during the day, exit on close</li></ul>This time I corrected the data for dividends.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-zNorFJuWgPE/UFuIz__GWnI/AAAAAAAADZU/usIkuZ6MrBY/s1600/fadeGapsStrat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="229" src="http://1.bp.blogspot.com/-zNorFJuWgPE/UFuIz__GWnI/AAAAAAAADZU/usIkuZ6MrBY/s320/fadeGapsStrat.png" width="320" /></a></div><div style="text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"></div>The results out-of-sample are pretty good, the strategy was doing well in 2011-2012.<br />A more realistic case is including transaction cost of about 0.03% , which is approximately 3ct for SPY. 1ct is IB commission, another two are needed for crossing the bid-ask spread.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-HRJe8WvJz48/UFuHYw4qNWI/AAAAAAAADZM/zue-bNNprcs/s1600/fadeGapsStrat_friction.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="229" src="http://4.bp.blogspot.com/-HRJe8WvJz48/UFuHYw4qNWI/AAAAAAAADZM/zue-bNNprcs/s320/fadeGapsStrat_friction.png" width="320" /></a></div>The Sharpe ratio for these strategies is still not solid enough for me to actually put my money on it.<br /><br />-----Sharpe----<br />buyAndHold 0.189366<br />fadeUpGaps 0.508378<br />fadeDownGaps 0.595578<br />fadeAllGaps 0.783124<br /><div><br /></div><div><br /></div><div>... and I still keep wondering, how can there be an edge while there seems to be no significant correlation between the night gap and the day session change.</div><div><br /></div></div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com1tag:blogger.com,1999:blog-1346045313195961352.post-32737547561354363082012-09-19T15:13:00.000-07:002012-09-19T15:22:59.518-07:00SPY opening gaps<div dir="ltr" style="text-align: left;" trbidi="on">There are quite some people in the blogsphere claiming that gap trading is statistically profitable. Just google for 'opening gaps' or something similar to get a bunch of links. Some claims are quite interesting stating >70% chance of a gap closing after a 'gap up'. Well, I imagine that it is possible to have a 70% 'closed gap' statistics and still have zero edge in trading the gap. I have looked into this topic about a year ago and first results were promising.<br />This time I took a look at the matter in a slightly different way: looking for correlation between overnight change of the SPY (previousClose-to-open) and the daily change (open-to-close) of the following trading session.<br />Below is a chart of cumulative daily percentage changes of SPY for about 3 years of data. Blue line is the overnight change and green line the day session change. It is immediately clear that day session is more volatile then night session. . Apart from that nothing really special in this chart.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-hc8xWulPK-s/UFpALd6VacI/AAAAAAAADYc/dyU1oE0I20M/s1600/spy_gaps_timeseries.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="http://4.bp.blogspot.com/-hc8xWulPK-s/UFpALd6VacI/AAAAAAAADYc/dyU1oE0I20M/s320/spy_gaps_timeseries.png" width="320" /></a></div>More insight comes from plotting the overnight return vs daily return :<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-W8X_7to8BLE/UFpBiuzEV9I/AAAAAAAADYk/UmnFb_7B3x4/s1600/spy_gaps_correlation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://2.bp.blogspot.com/-W8X_7to8BLE/UFpBiuzEV9I/AAAAAAAADYk/UmnFb_7B3x4/s320/spy_gaps_correlation.png" width="320" /></a></div>Judging by eye, there is no relation between nightly change and the daily session. Testing for correlation between the two gives : 0.000062 ... yes, <b>zero. </b>I'm not sure while my previous attempts at building a gap strategy produced positive results, but now I'm determined to get to the bottom of this...</div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com2tag:blogger.com,1999:blog-1346045313195961352.post-60301268354345083782012-03-26T14:18:00.000-07:002012-07-22T14:58:24.953-07:00Hidden cost of XIV<div dir="ltr" style="text-align: left;" trbidi="on">Two popular options to trade short-term volatility are VXX an XIV ETNs. By their design, they should provide exactly the same daily returns, but with opposite sides. Both should have similar yearly fee. However, I have noticed that XIV shows a consistent underperformance relative to VXX. Time for a quantitative investigation.<br />Imagine we would trade a portfolio of long VXX, long XIV and both legs having the same capital, rebalancing at the close. In this case, the daily returns of VXX an XIV should cancel each other out, and cumulative pnl should be flat. But in reality, a long volatility position using short VXX outperforms a long XIV by about 7% a year. Correcting for borrowing cost of VXX, which is about 2.5% , there is still about 5% difference!.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-ewa0FSoA_4A/T3Damq776KI/AAAAAAAADUA/eVF9HvWymUg/s1600/xiv%2Bvs%2Bvxx.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="287" src="http://4.bp.blogspot.com/-ewa0FSoA_4A/T3Damq776KI/AAAAAAAADUA/eVF9HvWymUg/s400/xiv%2Bvs%2Bvxx.png" width="400" /></a></div>The green line represents the path that a long XIV+VXX portfolio have followed since its inception. To make a realistic estimation of the daily relative loss, I have first filtered out the outliers. The red line shows an estimate of the daily cost of XIV relative to VXX.<br />Conclusion: playing short volatility using XIV will cost you about -0.028 % per day, relative to a short VXX position (borrow cost not included).<br /><br />P.S I'm not sure there is money to make by shorting both ETNs, daily rebalancing cost will most probably eat away all potential profits.<br /><br />Update 22/07/2012: here is a chart of an estimated price of XIV by taking a direct daily reverse of VXX. All prices are normalized (starting at 100).<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-IThibYpBcYE/UAx3Lxfv09I/AAAAAAAADUg/S8D8u54Me8Y/s1600/xiv_estimated.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="237" src="http://4.bp.blogspot.com/-IThibYpBcYE/UAx3Lxfv09I/AAAAAAAADUg/S8D8u54Me8Y/s320/xiv_estimated.png" width="320" /></a></div><br /><br /></div>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com2tag:blogger.com,1999:blog-1346045313195961352.post-84028875195444038802012-02-24T15:13:00.002-08:002012-02-24T15:26:24.508-08:00Add database functionality to Matlab with SQLite.Matlab data structures are fine for most of research work, but running a daily trading business is a different thing. A trader or an account manager often needs to maintain a list of trades, accounts, strategies, clients etc., often dealing with <a href="http://en.wikipedia.org/wiki/Relational_database">relational data</a> An SQL database is ideal for such a task, but most solutions (like MySQL) are are quite an overkill, as we don't need a concurrent client database, data trees etc.<br />A solution comes in form <a href="http://www.sqlite.org/">SQLite</a>, a serverless database engine that stores the whole database in a single file. It is widely used in anything from mobile phones to mainframes and runs on almost anything.<br />Quite some time ago I have written a <a href="http://matlab-trading.blogspot.com/2010/10/behold-ideal-database-tooling.html">post </a>on using SQLite in Matlab. As I am still a happy user, time for an update and a demo.<br /><b>Installation:</b> you only need to download and unzip the files from here: <a href="http://mksqlite.berlios.de/mksqlite_eng.html">http://mksqlite.berlios.de/mksqlite_eng.html</a><br />(note: you'll probably need to compile from source on a 64bit system).<br /><br /><b>Demo: </b>I have written a simple script to demonstrate how to keep a list of portfolio positions for thee separate accounts.<br /><br /><br /><pre class="brush:matlabkey">%{<br /><br />Copyright: Jev Kuznetsov <br />License: BSD<br /><br />demo of SQLite for portfolio management.<br /><br />%}<br /><br />% test sqlite<br />clear all;<br />clc;<br />mksqlite('open','test.db');<br /><br />tables = mksqlite('show tables');<br />disp(tables);<br /><br />%% create a new data table <br />mksqlite('DROP TABLE tbl_portfolios');<br />sql = 'CREATE TABLE tbl_portfolios ( id INTEGER PRIMARY KEY AUTOINCREMENT, accountName TEXT, symbol TEXT, position INTEGER)'; <br />mksqlite(sql);<br /><br />%% now add some random data <br />symbols = {'ABC','DEF','GHI','XYZ','AAA','BBB','CCC','DDD'};<br />accounts = {'acct1','acct2','acct3'}; <br /><br />%mksqlite('PRAGMA synchronous=OFF'); % speed tweak, see sqlite doc<br />mksqlite('BEGIN'); % bundle multiple inserts into one transaction, speed boost!<br />tic<br />for i=1:100<br /> symbol = symbols{ceil(length(symbols)*rand)}; % pick a random symbol from symbols<br /> account = accounts{ceil(length(accounts)*rand)}; % same for account<br /> position = ceil(1000*rand); <br /> <br /> fprintf('adding account: %s symbol:%s position:%i \n', symbol,account,position);<br /> <br /> % first, check if symbol is already in portfolio<br /> res = mksqlite(sprintf('SELECT id FROM tbl_portfolios WHERE accountName="%s" AND symbol="%s"',account,symbol));<br /> if isempty(res)<br /> fprintf('Adding symbol \n');<br /> mksqlite(sprintf('INSERT INTO tbl_portfolios (accountName, symbol, position) VALUES ("%s","%s",%i)',account,symbol,position));<br /> else<br /> fprintf('Updating symbol \n');<br /> mksqlite(sprintf('UPDATE tbl_portfolios SET position=%i WHERE id=%i',position,res.id));<br /> end<br /> <br />end<br />%mksqlite('PRAGMA synchronous=NORMAL');<br />mksqlite('END');<br /><br />toc<br /><br />%% now pull the data from database<br /><br />fprintf('\nGetting data from database\n');<br /><br />res = mksqlite('SELECT * FROM tbl_portfolios ORDER BY accountName ASC');<br />fprintf('Account\tSymbol\tposition\n-----------------------\n');<br />for i=1:length(res)<br /> fprintf('%s\t%s\t%i\n', res(i).accountName, res(i).symbol,res(i).position);<br />end<br /> <br />%% try some handy sql stuff<br />% unique account names<br />res= mksqlite('SELECT DISTINCT accountName FROM tbl_portfolios') <br />% sum of all positions in acct1<br />res= mksqlite('SELECT SUM(position) as sm FROM tbl_portfolios WHERE accountName="acct1"') <br /><br /></pre>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com6tag:blogger.com,1999:blog-1346045313195961352.post-55262774195031325682011-09-25T11:45:00.000-07:002011-10-12T09:54:44.781-07:00Getting started with Matlab & financeI am thinking about creating a series of screencasts showing how to use Matlab for financial research. This is the first one, covering the basics and aimed at people not yet familiar with Matlab.<br />The whole series would include (among others) topics like:<br /><br /><ul><li>Getting data from the web (yahoo, google, CBOE etc)</li><li>Aligning and filtering datasets</li><li>Nearest-neighbor classification</li><li>Designing & backtesting strategies</li><li>Interfacing with Interactive Brokers</li><li>Keeping track of strategy performance</li></ul><br /><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/CKhQ5yBRtQE?feature=player_embedded' frameborder='0'></iframe></div>Creating quality material takes a lot of time and effort, so I don't think that I will be able to offer whole material for free.<br />If you are interested in the series please let me know what you think by filling in the poll on the right.<br /><br /><a href="http://quantum.meplaza.nl/01_introduction.zip">Get source files</a>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com6tag:blogger.com,1999:blog-1346045313195961352.post-74701054064323548412011-08-10T14:04:00.000-07:002011-08-10T15:35:32.947-07:00XIV is stealing my money!With current volatility exceeding 40, it seemed like a good idea to short the VIX. Two very popular ways to do this is either to short VXX or go long XIV. I choose to go long XIV a couple of days ago, after making sure that it tracks the inverse daily return of VXX.<br />Now a strange thing has happened that both puzzles me and pisses me off: yesterday there was a tracking error of 2% between the two and today another 1%, both in my disadvantage. It seemed like somebody 'stole' 3% of my position! At this moment I am less than happy with this XIV product.<br />Take a look at this chart:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-qlEKGS80NIk/TkL4H1cS6AI/AAAAAAAADNA/T_KXS2Cv0q4/s1600/xiv_tracking_error.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://1.bp.blogspot.com/-qlEKGS80NIk/TkL4H1cS6AI/AAAAAAAADNA/T_KXS2Cv0q4/s320/xiv_tracking_error.png" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"></div><br /><div class="separator" style="clear: both; text-align: center;"></div>Here I plotted daily returns of VXX and XIV against each other, where the outliers have been plotted in red. Please note that there are another 2 outliers of similar magnitude, occuring on 4 and 5 January 2011, but these two cancel each other out pretty nicely.<br />I thought that both etfs were based on the same SPVXSTR index, but while their intraday path relative to each other is very stable (no arbitrage possibilities), the tracking offset was present throughout the whole day.<br />I understand that XIV could get a positive tracking error because it was banned from short selling on both days (just like VXX), but a negative tracking error is a mystery to me.<br /><br />Can someone shed a light on what is going on here???sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com16tag:blogger.com,1999:blog-1346045313195961352.post-10035544346525174412011-08-04T14:38:00.000-07:002011-08-04T14:38:17.234-07:00Are we in for a rebound?Today we've had a pretty heavy drop in the SPY, that has been preceded by a series of down days. It could be tempting to join the panicking crowd, but the crowd usually gets it wrong.<br />The put/call ratio and the VIX/VXV ratio tell their own story:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-2hO7evD6Jlk/TjsQ2HofjZI/AAAAAAAADMY/eeEBShK4THc/s1600/four_percent_drop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://3.bp.blogspot.com/-2hO7evD6Jlk/TjsQ2HofjZI/AAAAAAAADMY/eeEBShK4THc/s320/four_percent_drop.png" width="320" /></a></div>Both of them are pretty high, the situation that is usually accompanied by a pop in the market.<br />Disclosure: I'm long.sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com1tag:blogger.com,1999:blog-1346045313195961352.post-5521015636620837152011-06-15T14:29:00.000-07:002011-06-15T14:29:18.850-07:00Implied vs Realized volatility premiumThere is a funny thing that I've come across while trying to build a volatility model. According to my estimations the VIX is usually higher than the actual volatility (RV) realized for that same period. Just take a look at the following chart:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-CQ5xFyiKmfU/Tfkf3s7XfeI/AAAAAAAADHI/oEb46Jgc3WE/s1600/impliedVol_vs_realizedVol.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://4.bp.blogspot.com/-CQ5xFyiKmfU/Tfkf3s7XfeI/AAAAAAAADHI/oEb46Jgc3WE/s320/impliedVol_vs_realizedVol.png" width="320" /></a></div>In the top chart I've plotted the VIX (IV) and RV estimate for a 3-year period. The bottom chart is the difference between the IV & RV . To estimate the RV I've used a function from <a href="http://tradingwithmatlab.blogspot.com/">tradingwithmatlab blog</a>. It uses several common estimators for RV (I've used the average value for RV ).<br />It looks like IV on average is 7% higher than the RV. So one could just sell volatility and get a steady profit, right? Knowing that there is no free lunch in trading, it seems that I'm missing out something. Is my estimation of RV incorrect? Is it unjustified to compare VIX and SP500 volatility? All feedback is very welcome...sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com19tag:blogger.com,1999:blog-1346045313195961352.post-11175611327474562002011-05-16T13:30:00.000-07:002011-05-16T13:30:31.295-07:00Guess what, leveraged etfs don't decay!While writing the previous posts and doing the math I still had a feeling that something in my reasoning was not quite right. I could not put a finger on it, but with the help of the author of <a href="http://onlyvix.blogspot.com/">OnlyVix blog</a>, I seem to have figured it all out.<br />Let me start with an old problem of 50/50 chance of winning 1% every timestep. Here is a puzzle from E.Chans blog (and book):<br /><blockquote>"<span class="Apple-style-span" style="color: #333333; font-family: Georgia, serif; font-size: 13px; line-height: 20px;">Here is a little puzzle that may stymie many a professional trader. Suppose a certain stock exhibits a true (geometric) random walk, by which I mean there is a 50-50 chance that the stock is going up 1% or down 1% every minute. If you buy this stock, are you most likely, in the long run, to make money, lose money, or be flat? </span><span class="Apple-style-span" style="color: #333333; font-family: Georgia, serif; line-height: 20px;"><span class="Apple-style-span" style="font-size: 13px;">Most traders will blurt out the answer “Flat!”</span></span></blockquote><br /><span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="color: #333333; line-height: 20px;">...And they would be</span><span class="Apple-style-span" style="color: #333333; line-height: 20px;"> </span><span class="Apple-style-span" style="color: #333333; line-height: 20px;"><b>right. </b></span>To prove this, let's write down a binomial tree for this case. I'll use 10% step to simplify the math:</span><br /><span class="Apple-style-span" style="font-family: inherit;"><br /></span><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-6iYbDklTmeU/TdGDNaFzNbI/AAAAAAAADEY/CnQABgAMO9s/s1600/binomial_tree.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="298" src="http://4.bp.blogspot.com/-6iYbDklTmeU/TdGDNaFzNbI/AAAAAAAADEY/CnQABgAMO9s/s320/binomial_tree.png" width="320" /></a></div><span class="Apple-style-span" style="font-family: inherit;"><br /></span><br /><span class="Apple-style-span" style="font-family: inherit;">Here we start with initial 100$. Every branch has 50% probability. Notice that the expected value at each time step is exactly 100$. However, on the third timestep the most probable value is 99$ .</span><br /><span class="Apple-style-span" style="font-family: inherit;"><br /></span><br />To double-check it, I've run a Monte-Carlo simulation of the above problem. Here is the result:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-JIj3Qq1aDME/TdGBZpGM_MI/AAAAAAAADEU/a3h9oZWe0Dw/s1600/mc_single_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://2.bp.blogspot.com/-JIj3Qq1aDME/TdGBZpGM_MI/AAAAAAAADEU/a3h9oZWe0Dw/s320/mc_single_1.png" width="320" /></a></div>The average value is 0, while the median is -0.5% for 100 steps or -0.005% for one step.<br /><br />The case for 5% change per timestep looks like this:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-XuNmW0lENY4/TdGFm2lRRVI/AAAAAAAADEc/6E6sZTsmAEk/s1600/mc_single_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="230" src="http://1.bp.blogspot.com/-XuNmW0lENY4/TdGFm2lRRVI/AAAAAAAADEc/6E6sZTsmAEk/s320/mc_single_2.png" width="320" /></a></div>The distribution shifts to the left, bu again, average value is zero, and median is -0.1176.<br /><br />So the answer to above puzzle is indeed <b>'flat'</b>.<br /><br />Now returning to a leveraged pair like FAS&FAZ, here is a Monte-Carlo simulation of a leveraged pair:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-ln-lryDXT9E/TdGILd2E2JI/AAAAAAAADEg/WqEF360aCsQ/s1600/3x_pair.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://1.bp.blogspot.com/-ln-lryDXT9E/TdGILd2E2JI/AAAAAAAADEg/WqEF360aCsQ/s320/3x_pair.png" width="320" /></a></div>Here I've used a normal distribution for returns of the underlying with sigma = 1%. Once again, the average return over 100 periods is zero, while most of the occurrences are negative.<br /><span class="Apple-style-span" style="font-family: inherit;"><br /></span><br /><span class="Apple-style-span" style="font-family: inherit;">This means that leveraged etfs don't decay over time, they just <i>look</i> like they do, because that is the most likely outcome.</span><br />So here we go, contrary to common belief, the leveraged etfs don't decay after all!sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com25tag:blogger.com,1999:blog-1346045313195961352.post-78903750979470028112011-05-15T03:53:00.000-07:002011-05-15T03:54:37.023-07:00The problem with shorting leveraged etfsAs I've described in my previous post, inverse etfs decay relative to each other. After looking at their charts it is not difficult to imagine earning 'easy mony' by shorting a pair of leveraged etfs. Max Dama <a href="http://www.maxdama.com/?p=443">has done this</a> and there are more people doing this according to Google, but I think this type of strategy is pretty risky.<br /><br />Here is an example of 'easy money'. I've simulated a random walk (upper chart in blue) that has 50/50% chance of going up or down every day. And it always moves exactly by 3% (log, so it should be flat over the long run ). From this reference I've created two leveraged trackers, with +2x and -2x leverage, I'll call them 'up' and 'down'.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-Bj-WLC6zfHQ/Tc-qpjhsZyI/AAAAAAAADEM/MJcZdvJuLRA/s1600/good_decay.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://4.bp.blogspot.com/-Bj-WLC6zfHQ/Tc-qpjhsZyI/AAAAAAAADEM/MJcZdvJuLRA/s320/good_decay.png" width="320" /></a></div><br /><br />Suppose we start a 100 day period with a pair consisting of equal amounts of capital in 'up' and 'down', both equal to 100$. So the pair value on day 1 is $200. Pair value is plotted in the lower chart. Without a trend in the reference, the pair value decays at a constant rate <i>exp(0.5log(leverage*(1+daily_delta))+0.5log(<span class="Apple-style-span" style="font-style: normal;"><i>leverage*(1-daily_delta)</i></span>))</i>. If we short both 'up' and 'down' the lower chart will flip upside down, producing pretty good looking pnl.<br />But before you short sell every available leveraged etf out there, take a look at the next chart:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-EmDaqYKYMFE/Tc-u3CjODPI/AAAAAAAADEQ/m-QsEZGuc3o/s1600/bad_decay.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="http://2.bp.blogspot.com/-EmDaqYKYMFE/Tc-u3CjODPI/AAAAAAAADEQ/m-QsEZGuc3o/s320/bad_decay.png" width="320" /></a></div>All the parameters here are the same, only the underlying has two brief periods of consecutive wins or losses. This results in two heavy spikes in the pair value. If we would have shorted both 'up' and 'down' the result would be a pretty heavy drawdown. No easy money here...sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com7tag:blogger.com,1999:blog-1346045313195961352.post-83074871244327245212011-05-14T11:25:00.000-07:002011-05-14T12:42:37.965-07:00FAS vs FAZ - inverse etf behaviorThere has been a lot of talk about leveraged etf (under) performance . In general, these etfs seem to underperform their benchmark. A google search for 'leveraged etf decay' will provide a couple of hours worth of reading material, so I will try to limit information redundancy to a minimum. I'll limit myself to a single sentence introduction: 'leveraged and inverse etfs are based on the <i>arithmetic</i> returns of their benchmark, which introduces a negative tracking error'.<br />If you have little idea about what I'm talking about, take a look <a href="http://www.calculatinginvestor.com/2011/04/29/geometric-vs-arithmetic/">here</a> for an explanation of the difference between the arithmetic and geometric returns.<br />So I'll continue the examination of inverse etf dynamics from what is already known: underperformance.<br /><br />Let's first take a look at the relation between FAS and FAZ. Both are 3x leveraged versions of the same underlying index, FAZ being the inverse one.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-0C62OELqBL4/Tc64bM6tb6I/AAAAAAAADD8/Yv_weIKsoYM/s1600/fas-faz_price.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="250" src="http://1.bp.blogspot.com/-0C62OELqBL4/Tc64bM6tb6I/AAAAAAAADD8/Yv_weIKsoYM/s320/fas-faz_price.png" width="320" /></a></div>Here can be seen clearly that while the etfs move in the opposite directions, FAS in the long run outperforms FAZ.<br />Their daily <i>arithmetic </i>returns however are performing exactly as advertised:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-izTUEkgVtSg/Tc65M36XKPI/AAAAAAAADEA/PfYRnVNNVbA/s1600/arithmetic_returns.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="239" src="http://2.bp.blogspot.com/-izTUEkgVtSg/Tc65M36XKPI/AAAAAAAADEA/PfYRnVNNVbA/s320/arithmetic_returns.png" width="320" /></a></div><br />However, anybody holding a position for longer than one time period (being a day) should be only interested in <i>geometric </i>returns, or <i>log</i> returns.<br />When log returns of these two are examined, the picture changes:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-MJ1AiiYmmf8/Tc66aO1Q5-I/AAAAAAAADEE/ItAL-O-cnJk/s1600/geometric_returns.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="239" src="http://1.bp.blogspot.com/-MJ1AiiYmmf8/Tc66aO1Q5-I/AAAAAAAADEE/ItAL-O-cnJk/s320/geometric_returns.png" width="320" /></a></div>Instead of following a straight line, the returns are skewed in favor of FAS. The green line here is a theoretical estimation of inverse relation based on algebraic returns.<br />For example: FAS gains 10% on a given day and FAZ follows with a 10% decline. In log returns this would translate to FAS: <i>log(1.1) = 0.0953 </i>FAZ:<i> log(0.9)=-0.1054. </i> The log returns are not equal (duh!) but skewed in favor of FAS . When the position is held for a longer time and the pair moves 10% every day (no matter in which direction), we loose approx 0.5% per day of the total position.<br />Please take a note that this 'skew' is not about leverage, but inverse algebraic relationship. Leverage only provides more daily movement, exaggerating the skew.<br />A handy chart below shows the under performance of inverse etf as a function of its underlying daily change. One can see that the error is relatively small for <1% moves, but increases rapidly with bigger moves.<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-kgaC0cr50g4/Tc7H0-zQSlI/AAAAAAAADEI/MkBbpGbvkSE/s1600/inverse_loss.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="239" src="http://3.bp.blogspot.com/-kgaC0cr50g4/Tc7H0-zQSlI/AAAAAAAADEI/MkBbpGbvkSE/s320/inverse_loss.png" width="320" /></a></div><br /><br />The difference between geometric and algebraic returns has been explained by E.Chan on his <a href="http://epchan.blogspot.com/2006/10/maximizing-compounded-rate-of-return.html">blog</a> (and in his book) . However, he made a mistake in the calculation of average loss per time period stating it to be -0.5%.<br />When we have a 50/50 chance of winning or loosing 1% , in fact the expected return per minute is exp(0.5*log(1.01)+0.5*log(0.99)), which translates to -.005 % per minute, which is equivalent to -7% in 24 hours ;-).<br /><br />There are a couple of very interesting strategies that can be derived from this asymmetry, if one can handle the math and rebalancing logic.sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com3tag:blogger.com,1999:blog-1346045313195961352.post-53998478956015163182011-05-11T13:51:00.000-07:002011-05-13T13:22:49.784-07:00In trading no one should ever be absolutely sure...and the one who is will get punished sooner or later. This has happened countless times, to traders of all skill levels. Once you think you've got a trade on your hands that can't possibly go wrong, you're brewing a recipe for disaster. Everybody knows this, me included, however I just barely escaped such a situation by plain luck.<br /><br />A couple of days ago I ran my spread scanner and the good old GLD-GDX spread caught my eye. It was so streched up, that I decided to trade it. In fact it was at an extreme that has not happened for a very verly long time. This spread is a 'classic' for spread traders and it has been mean reverting for years now. I'm was sure that it would mean revert again, and quickly got on board. The only thing that prevented me from betting big was the shortage of free cash and I did not want to close any other positions. So I ended up with a usual size of a spread bet, that I don't allow to produce more than 2% of portfolio volatility.<br />What happened next is a short story. The spread moved further against me, knocking 2% off my portfolio. A pity, but not a disaster. <br />The good part is that I can still handle this situation with calmness and make reasonable decisions about keeping this spread or taking a loss. Things could be different if my position was larger and I lost months worth of work on a single trade.<br />This reminded me again that trading should not be convinced with gambling or one will end up the as >90% of amateur day traders- with a blown account, but more on this later...sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com0tag:blogger.com,1999:blog-1346045313195961352.post-46500392499906314342011-04-16T09:34:00.000-07:002011-04-16T09:34:48.722-07:00When Sharpe is uselessDuring development of an intrady strategy I've come across a case where Sharpe ratio (and Sortino too) is a very bad way to benchmark performance. To illustrate this case, imagine a strategy that enters a position and 'brackets' the exit levels with a limit and a stop-loss order. This results in the pnl distribution as shown in the graph:<br /><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-UwEMIPfGbng/TanETBdVVXI/AAAAAAAADDw/Kx300Htpcpo/s1600/deltas.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="220" src="http://3.bp.blogspot.com/-UwEMIPfGbng/TanETBdVVXI/AAAAAAAADDw/Kx300Htpcpo/s320/deltas.png" width="320" /></a></div>The position is either closed at the limit or the stop-loss level, but not in between. Because Sharpe is all about normal distribution, it just does does not work for this case. A better solution here is to use win %, especially in the case of symmetric brackets.sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com4tag:blogger.com,1999:blog-1346045313195961352.post-55608802803050109412011-03-03T13:50:00.000-08:002011-03-03T15:00:41.576-08:00HOWTO: Signal filtering, the better way.It is hard to find a trading system that does not employ signal filtering through use of <i>sma, ema, wma</i> and the likes. I'm not going to dive in detail on the inner workings of these filters as this info is widely <a href="http://en.wikipedia.org/wiki/Moving_average">available</a>.<br />Filters that prioritize recent data are more suitable taking market dynamics into account but implementing these filters using a <i>for</i> loop in Matlab is anything but efficient. Luckily, Matlab provides the <a href="http://www.mathworks.com/help/toolbox/signal/filter.html">filter </a>function, that can be used to calculate all these types of moving averages and even more.<br />In the code attached to this post I'm going to show how to filter signals based on an excellent <a href="http://quantum.meplaza.nl/Ehlers%20(Hybrid%20FIR%20IRR%20Filters).doc">article written by John F.Ehlers</a>. (more articles can be found <a href="http://www.mesasoftware.com/technicalpapers.htm">here</a>) Please read it before playing with the code. I've implemented the different filters mentioned to compare their characteristics.<br /><div class="separator" style="clear: both; text-align: center;"><a href="https://lh5.googleusercontent.com/-qjVL0HCOdrU/TXALsNKoXuI/AAAAAAAADDQ/CF_6t7AJMs8/s1600/filter_application.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://lh5.googleusercontent.com/-qjVL0HCOdrU/TXALsNKoXuI/AAAAAAAADDQ/CF_6t7AJMs8/s320/filter_application.png" width="320" /></a></div> Looking at the data, I must draw a conclusion that the hybrid filter provides an improvement in terms of lag over the standard ema.<br />Using this hybrid filter one can create a simple indicator of the current price imbalance, like shown in the next graph. For this I've used the difference between the current price and <i><a href="http://quantum.meplaza.nl/emaHybrid.m">emaHibrid(price,0.85,0.3)</a>. </i><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://lh5.googleusercontent.com/-lP1f1UMWLzs/TXAaRNooqcI/AAAAAAAADDU/EGz4a3ioArs/s1600/hybrid_oscillator.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="230" src="https://lh5.googleusercontent.com/-lP1f1UMWLzs/TXAaRNooqcI/AAAAAAAADDU/EGz4a3ioArs/s320/hybrid_oscillator.png" width="320" /></a></div><br /><br />Source: <a href="http://quantum.meplaza.nl/demoFilter.m">demoFilter.m</a>, <a href="http://quantum.meplaza.nl/emaHybrid.m">emaHybrid.m</a> data: <a href="http://quantum.meplaza.nl/price.mat">price.mat</a>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com5tag:blogger.com,1999:blog-1346045313195961352.post-769435830063849552011-02-28T10:21:00.000-08:002011-03-03T13:55:58.770-08:00Gaps, part 3Recently I've come to realise that I've made quite a mistake calculating the 'fill' ratio for the gaps. It's time to correct it, before somebody will embarrass me in the comments ;-).The problem is that I've used <i>asymmetric</i> boundaries when calculating the statistics. By doing this, the statistics shift in favor of the nearest stop. The closer the stop, the higher the chances of hitting it, but by no means does this mean profit. And previous close is usually not far from the open so chances of hitting it are quite high.<br />To make statistics fair, I've set the levels <i>symmetric</i> around the open, like this:<br /><i>gap = open-prevClose</i><br /><i>winLevel = open+gap</i><br /><i>lossLevel = open-gap</i><br /><br />With this calculation* the chances are quite different and closer to my results using the intraday data:<br />down gap: 52% fill<br />up gap: 48% fill<br /><br />*geek note: when both win and loss levels are reached for the same day, it is counted both as win and loss (with daily ohlc data there is no way of knowing which one was reached first) . Data for SPY Feb.2000- Feb-2011.<br /><br />Code: <a href="http://quantum.meplaza.nl/dumbGaps.m">dumbGaps.m</a>sjevhttp://www.blogger.com/profile/17452562180989360928noreply@blogger.com2