Quantopian Blog

Paper Trading and Live Trading

2013-05-20T14:54:47Z

We announced live trading and paper trading earlier today (get started by cloning the sample algo). The response has been great. I particularly loved this quote from Pandodaily:

This seamless integration is key, and also rare in the industry, with most institutional-grade products demanding that algorithms be rewritten between the paper-trading and live trading environments. In addition to being an overall pain in the ass, this structure also introduces additional technical risk in that it’s easy to make an error in rewriting the algorithms. Quantopian has bridged the gap between idea and execution better than any consumer-facing product that I’m aware of.

Paper Trading: You can now run an algorithm in “paper trading” mode. Paper trading is a big step for anyone who is getting ready to live trade. There’s no way to “overfit” an algorithm that is in paper trading mode – this is an out-of-sample test that simply can’t be beaten. You can fool yourself by just backtesting over and over again. But you can’t fool yourself while doing paper trading.

Paper trading runs off of a live data feed (actually 15-minute delay, but effectively live). Quantopian bundles the live feed into minute bars, and pushes those bars into your algorithm. Your algorithm issues any orders it thinks it should, and the Quantopian backtester sends back any order fills. You get minute-to-minute updates on your returns, risk metrics, etc. You get to watch the results live.

The data feed for this is new to Quantopian. We’re using NxCore from Nanex. This feed is interesting because it hasn’t been completely cleaned. As a realtime feed, it includes mis-entered data and other anomalies. We may need to write defensive code in our algorithms to help identify and discard bad prints.

Paper trading is free. To start, you first have to have an algorithm that you ran a full backtest with. Then click the “live trading” button and you’re off!

Live Trading: We’re ready to start live trading with a pilot group of traders. If you’re interested in live trading, make sure you’re logged into your Quantopian account and then sign up for the pilot program. We’ll select new additions to our pilot group from the pool of applicants.

Live trading isn’t robust enough yet for general use. When you have a bug in paper trading, it’s frustrating, but the stakes are low. When you have a bug in live trading, the stakes are much higher. To mitigate that we’re doing a lot of hand-holding with the pilot group and working with them to launch their algos. I tell you – it’s very exciting to see orders sent to Interactive Broker and come back as trades in Quantopian!

Thomas Wiecki on Science & Python

2013-05-02T17:21:35Z

Thomas is the only one on our team with a pure-science background. We took a few minutes to ask him about Python usage in science.

Do people in the scientific community actually use Python?

More and more scientists are learning and using Python on a regular basis. Python is a programming language that fits your brain, so it’s reasonably simple for people to begin using. A lot of scientists already use numerical programming languages like MATLAB. However, while these tools make it very easy to do data analysis, general purpose programming paradigms (like object oriented programming) were often only added as an afterthought and are more cumbersome. Python, on the other hand, is a general purpose programming language first which then has been extended to allow scientific data analysis.

Are there specific scientific areas that translate particularly well to Python use?

Python is gaining a lot of ground in neuroscience and neuroinformatics. There exist quite a few high quality packages for data collection and analysis, parameter search and optimization, stimulus generation and more. Other areas like bioinformatics and astronomy are also embracing Python.

What do you think is needed to encourage more scientists to try their hand at Python?

Teaching programming needs to be better integrated into general science education and Python is a great place to start, for the above mentioned reasons. Software Carpentry is an organization that frequently runs bootcamps across the world to help scientists learn about computation and programming. There are also some great groups like PyLadies that are working to get more women excited about programming in Python.

However, setting up a Python environment and knowing which packages to use for which purpose is still a weak spot currently. Python distributions like the Enthought Python Distribution or Anaconda are starting to improve on this.

As a scientist, what is appealing about Python?

The excellent IPython package offers a web-based interactive notebook which allows you to do you data analysis and intersperse that with text describing each step. These notebooks thus contain all relevant information in a single file and really allow everyone to go step by step through the analysis and see how they were generated. It makes the sharing of data and analysis much, much simpler.

Making data analysis more transparent and thus easier to reproduce is a very timely topic. Scientific research is becoming increasingly sophisticated. However, a standard research paper often does not contain enough information to reproduce the results just based on the methods section – or the research relies on complex computer software that is often not published together with the paper.

This issue of reproducability also extends to algorithmic trading research where there are many articles describing strategies that seem to work very well on paper. But without access to the code and data it is very difficult to convince yourself of their validity. On Quantopian, our community has started to implement some of these strategies to try to reproduce their results. Ultimately, it would be amazing if the authors would implement their strategies themselves on Quantopian when they publish a paper on a trading strategy. This would make it very easy for everyone to reproduce the results but also extend the algorithm with one’s own ideas!

We Are A Boston Company

2013-04-16T18:09:14Z

At Quantopian, we’re all feeling the emotional pain caused by yesterday’s attack. Thankfully, none of our friends and family were injured. But Boston is home for us, both physically and mentally, and this touches us all. Our hearts go out to everyone, from runners to victims to volunteers to our public safety professionals. May we all recover quickly.

We think that one of the best things we can do, that anyone can do, is to just keep on going. Terrorists can shock us, but they can’t stop us. We’re all here in our office in Downtown Crossing, working to build a product that will change the world.

Boston doesn’t stop being Boston.

public domain image via Wikipedia

We’ve Been Selected to Demo at FinovateSpring 2013

2013-04-16T13:12:38Z

We’re so excited that we’ve been selected to demo at the world’s top financial technology event – FinovateSpring 2013.

More than 60 hot companies, including us, will make the trip to San Francisco to demo their latest innovations to audiences in just seven minutes. Fawce and Jean will take to the stage to reveal a big announcement, live on May 14-15.

FinovateSpring is a two-day showcase that highlights the best innovations in financial and banking technology from a mixture of established companies and brand-new startups. Finovate selects startups to demo new technology based on the following criteria: novelty, innovativeness, potential, wow factor and diversity.

We’re humbled that Finovate has chosen us to participate in their show, and we’re looking forward to working with the Finovate folks and seeing all the other cool technologies.

Be sure to follow @quantopian and our blog for continued updates leading up to and during the show. It’s going to be an exciting few weeks!

Parameter Optimization with Zipline, PiCloud, StarCluster, and IPython Parallel

2013-04-11T21:54:50Z

Financial trading algorithms often have parameters you have to set manually. For example, the classic dual moving average cross-over has two free parameters: the number of days for the short moving average and the number of days for the long moving average. Once the short mavg crosses the long mavg, you trigger a buy order of the stock.

Using Quantopian, you can explore how the two parameters interact with each other. Open the Quantopian example in another browser, and click the Clone button (no account required). The short window is defined in line 7 of the code, and the long window is in line 8. Change them to to 5 and 10 and click build to see the result. Change the windows again to 1 and 15 and click Build to see the result. As you can see, the results of your choices don’t follow any obvious intuition.

To get the most bang for your buck you want to set these parameters to a value that optimizes some objective function. For example, you might want to find the parameter value that maximizes cumulative wealth, or minimizes the risk you are taking on. Fortunately for us, this is a very well studied domain known as optimization. Unfortunately for us, most devised algorithms are not applicable to our problem.

Why do classical optimization algorithms like gradient descent or convex optimization not cut it? The problem is that these optimization algorithms often have strict requirements on the objective function. As the name implies, gradient descent requires you to be able to compute a gradient. Convex optimization requires the objective function to be, well, convex. However, the relationship between how parameters of trading algorithms influence the objective function can be much more complex. If the parameter is discrete like the number of days in a moving average we certainly won’t be able to compute a gradient. In addition to these problems, running a trading algorithm on a large data set can take quite a long time. Thus, any sequential optimization routine is out of the game. What instead is required is a global method that can be run in parallel.

Running Zipline in parallel in the cloud

Zipline is our open-source backtester. While our ultimate goal is to make optimziation available on Quantopian there is some more research needed to make this happen. However, for the adventurous among you, here are some instructions on how to explore parameter ranges of an algorithm in parallel in the cloud:

1. PiCloud

PiCloud is a cloud computing service that makes it very easy to set up a cluster in the cloud and distribute work to it. For your convenience, we created an official zipline environment (called “/quantopian/zipline_official”) that you can use to run your algorithm. An example IPython Notebook that demonstrates how to do this can be found here.

2. StarCluster + IPython Parallel

StarCluster is a Python package that greatly simplifies the process of setting up a cluster on Amazon EC2. A plugin allows you to run an IPython Parallel cluster on top of your EC2 nodes. You can then distribute your jobs to this cluster from your local IPython session. I recently gave a talk about this at PyData SC’13. Here are the accompanying slides, IPython Notebook, and video.

Zipline in the Cloud: Optimizing Financial Trading Algorithms from PyData on Vimeo.

Optimization algorithms

Above I outlined how one can run zipline in parallel to explore the parameter space over equally spaced points — this is generally known as grid-search. However, in most cases searching the whole parameter space is incredibly wasteful. Fortunately, this is an active area of research in Machine Learning called hyperparameter optimization from which we can borrow.

At first I have been looking at genetic algorithms and particle swarm optimization as they are heuristic and require few assumptions about the parameter landscape. Lately, however, I am more closely looking at Bayesian Optimization using Gaussian Processes to find points that most likely will lead to an improvement given some regularity assumptions on the objective function (see the image below). The paper describing this idea was presented at NIPS 2012. There’s even an open-source toolbox that implements this method (see also this blog post by Michael Hughes). Finally, you might also be interested in hyperopt, a new Python module to define and search a hyper-parameter space.

I will discuss these methods and some extended ideas in more detail in a future blog post. If you enjoyed this blog post, follow me and quantopian on twitter, and read the first installment on Walk-Forward optimization.

Algorithm Development Competition with Upgrade Capital

2013-04-08T15:14:50Z

It’s a problem — students interested in exploring career opportunities in quantitative finance often do not know where to start. It’s difficult to gain access to recent non-academic quantitative research, and doing hands-on work requires significant investment of time, money — or both.

At Quantopian, we encourage people to expand their understanding of quantitative finance by providing free, accessible educational resources, as well as a community of peers and experts. Take a look at our community and you’ll see plenty of examples of collaboration and people tackling challenges together. This commitment to education is the motivation behind a partnership with Upgrade Capital to enable all students with an interest in quantitative finance to gain hands-on experience in the field through an algorithm development competition. The competition will also help the industry identify exceptional young quants on the basis of their work rather than their resumes.

Our platform is an excellent starting point for exploring the practical applications of quantitative finance, because it allows anyone with a working knowledge of Python to dive right into the source code for existing trading algorithms and begin tweaking. Upgrade Capital caters to students with pre-existing code libraries or making use of data inaccessible through the Quantopian platform. During the out-of-sample testing period (May 6th through June 7th), these students will be asked to submit their orders to Upgrade through a proprietary virtual portfolio management API.

So here are the details on the competition:

Restrictions

Students are allowed to trade all US exchange-listed stocks, ADRs, and ETFs. Shorting is allowed but leverage is not.

Trade volumes for orders submitted through the Upgrade Capital API are limited on the basis of available liquidity; you can read more about Upgrade’s platform here.

Timeframe

I. Development

Students will have until 11:59pm PST on May 5th to develop and backtest a trading algorithm. This work can either be done on Quantopian’s platform or offline.

Each competition participant is required to submit a document explaining their algorithm and its performance in backtests no later than 11:59pm PST on May 12th.

II. Testing

Algorithm performance from May 6th until market close on June 7th will be used as an out-of-sample test. Competition participants may submit a supplementary note explaining performance during this period but all documentation needs to be received no later than 11:59pm PST on June 16th.

III. Evaluation

As no single metric provides a comprehensive understanding of performance, winners will be selected by a panel of traders and risk management experts on the basis of their write-ups as well multi-dimensional analyses of each competition participant’s performance.

The names of competition winners will be announced on June 21^st. The winners will then be publicly profiled through both Upgrade Capital and Quantopian.

How to Sign Up

Registration for the Spring 2013 Algorithmic Trading Competition ends at 11:59pm PST on April 14th. If you would like to participate, please sign up here.

Short Interest Data Analysis from Jessica Stauth

2013-04-04T21:53:15Z

We recently hosted Meetups in Boston and New York that featured Jessica Stauth, PhD, Director of Quant Product Strategy for Thomson Reuters. Jess has some pretty impressive credentials – at Thomson Reuters she works to provide data and analytics tools to quantitative investment professionals. She was working at StarMine Corp. as a quant research analyst when it was acquired by Thomson Reuters.

At the meetups, Jess spoke about finding alpha in short interest data. With the commoditization of such basic quant factors as value and momentum, in recent years systematic investors have turned more and more to sentiment based alpha signals. Aggregated open short interest level provides a profitable, low turnover signal rooted in buy-side sentiment, aka “the smart money.” During her presentation, Jess discussed the basics of short selling and data availability, the proprietary formulation of the StarMine short interest model, and a range of sample trading strategies. It was packed with a lot of useful informational – you can view her slides below, and if you’d like to learn more there is also a Thomson Reuters whitepaper with further information.

You can also take a look at her signal as implemented on Quantopian. Jess worked with us to implement it. It’s not the full signal that Thomson Reuters provides, but the result is nonetheless very interesting. What’s particularly neat is that you can use Fetcher to get updated short interest. Check out the algorithm, and click “clone” to make your own copy to test with.

As always, feel free to let us know if you have any questions. We’re here to provide a helpful and positive learning community for all quants.

Algorithmic Finance Meetup: Starmine Short Interest Talk from Quantopian

Video: Wes McKinney, Algorithmic Finance Meetup

2013-03-25T21:27:48Z

A few weeks ago we held an algorithmic trading meetup in Boston where Wes McKinney spoke to a jam-packed room of quants eager to learn more about pandas, Python and algorithmic trading. For those of you who aren’t familiar with python star Wes McKinney, he is the co-founder of Lambda Foundry, creator of pandas, author of “Python for Data Analysis,” and our newest advisor.

At the meet up Wes spoke for an hour about his career, the benefits of using pandas for algorithmic trading and performed several live demonstrations. It was a great opportunity to see a master developer in action, and we’re posting a video of his presentation here as a resource:

You can download Wes’s iPython notebook here.

Our goal for these events is to speak on topics that are both interesting and educational to the quant community, so we welcome any suggestions on what you would like to see covered in the future!

Quantopian Reaches Users Around the World

2013-03-22T18:34:46Z

We recently pulled this cool image from Geckoboard, highlighting a 24-hour span of Quantopian backtests. The dots correspond to cities from which users logged into Quantopian, cloned/created/modified an algorithm, and ran a backtest. The brightness indicates the number of users from that city in the same 24-hour time period. Check out the world of Quantopian!

How Our Meetup Got Hijacked

2013-03-20T00:28:46Z

I learned two big lessons about Meetup last week. First: check your group settings carefully. Second: sometimes you have to “be cruel to be kind.”

We’ve been working hard to grow and nurture Meetups centering on algorithmic finance, in New York City and Boston in particular. We’re committed to developing a community of smart, interested individuals, and the mix of the online community with in-person meetups has been working well.

In late February we discovered that someone not affiliated with Quantopian had created an event in our NYC group. I was surprised, and a bit taken aback. I considered deleting the event entirely, but I also didn’t want to discourage community participation. “If someone wants to help the community, then I should let it happen,” I convinced myself.

Unfortunately, the event in question turned out to be operating under a false flag. This was not someone trying to benefit the community, it was someone promoting an MLM scheme – clearly not something Quantopian would support! The meetup was terrible – a long, boring sales pitch, with none of the learning and personal interaction that we’d been working so hard to foster.

I’ve since changed the group settings so that this sort of thing doesn’t happen again. In Meetup.com under Group Tools there is a listing for Group Settings, and one of the settings makes new events “organizer-only.” Now, only I can create events within the group. We still encourage community participation – my plan is to work with anyone who wishes to help advance quantitative thinking and algorithmic trading. But I need to do it while ensuring that the content is high quality and beneficial to the community.

In retrospect, I should have just cancelled the event right away. I gave the person who posted it the benefit of the doubt, but I should have trusted my instincts and been more aggressive in managing the group. I didn’t want to be a “corporate suit,” and I was afraid that I would be seen that way. I now realize I was thinking like a community steward, and I should have listened to my instincts.