Milk Trader

Takes Two To Tango

2012-04-17T16:23:00.001-07:00

Yoga on the guitar:

Nasdaq 100 going to 5,500 (in 14 months)

2012-03-28T19:53:00.000-07:00

Alright, it's a bold statement. And as you will see below, it's based on the premise that this is an interation, or a a "quote" -- recurrence of 1999. But that's what it is. Let's start with exhibit A:

Wow, that's a wicked weekly chart. Clearly oversold, right. Well that was 1999. Let's see where we are now. Stage hands: Exhibit B!

This is where we are now. Not to say history repeats itself, but if it ever did what do you suppose would happen next? Well if you are guessing a great meltdown, think again. Stage hands, please present Exhibit C!

Conclusion: We have room to go.

Mission Aborted

2012-03-13T14:15:00.000-07:00

They say that markets can remain solvent longer than short sellers can maintain their sanity, and I think this holds true for our current situation. I'm aborting the short bias. I'm also re-assessing whether I will trade equities ever again, but that's a different story. This market is truly unreal. It's on the fat tail distribution of all markets that have come before it, and it continues to stretch the tail.

What started out as a great year with some killer trades in energy and the euro have been vaporized by trying to short sell this bull market. I was able to dodge the rippers for the most part from 1308, but today's abnormal 1.8% rally has taken me out.

I don't doubt that this is a market will pull back, but I can't risk the account given the circumstances. I'm still short a little ES from 1362 and quite frankly, I'll be happy to get my money back.

Screencast: The Making of 17018d5488

2012-03-06T05:28:00.008-08:00

The following screencast demonstrates the use of R, the quantmod R package and bash to process SPX data from 1950. An explanation on how to access a git repository that includes the plots and the R console history is also provided. This screencast produced the data and the plot for the 17018d5488 post on algorithmzoo.com

17018d5488.mp4 from milktrader on Vimeo.

Alternate site:

Part 1/2

Part 2/2

The Unbearable Being of Lightness

2012-02-17T20:48:00.000-08:00

Alright, let's get serious for a moment. Check that, let's "quote" -- not get serious for a moment. Let's instead pretend. That things that are bad are contained by wise sage-like men and women. That there is always an accounting technique that explains why things don't add up. That bull markets never behave badly.

Those of us who prefer listening to Beethoven's Last Quartets are being assaulted by Cindy Lauper melodies of the banal. While prancing optimists make money every other breath, plodding realists are forced to cover or hedge or both. It's maddening when dumb people make money at the expense of smart people.

But all the smart ones losing money lately have learned one thing that will be lost on the Pollyanna. How to survive when the market goes against you. You see, realists are quite accustomed to a slow unwinding. Trivial optimists have no idea what it feels like to have things not play out according to their charts, and to the levels that have been foretold by a 10th century mathematician.

For now, bulls are right. Bears are wrong. Those who claim to be neither are either scalpers (which there's nothing wrong with that) or delusional. It's almost too much to withstand the preening of long-term agnostics.

My name is Milk and I am short this market

2012-02-08T14:26:00.001-08:00

There aren't many of us left. We've lost many to attrition and a few to defection. Our group grows smaller with each passing day of higher highs and higher lows. We are short this market and the object of scorn and ridicule. We are being kicked around like degenerates addicted to pain. Not only have we suffered damage to our trading portfolios, but we have to deal with intervention attempts from those know better than us. They know this is a bull market and that means you should not be short. You should be long.

Some of us are bears, other bulls and still others indifferent. Some of us see a "quote" -- healthy pullback. Others see us going to test 1160 and below. A few see us going to zero (not many, mind you). In any case, it's difficult to have a meaningful conversation with the trend-is-your-frienders.

We point out that "we're a bit stretched here as we sit about 6% above the 50-day moving average."

"Nope, doesn't matter. You're just an unhappy vagrant who is suppressing anger because you missed the move."

"But we are getting a little frothy up here and people who have no business trading are making money."

"Ah, so you're not only bitter, but you're jealous."

"The european crisis is far from resolved, can't you see the chicanery going on there?"

"Doesn't matter, already priced in."

"Nobody is buying puts!"

"Nobody? Obviously some people are. Now you're exaggerating"

And so on. In the end, everyone trades what they see. Not everyone sees the same thing.

Be careful fellow contrarians. If holders of near zero-interest rate bonds get coaxed back into this market it can get very bad for us. Once that happens, it will be lights out for any and all skeptics (at least those with short positions).

My name is Milk and I am short this market.

batty natty

2012-01-24T07:50:00.001-08:00

Rumors of a hedgefund blowing up and frantically selling out a long position. Mild winter weather across the country. Oversupply. Calls for prices to reach $1 and below. All the melodrama you could ask for is playing out in natural gas.

I glibly declared the sell-off ridiculous and went long the Feb contract of the NG future a few days back. No stops, because I was confident. When I woke in the morning I was awash in a 50+ tick profit and asked my 7-year old if I should take the $500 or not. He said 'take it Dad.' Of course I didn't. I judiciously placed a stop at break-even though. It took all of about an hour for my stop to get hit, and for 50+ ticks evaporate into a 1 tick profit.

I can say I felt annoyed. But also amused that something like this could happen so suddenly. I got back in long that same morning and after a meager 10 tick profit decided to place a stop at break even again. And again, it was hit in short order. I'm thinking, hmmm. This is curious indeed. One more trade for the morning. This time it goes against me right away for about 15 ticks. Wow, what a fine mess I've gotten into I think to myself. I take a breath and wait. It comes back to break even. It catapults to plus 16 ticks in what seems like seconds and I get out. Phew. It was fun. My 7-year old was unimpressed though that I blew a $500 profit. This is toy money (as in money for toys) we're talking about here!

Next day I'm at work and not trading. That's when the real rally happens of course, when I'm flat. Did I miss making over $4,000? No, not really. That's just not the way I see things. I get long that evening and repeat my previous mistakes. With my position positive 35+ ticks by morning, I set a stop at break-even. Where do you suppose it ends up going in the next couple hours? Yep, out for a 1-tick profit.

Mental note to myself. Take 30 ticks when you can. Next, I go long again about 20 ticks from where I was stopped out and push aside the feeble thoughts of ... 'if you didn't get stopped out, you'd be up 20 now ...'. Yeah sure. Whatever. I'm long now. It takes but a few minutes and I'm up 20 ticks again. Stop at 10 ticks. Up 30 ticks, stop at 20. And so on until I have a locked in profit of 40 ticks. It gets filled and I'm done for now.

When my 7-year-old comes home from school today, I'll have a better story to tell. I have no doubt he'll expect some free money for Hero Factory toys. I'll probably give it to him, along with his brother. You can never have too many legos in the house.

eurochatter

2012-01-18T11:06:00.001-08:00

Things have been quite at the zoo recently so I've been assigned a project to capture chatter amongst the resident creatures. I'm taking a very learn-as-you-go approach to this challenge and so far have managed to create a working app that captures the twitter stream chattering about the eur/usd currency pair. You can see my progress at www.eurochatter.com but be fore-warned that it's a work in progress. Once you start seeing ads you'll know that I've solved most of the programming issues.

Here is a screenshot of what it looks like today:

Of course, this will not likely be its final styling format, but it's a start. The chart is generated by an open-source javascript library and has some cool features including the ability to get the values of various datapoints by mousing over the chart. Also, you can zoom in. Reloading the page brings it back to its original state.

The programming behind the app is a Ruby. It uses the Sinatra gem to serve the web page and it's currently being hosted at Heroku. I'm also using a Mongo database in the cloud, courtesy of MongoLab.

I still have a substantial hurdle to overcome in this app with regards to managing the database. I'm currently using a capped collection to store all tweets. I query the database as to its total and that's how the data is generated for the chart. But, as you can imagine, a capped collection reaches a limit at which point its total doesn't change. Only the contents does. I'm thinking that in the code block where I insert records into the database I can increment something there to avoid this limit problem, but I'm not sure.

Another problem will be that when I push changes up to the hosting company, the record of timestamped tweet totals will be erased. This is because the tweets.csv is being updated on Heroku servers. Meh, just another problem to solve. Kinda like a crossword.

I hope you enjoy the site. And remember to click on the ads after I get them up. I promise not to put up too many.

A Simple R Script for Traders

2011-11-21T08:36:00.001-08:00

Now that we've got the Python implementation under out belts, let's do the same thing with R. We'll still be able to pass command-line arguments to get a quick look at what our stock of interest is doing during the day. And we start with the familiar incantation of enabling our script to run from command-line.

$ chmod +x yahoo.r

This is a further abuse of starting the story in the middle. So let's get to the beginning. Here is the R script that we'll be invoking from command-line.

#!/usr/bin/Rscript
require(quantmod)
arg <- commandArgs(trailingOnly = TRUE)
stock <- getQuote(arg)
last <- stock[2]
high <- stock[6]
low <- stock[7]
pctile <- (last-low)/(high-low)
bingo <- round(pctile * 100)
b <- as.integer(bingo)
sprintf("%s is trading at the %ith percentile of its daily range", arg ,b)

We only need to call one package here, the inimitable quantmod. That's were we get access to the getQuote function. From there, it's basic boilerplate. I am by no means a professional programmer, so there are some issues with this script that I'm just living with for now including the fact that when this is invoked, R just doesn't shut up with it's messages. Let's run this on ^GSPC, which is the Yahoo symbol for S&P 500. You'll see what I mean.

$ ./yahoo.r ^GSPC
WARNING: ignoring environment value of R_HOME
Loading required package: quantmod
Loading required package: Defaults
Loading required package: xts
Loading required package: zoo
Loading required package: TTR
Loading required package: methods
[1] "GSPC is trading at the 3th percentile of its daily range"

First, you notice that the first line is what we typed in command-line. Starting at WARNING is where R starts to chatter. The last line is the one we expect to get returned. (Happened to catch the market at a low point). All of this Loading is necessary, but I'd like to silence it.

Our second issue is that '3th' isn't a word. It's supposed to be '3rd'. This can be programmatically corrected, but then the code starts getting dense. Better to live with it or find a better way to phrase the return phrase.

Also with quantmod, you can go to the races and calculate all manner of technical analysis data and have that output to terminal. Now I need to go figure out what's going on with the market...

EDITOR NOTE:

There is nothing going on in the market so I decided to start googling how get rid of Loading messages. Sure enough I found suppressPackageStartupMessages.Now I have to solve the WARNING message issue. That one I'm afraid is going to be a rat's nest.

A Simple Python Script for Traders

2011-11-20T13:38:00.001-08:00

My latest polyglot excursion has landed my in the world of snakes. I've started playing around with Python a few weeks ago and have to report that I'm very pleased with how well this programming language performs. I decided to take it out for a little run on Yahoo stock data and came up with a little script that returns the current state of your stock in relation to the current day's trading range.

To run this script from command line, you first need to give it execution permission. This is time to call the oft-run bash command:

$ chmod +x yahoo.py

Of course I'm getting a little ahead of things. We haven't defined the yahoo.py program. It utilizes three modules: re (regex), sys (defining argv) and urllib (for opening an url). These modules provide the functions we need. Some more preliminaries. We start our program with the shebang line so the chmod invocation actually works. Next, I have used a main() function here when you really don't need it in its current implementation. The last two lines make reference to the main() function sentinel but it's virtually useless for our purposes. The reason to include it is if we decide to grow this program to do other stuff and import it into another script at some later time. Okay, here it is.

#!/usr/bin/python
import re
import urllib
import sys
site = "http://finance.yahoo.com/d/quotes.csv?s="
sym = sys.argv[1]
h = urllib.urlopen(site + sym + "&f=shgl1")
j = h.read()
h.close()
k = re.findall('(\w+)",(\d+.\d\d),(\d+.\d\d),(\d+.\d\d)',j)
stock = k[0][0]
hi = float(k[0][1])
lo = float(k[0][2])
last = float(k[0][3])
z = (last-lo)/(hi-lo)
def main():
if z >= 0.5:
print "%s is trading in the top half of it's range" % stock
else:
print "%s is trading in the bottom half of it's range" % stock
if __name__ == '__main__':
main()

There really isn't much to it, as you can plainly see. I've used the main() function to print two options based on the trading price of your stock. The way to input the stock you're interested in is to pass it as a command-line argument like this:

$ ./yahoo.py SPY

The sys.argv[1] is the first argument from command-line (that's not totally true, but this isn't a computer science course either) and that's all this little script can currently handle. The url address is a little cryptic, so I've chunked it out into three sections. First the http:// part, which is always the same. Then the stock you entered in command-line. And finally a bizarre looking string that has Yahoo values that you're interested in. In our case, we want the stock name, high, low and last price. This gets converted into a string that our regex engine begins to work on. It returns a list of tuples (or single tuple in our case), where we can subset the values we're interested in.

One can see how this program can easily be expanded to include much more information. You could also offer the option to pass flags to the script which tells it what path you want to take to get your desired result. I'm thinking it may be useful to pass a set of stocks to the script and have it tell what percent of the group is doing well or not that day.

This program is obviously a toy program as written, but it's a good beginning for something a little more useful. I'm setting up camp in this snake pit to check it out. I'm not scared of snakes. At least not yet.

My Kids Are Correlated

2011-11-09T09:09:00.001-08:00

I have three kids. Two boys and a girl. They are so different in so many ways that it's amazing they all come from the same genes. But sometimes, they're correlated. This week I told them that they are absolutely banned from any electronic devices after 8:00 pm on school nights. They all reacted the same way. Absolute incredulity. "How can this be?" they all three asked.

They all have different plans of defiance, but they will all fail. Because I'm one step ahead of them. They are all facing a unmovable force and they know it. They just want to make sure.

If I were to try to describe my kids, the last thing I would use is a mathematical concept that regresses them on a linear surface. Even though they appear to be behaving in unison, it's only because they all share a dominant force. And that would be their mother. In that way, I'm correlated to them.

The other night, just before the 8:00 pm pumpkin call, we all decided to dance to some tunes. It was a most interesting thing to observe how they were correlated to the notes and lyrics of the song. They knew the lyrics and sang them in unison. They were correlated.

In the right environment, everyone and everything is correlated. Once that environment erodes, there is the inevitable "quote" -- decoupling. Our song was over and they all went to their separate bedrooms. They decoupled from a momentary correlation. Or was it an opaque relationship? I'm still tapping my foot trying to get it.

Algorithm Zoo

2011-10-20T14:15:00.001-07:00

I suppose it was inevitable that once I started messing around with statistics and curve-fitting of data, I would find myself in a menagerie of equations and scripts. And so I'm moving into the Algorithm Zoo. It's my new home. It seems like a friendly place so far. The animals misbehave themselves as one would expect. And the weather is often nice.

I'll be honest though and admit that I think I've seen some unnatural things. It may have been my imagination, but sometimes at night I swear I see strange creatures roaming around outside of the exhibits. I haven't gotten to visit some places yet, including a secret laboratory cordoned off in the back. I think I'll have access to it at some point. I'll report back here about anything I find that may be of interest.

I'm spending most of my time at the zoo messing around with data mining market behavior, and plotting out distributions. The work is not so easy I'm afraid. It turns out that this menial set of tasks I've been assigned is fraught with danger. Avoiding the perilous OBO error keeps me up at night. Data mining is very willing to accept your erroneous code so great care must be taken in the work. Already, I've had one exhibit taken down by management.

I'm restricted at the zoo as to how much color I can add, but I'll try to push the envelope when I can. Stop by and say "quote" -- hello. Great trading! I'll see you at the zoo!

The Luckiest Man Next Week

2011-09-22T08:05:00.001-07:00

My best birthday present ever was the birth of my daughter almost 12 years ago. Next week I get to celebrate the oddity again, of sharing my birthday with my child. Of course I'm not the only person on the planet whose birthday coincides with their child's, but I feel strangely privileged regardless. "You're birthday's coming up," I say. "Our birthday Dad." Dad smiles inside.

The chances of you sharing a birthday with your child are obviously 1 in 365. It's a longshot, but it happens. Odds mean nothing though. Who cares about them? You've got something special.

It was about two years ago that I had a parent/teacher conference with my daughter's substitute teacher. The regular one, as odds would have it, was having a baby. We started talking about babies and I mentioned, quite proudly, that I share the same birthday as my daughter. The substitute did the pencil-drop and incredulous stare that you get from someone who doesn't believe what you've just told them. "Yep," I said. I thought she would make some comment about "what's the odds of that happening" but instead she replied, "Wow, so do I."

I knew I wasn't the only one, but what are the chances you'd meet someone who shares your birthday secret at a parent/teacher conference? Chances are I would never have met this woman if the regular teacher hadn't gotten pregnant, and what's the chances of that?

"Our birthday is in September," I said.

"Really, so is ours. September 27 in fact," she replied.

This time, it was my turn to drop the jaw and stare with incredulity.

Next week on September 27, my daughter will turn 12. I am one of the luckiest men around.

Get Used To It

2011-08-18T08:10:00.000-07:00

This is a brain teaser. You've been warned.

The S&P 500 is in a bear market (defined as the 50-day MA being below the 200-day MA) 30.8% of the time. Also, the S&P 500 has experienced single-day 4% declines 0.242% of the time. Of the times we experienced single-day selloffs exceeding 4%, 75.7% of the time we were in a bear market.

How much more likely are we to experience single-day losses exceeding 4% now that we're in a bear market, compared to the likelihood of such an event in a bull market?

Odds are you stopped reading this puzzle after the first three sentences. In the unlikely event that you're still with me, I've created a link to some code that holds the answer. It's a prolog script whose knowledge-base is derived from data-mining in R.

Making Stuff is Scary

2011-08-15T07:20:00.000-07:00

My daughter's best friend lives just down the street. Her mother runs a cupcake shop that's just a little further down the street. Being eleven going on sixteen, my daughter fancies herself a "quote" -- worker at the shop. She's not paid in actual money, given child labor laws and all, but sometimes she brings home some cupcakes. The family enjoys them, fighting over certain ones but eating them all in the end. When it's scary story night at the dinner table, I'm always ready with a fan-cheeky paluki tale if not a binglebot story. Our daughter tells the scariest stories though. The stories of what happens to ingredients that come in the back door of the cupcake shop.

As she relates it, all manner of horrifying things happen to the unsuspecting goods. Eggs get broken, their shells carelessly discarded. Butter gets melted, sugar gets pulverized to dust and flour gets churned while wet stuff gets incorporated into it. My youngest son can almost not bear these stories. His eyes drop to table level when the story starts.

After dinner and our night-time routine, when the lights are dimmed and the doors locked, I sometimes go into my own laboratory. What happens here though is simply too scary to tell to the children. Our bed is not large enough for three extra children, all of whom would likely come during the middle of the night from nightmares if they new this story. In my lab, I do things to data that must not be spoken of.

My intentions are good. I'm not trying to create the bizarre creatures that sometimes form from seemingly nowhere. I'm trying to manufacture nice things. Sweet things that people can enjoy. But sometimes the ingredients get mixed in just the wrong way. It's why the lab is double-bolted and hidden in a secret location. Protected with lasers and in-the-wall machine guns. Alright, no lasers or guns, but it is, shall we say, secure.

The latest malformed creature to escape is the product of my Bayesian Prolog machine. I either put in the wrong ingredients (not likely) or my logic machine is missing a critical sprocket or gear. I'm not sure yet. The plan after the initial, requisite clean-up effort is complete, is to create a smaller version of this machine where I know what the answer should be. If I get a working model at the simple level, I can expand it. Here is the prolog code that I have so far. The mysterious ! symbol in the rules section is a cut, and behaves like the ifelse in an imperative language. Careful, like I said it's either missing something or something is in the wrong place.

prob(spx(bear), 0.3039216).
prob(spx(bull), 0.6960784).
prob(spx(up_two), 0.02173633).

conprob(spx(bear) ^ spx(up_two), 0.0125129).
conprob(spx(bull) ^ spx(up_two), 0.009158927).

isprob(A,P) :-
prob(A,P), !.
isprob(A^B, P) :-
conprob(A^B, P), !.
isprob(A^B, P) :-
conprob(B^A, PBA),
isprob(A, PA),
isprob(B, PB),
P is PBA * PA / PB.

I used my new R data mining script to get the percentage values you see in the first part fact section. I also used R to get the conditional fact values. The function downloads all the SPX data (passed in as parameter sym="^GSPC" in the function) from Yahoo with the following line:

x <- getSymbols(sym, from="1900-01-01", auto.assign=FALSE)

I set auto.assign to FALSE because I need the object to be manipulated in the function. I return an object after doing simple look-forward and look-backward manipulations. I also employ the TTR R package's SMA() function to keep a running total of the 50-day and 200-day moving averages. I use these average to define bull and bear markets. Let's define the S variable as containing the complete set of data from the data mining function. To get a slice of it (ie, bear and bull markets) requires a simple indexing operation. Like this:

> bull <- S[S$r.50 > S$r.200]

The r.50 and r.200 vectors are where the 50- and 200-day average values are stored. I'm not sure why I called them r.50 and r.200. Come to think of it, it's not very intuitive is it? I'll get to that later. In any case, to get the percentage of time the SPX is a bull market requires a grade-school calculation:

> nrow(bull)/nrow(S)
[1] 0.683243

Now you know where I get the probability fact that goes into the beginning of the Prolog script. The contingency fact (I think this is where my problem is) is calculated below. The bull.up object is all instances where the return was greater than 2% and the market was in a bull regime.

bull.up <- S[S$RET > 0.02 & S$r.50 > S$r.200 ]

Then to get the value:

> nrow(bull.up)/nrow(S)

[1] 0.009158927

Well, after all of this is compiled in a Prolog session, the following query generates the following result:

?- isprob(spx(up_two) ^ spx(bull), P).
P = 0.000286004363470997

But a direct calculation in R gives the following result (which I trust more):

> ans <- bull[bull$RET > 0.02]
> nrow(ans)/nrow(bull)
[1] 0.01340508

So it's back to the algorithm drawing board I suppose. Tomorrow night is joke night at the dinner table. I think I've got some good material.

The Apocalypse Data Mining Trade

2011-08-11T06:53:00.000-07:00

The logic behind this trade is based on the observation that the SPX has responded consistently through the decades in the subsequent days after an Apocalypse day (6.5% selloff or more). Today is Day 3. The in.3 column are the historical returns for Day 3 from the close of Apocalypse day. The S&P 500 has never closed below the close of Apocalypse day by Day 3. This is also true of Day 2 (which is not shown).

The 30-day return ranges from a loss of 24% to a gain of 28%. There have been 8 of 12 times when the return was positive. The average return is 1.7%.

I went long SPY at $111.46 in pre-market today. I'll close it in 6 weeks and my returns will become part of the distribution.

This Time Is Not So Different

2011-08-09T09:04:00.000-07:00

The first time it happened was September 26, 1955. Earlier that same year, Ray Kroc started McDonald's, Elvis made his first TV appearance and Congress ordered all US coins bear the "quote" -- In God We Trust. Just two days prior, President Eisenhower suffered a heart attack. That was the first time the S&P recorded a single-day 6.5% loss. In the intervening years it would happen 11 more times. Yesterday was the thirteenth time it's ever happened.
To say this time is different is self-evident. McDonald's is not a start-up and Elvis is dead. Of course it's different. This is enough to persuade most that data mining what happened afterwards is a futile effort. Most of the time, what happened next is an almost perfectly distributed result around a mean of zero. But this event has a unique distribution on what happened next. The next day has historically experienced a positive return. Not every time, but just 11 of the past 12 times. Based on that edge and with the help of a little Haskell and R, I calculated that on Tuesday August 9, 2011, the SPX should close at 1160.65 (calculated when ES futures where trading around 1080 mind you). This is of course a silly prediction. But it's a marker of sorts. It tells me that pressing a short is probably a really bad idea.

It wasn't enough for me long the future. I'm thinking next time I see something like this it should be worth a shot though. If you look at returns over Day 2 and Day 3 after such a precipitous drop, you'll see that every time the index experienced a positive return. Every time. On Day 2 the total returns from the close of Apocalypse day had a minimum of 0.82% and a maximum of 14.91% with a mean of 6.38%. On Day 3 the min was 0.71% , max was 13.94% and mean was 5.94%.

Present events are not bound to the past. Obviously if the President had a heart attack three days ago, we would have heard about it by now.

Logical Nonsense

2011-07-31T05:20:00.000-07:00

In 1979, rock band Supertramp released their Logical Song. It's a plaintive melody that compares two sets of values with an obvious bias against acting "quote" -- logical. But what's wrong with logical? Isn't the world made that way, and doesn't our ability to perceive it correctly rest on our understanding of its constraints? Our long-haired, falsetto-voiced friends from the 1980s may have been unto something though. Sometimes logic will lead you to draw silly conclusions.

A good way to test this idea is to run a simple program in Prolog, the logic programming language. This obscure little programming language is particularly useful in sorting out logic problems from scheduling applications to Bayesian networks. It does voodoo on facts and rules to answer questions. Let's try it out with some simple facts. I'm using the Yap implementation by the way. The following code snippet looks like a function, but read it differently. Read the following snippet as saying "euro is the currency of greece". Ignore the lower case for now. It must be. Look into it later if it's important to you.

currency(euro, greece).

This is a fact in prolog. It uses it to build its perception of the world that you're describing. Let's put some more in our program.

    currency(euro, germany). 

    currency(euro, france).

    currency(euro, spain).

    currency(euro, portugal).

    currency(euro, italy).

There are more we can add, but that'll do for now. Next, we'll create a rule to expand the knowledge of the program.

friends(X, Y) :-

        currency(Z, X), 

        currency(Z, Y).

We've introduced some variables and they're upper-case. Again, it must be so. We've also introduced a strange symbol with :- that looks to separate two parts of an argument. Indeed, that's exactly what it does. Sometimes its called the neck because it joins the head to the tail. Prolog considers the tail first and if it is true, applies the head to its knowledge as being true. Read the tail first. If Z is the currency of X and Z is the currency of Y (they share the same currency), then ... read the head ... X and Y are friends.

Save the above script in a file called fiasco by doing something simple from command line such as this:

$ vi fiasco

:wq it and in the same directory start up a Prolog session and type in the following after the ?- prompt:

?- [fiasco].

This should get the program ready for query. Let's ask a question, which is basically the tail of a predicate.

?- friends(greece, germany).

yes

We ask if Greece and Germany are friends and based on the knowledge that we've input it tells us that yes it's true. They are friends. This is logical and though you may argue it's nonsense, it really isn't because we've made it so. But now let's push this idea a little bit more.

?- friends(greece, greece).

yes

Well, there you have it. Greece is friends with itself! Now that is truly ridiculous. But it is logical because it meets the requirements we've put in place. It turns out that logic sometimes needs a little filtering to make sense, something humans are quite good at. And since we're so good at it, let's let Prolog in on our secret with the following snippet, added to the friends rule:

dif(X,Y).

You'll need to change the period after the previous tail sequence to a comma because comma is a conjunction operator while the period concludes the tail.

To recompile the program after you've edited it, type in the following:

?- [-fiasco].

You can't be friends with yourself because friendship is a relationship between two, duh Prolog. You don't get by with a little help from yourself, you get by with a little help from your friends. It has been written in song, and it is true.

Fruit Fly Rodeo

2011-07-23T13:30:00.000-07:00

I was reading up on the debt-ceiling crisis and thought yawn, can we get a little more dramatic about this? Just like the Greek or Portuguese or Italian or Irish debt crisis. Whatever, that's all we hear about anymore. Everyone's got troubles with their debts and spending. These things never get solved because some Bilderberg would be out of a job. And that's when I saw the first fruit fly in my kitchen. Annoying little bugger. They're not fast so I slapped him between my palms with a clap. There, back to reading up on Obama and Boehner making jello or some such thing.

But then I saw the second one. And a third. Time to step back from my twitter-generated debt-story links and find what piece of rotten fruit was generating interest for these bothersome pests. I knew that contrary to folk wisdom, fruit flies are not spawned from the fruit. They come from elsewhere (probably my neighbor's trash) because they're attracted to fermenting sugars generated by a rotting fruit. I found an over-ripe onion and potato, but no fruit. Probably the end of my troubles for now, I thought, but I wasn't sure.

I decided to get on the offense and fashioned a fruit-fly trap. I filled a jar with compost-ready fruit and covered it with cellophane that I poked through with tiny one-way holes. I hated sacrificing perfectly good compost for this idea, but sometimes that's what it takes. I made two other traps in different flavors. My thought process here being that I'd give the tiny vultures a choice of trap to make it a little less conspicuous. There's nothing more conspicuous to a fruit fly than a single trap.

My bet is that the fermenting Greek fruit is the rottenest trap of the group and that's where the fruit fly swarm ends up going. So I'm short the euro and long the dollar. I just hope they hate jello.

FULL DISCLOSURE: I'm short from 1.4361 going into the weekend. This was before Obama held a press conference at 7 p.m.

How To Make Money When You're Confused

2011-07-16T07:33:00.000-07:00

I bought TLT about three weeks ago for $96.85. It's an ETF that tracks 20+ year US Treasuries. I owned this stock last year and when I was doing taxes discovered that my lazy strategy of selling calls and puts and collecting dividends actually yielded some nice returns. But I sold out completely earlier this year (for a loss) because it was "quote" -- technically in a bear market phase. And I listened to some commentary that said bonds were going to zero, or something like that. I can't remember. All I remember is that I was scared. Three weeks ago I wasn't scared anymore and I jumped back in. Three days after I bought back in I was scared again. I picked a local top to go long. Again.

When I bought back in, I did so with gusto. Not only did I buy the stock, but I sold a naked put at 96 (with July expiration). I was taunting the market (not really, but kinda) to go lower. Technicals were in my favor with a golden cross in play. The put was sold for $1.07. This was very clever, I thought to myself, because if the stock goes down I will have added to my collection at a lower price. If I am exercised, I will own the stock at $94.93. I can live with that. My new cost-basis will be $95.89.

Adding to loser is a sticky issue. If you hear never-add-to-a-loser evangelism, it's likely coming from the pulpit of a scalper. Traders with a horizon of greater than a couple days don't mind building a position over the course of a week, and they are successful with doing things this way. If you're adding with the "it can't go lower" mentality, then go back to paper trading. Or just don't add. I want to play TLT all year, so I can add lower.

The problem with planning to add lower is if the stock actually goes lower and you have to follow through with your plan. When I bought my first lot of TLT along with my naked put, the stock started down. I suffered the typical anxiety, confusion and frustration that comes with picking a top. My solution was to sell an out-of-the-money call immediately. I sold the 97 call (July expiration again) for $0.93. Now I felt better because if TLT's little pullback became a death spiral, then I would own stock at even a lower cost-basis. I just paid myself $2.00 to help deal with the stress of it all.

Over the next couple days TLT continued to sell off. Maybe Gross was right. It sold off four days straight and traded in the low 93s. Uggh. This is when I started data mining how often this type of sell-off occurs, and what happens next. It turned out that there was a small edge for a bounce, so I stayed pat. We bounced off the lows and I let it ride towards expiration.

When expiration week rolled around, TLT was starting to show some life. On Tuesday, Wednesday and Thursday, it breached my short call strike of $97 and now I began to worry that I was giving up my TLT on the cheap. Who knows, maybe Hendry is right and we could go to the moon. Then how would I feel? I breathed the box and decided to let this ride. I would entertain myself watching it trade between $96 and $97, because if it closed there on Friday I would keep the $2.00 synthetic dividend and be on my merry way. On Thursday we closed at $96.01. The drama was becoming intoxicating. On Friday we gapped down to $95.29 and I consoled myself that I would double my TLT position come the weekend. But then we rallied and began trading above $96 again, but below $97. TLT was trading at $96.01 an hour before the close and closed at $96.17. I keep the proceeds with no exercise to the call or the put.

The technical name for my trade is a short strangle. There is no upside risk if you have stock to cover the call, unless you consider getting a small profit a risk and that's just plain dumb. The risk is to the downside. If TLT goes to zero, I lose it all. Twice because of the naked put.

If you have a favorite stock you trade and don't mind getting extra or giving away what you have, short strangles and straddles might be a candidate. I say might. The TLT trade requires one-contract un-leveraged capital north of $19,000 so I don't recommend this strategy until you can become comfortable with being confused.

Last Time That Happened

2011-06-30T21:25:00.000-07:00

I bought a lottery ticket yesterday. I hardly ever buy them. The last time I did I lost a dollar. Actually, every time I've bought a ticket I've lost. Yesterday I was at the local gas station in line behind a bloke who had a comprehensive folder of tickets. Kinda like those extreme coupon people, but he had lotto tickets instead. I thought to myself "why not?" This morning I checked the numbers. I lost my dollar. I hope whoever won it is happy to have it.

We are all compelled to use our experiences as factors to our judgment. There is no guarantee that history repeats itself, of course. But it's good to have some perspective on last time something happened. Sort of like the meltup we had the last four days. The S&P 500 returned over 0.85 percent for four days in a row. Wow. When was the last time that happened?

Well, good thing we have R to do some data mining for us. Here is a nifty function that returns a data series complete with my own preferences on how to view things. It defaults to using the Federal Reserve of St. Louis for its data source. That takes us back to 1957.

future <- function(sym="SP500", source="FRED"){

    require("quantmod")

    x          <- getSymbols(sym, src=source, auto.assign=FALSE)
    x$ret      <- dailyReturn(x, type="log")
    x$ago.1    <- lag(x$ret, k=1)
    x$ago.2    <- lag(x$ret, k=2)
    x$ago.3    <- lag(x$ret, k=3)
    x$ago.4    <- lag(x$ret, k=4)


    x$next.1   <- lag(x$ret, k=-1)
    x$next.2   <- lag(x$ret, k=-2)
    x$next.3   <- lag(x$ret, k=-3)
    x$next.4   <- lag(x$ret, k=-4)
    x$next.5   <- lag(x$ret, k=-5)
    x$next.6   <- lag(x$ret, k=-6)
    x$next.7   <- lag(x$ret, k=-7)
    x$next.8   <- lag(x$ret, k=-8)

    x$in.1     <-    exp(x$next.1)-1
x$in.2     <-    exp(x$next.1 + x$next.2)-1
x$in.3     <-    exp(x$next.1 + x$next.2 + x$next.3)-1
x$in.4     <-    exp(x$next.1 + x$next.2 + x$next.3 + x$next.4)-1
x$in.5     <-    exp(x$next.1 + x$next.2 + x$next.3 + x$next.4 + x$next.5)-1
x$in.6     <-    exp(x$next.1 + x$next.2 + x$next.3 + x$next.4 + x$next.5 + x$next.6)-1
x$in.7     <-    exp(x$next.1 + x$next.2 + x$next.3 + x$next.4 + x$next.5 + x$next.6 + x$next.7)-1
x$in.8     <-    exp(x$next.1 + x$next.2 + x$next.3 + x$next.4 + x$next.5 + x$next.6 + x$next.7 + x$next.8)-1

food       <- paste(sym,"future.returns",   sep=".")

assign(food, x, envir=.GlobalEnv)
}

If you run this program, you will get an unwieldy monstrosity. First, re-assign the name of the object returned by the function to S. Then, you need to do a little indexing to get the data you want. To get only those days where the last four days each had returns that exceeded 0.85%, run this little command from the R command prompt:

s <- S[(exp(S$ret)-1) > .0085 & (exp(S$ago.1)-1) > .0085 & (exp(S$ago.2)-1) > .0085 & (exp(S$ago.3)-1) > .0085 ]

It's a little verbose, but I prefer using log returns so sometimes it requires a little extra jumping around. This gives you a smaller data set, of length 9. We are interested in next.1, which are the returns the next day. Again with the dog-and-pony show with logs, but it must be done. It makes sense to plot it with a histogram, so let's do that all with one fell swoop.

hist((exp(s$next.1)-1), col="burlywood", main="Next Day Returns After Four Consecutive Up Days\n of Over 0.85% Each", xlab="1957 to Present")

And here is the resulting chart:

There you have it. Seven out of nine times the next day is a loser. Even with those odds, I would hesitate to bet a dollar on it though. Okay, maybe one dollar.

Hello, Goodbye Linear Regression

2011-06-26T10:06:00.000-07:00

Summer is the time to visit family. Kids get some grandma and grandpa time, parents get to spend some time with siblings and their parents or in-laws. It's a grand time of the year. For some, these summer relationships are a little "quote" -- complicated. "Well it's good to see things are going well." Ten minutes pass. "I think we'd better go to the airport a little early since there is so much traffic and with the new security issues and all." Hello, goodbye is a common meme.

I decided to pick up a book on statistics this past week to help pass some time during these hot days. I can't remember when we started a relationship, but I know it's been a long time. And often frustrating. After pondering how linear regression modeling can help me with building trading models, I decided that it can't. These statistical techniques have too much baggage and I don't have time to deal with it. Everything has to be perfect for them. Zero mean of disturbances, constant variance of errors (homoscedasticity, if you prefer six-syllable words), non-stochastic predictor variables. Really? People live like this? In what universe if you don't mind me asking?

I've also changed through the years, so maybe I shouldn't be too hard on linear regression. These days, I much prefer questions that are framed where the dependent variable is dichotomous. This is asking too much from linear regression modeling. Sure they try with dummy variables, but then they suffer from nonsense predictions given the nature of boundary problems. They try to compensate for this with truncating regression lines at boundaries, but then they're saying that predictor variables have no influence at these points, which is troublesome as I'm sure you'd agree. By the time they start modifying their straight line predictions to an S-shaped curve, which is more appropriate, we have our last vestige of hope that this relationship can be meaningful dashed. You see, once you start curving a linear regression model to accommodate for a boundary, you've violated just about every assumption that linear regression has. There is probably a warrant out for your arrest. Not to mention that you're saying sayonara to making sense of additivity with this approach.

There is a statistical technique related to linear regression called non-linear regression. If you familiarize yourself with her it will remind you of when you first got married and pondered how such a lovely person could come from such people who have so little in common with you.

Trading Model Derby

2011-06-20T17:36:00.000-07:00

This past year was the first year I made two models. That's because my youngest son has joined his older brother in Cub Scouts and the annual ritual of Pinewood Derby Car building became twice the fun. I've been "quote" -- helping -- my oldest for two years now, so I've got some experience. I've learned quite a bit from watching how others have built their cars. I look at both winners and losers to form my theories on how a model should be constructed. Losers typically have some piece of Lego attached to it, creating a pretty car with catastrophic drag. The winners tend to be modestly built (not over-engineered, mind you) and properly weighted.

This year as I had both models resting on a table near my kitchen, my neighbor came over and decided to make some comments. This is the neighbor who takes great pleasure in contradicting everything I say. I could say "the sky is blue" and he would invariably respond with "the sky is not blue." I call him Null Hypothesis. I take pleasure in watching his silly comments laid waste, but sometimes it takes more effort than I'm willing to expend at the moment. This year he proclaimed "those cars will lose, they are too simple." I could not prove him wrong on the spot, but the die was cast. I responded "they will not lose, but you will lose. These cars will win first place!" It would take racing day results to reject my neighbor's proclamation. I couldn't wait.

While it's true that I'm a Cub-Scout Pinewood Derby car-builder dad by day, I'm also a algorithm trading model builder by night. I've got the Bumblebee, Gnat, Monkey, Pelican and a whole host of others sitting on the shelf. These vary in construction from simple trading-rule algos to more sophisticated predictors that implement radial-basis function support vector machines. I do suffer from an ambitious curiosity as to how others are building their models though, so I tend to look around and survey the landscape often. I've lately glanced over to a group of builders called Econometricians. No, not electricians. I mean practitioners of the black art of Econometrics.

Econometrics is the science, art and voodoo of building financial models. They are typically frequentists so I found it important to familiarize myself with their methods and terms. When they build their models, they save the wood chips for analysis. Yes, they are indeed a serious group.

One of their favorite modeling techniques is called regression. All types including simple linear regression, polynomial and multiple linear regressions. Regression is basically the process of fitting a line or curve to describe the relationship between a dependent variable and an independent variable or variables. Kind of like the return tomorrow of the S&P 500 will have a relationship with the return of the range of the VIX today. Something like that.

Building the model is not the difficult part. It's analyzing the model and the wood chips (commonly referred to as disturbances) where things can get a little complicated. If the wood chips are too cozy with each other, as in they are serially correlated, there's a problem. If the goodness of fit (R-squared) is low, there's a problem. If the F-statistic isn't large enough or the p-value isn't small enough, there's a problem. You spend most of the time trying to prove that the opposite of what you're trying to show is false. I can sort of relate to this, with my neighbor being who he is. But I know what my neighbor looks like, a 50-something male with kids already out of the house. They don't tell you up front who their pesky neighbor is so you need to listen carefully. Actually, you don't have to listen because I'll just tell you. It's the hypothesis that there is no relationship, or that the slope of the regression is zero.

It isn't good enough to reject this null hypothesis of course, but that's the minimum. I'm going to take a shot soon at this wily art and build myself my own model or two. I'm expecting the first few attempts to fail miserably because this process includes a laundry list of assumptions that I can't keep track of ranging from expected mean of errors is zero to explanatory variables must be non-stochastic. I really can't tell you if anyone ever races one of these models though, so I can't tell you how well they will perform.

You have to race your model eventually, and on race day back at Cub Scouts I was able to prove my neighbor Null Hypothesis wrong. My boys' cars didn't lose. They won a couple of heats. I wasn't really right to say they would win first place because they didn't (silly dad forgot to sand off the burrs on the nail axles!), but at least my neighbor Null Hypothesis was rejected. Next year, the Den Leaders have agreed that dads will be able to build their own dad-only cars for a special race. The only rules are no thermonuclear powered devices. I can't wait.

vRoom vRoom : Speeding up R with C

2011-06-08T14:29:00.000-07:00

Many times you don't want to trouble friends for help with menial tasks like moving furniture. But sometimes you need to step out and ask. Your friends are always happy to help, and after the heavy lifting is done you see how easy it can be. R likes to move furniture. It's okay with moving a small table across the room, but when you need to bring a large sofa up three flights of stairs, it's time to ask for help. Your best friend is C.

The basic idea behind speeding up R with a C script is to write a C script, compile it, load it into the environment (ask it over to the house, so to speak) and call it from R with a built-in R function. There are some choices here but for our test case, we're going to use the .C() function.

A couple ground rules first. The C function must be of type void (does not return nothing) and needs to accept pointer arguments. You define the argument in R as an object, and it gets passed as a pointer to the C function. R objects are basically pointers anyway so this is not terribly difficult. Let's illustrate before this becomes a lecture on pointers and objects.

Here is the code for an R function that uses an R loop to print out permutations of a double loop. It's the guts of a brute force search but only returns a string saying that it was able to locate and print each permutation. You can either copy and paste it into R or save it and source() it.



 twoloop <- function(){

for(i in seq(10, 50, 1 ))

  for(j in seq(80, 240, 4))

cat("Parameter one is", i, "and Parameter two is",j,"\n" )

}

We are stepping from 10 to 50 in increments of 1 (40 events) and also from 80 to 240 in steps of 4 (40 events again). The permutation total is those two multiplied, which comes to 1600. The size of a large sofa basically. R can do it, but it takes some time because of the explicit looping. Here is the tail of the output along with performance statistics. The system.time() function is used to calculate performance.

system.time(twoloop())

Parameter one is 50 and Parameter two is 224
Parameter one is 50 and Parameter two is 228
Parameter one is 50 and Parameter two is 232
Parameter one is 50 and Parameter two is 236
Parameter one is 50 and Parameter two is 240
user system elapsed
0.070 0.031 0.139

To get some help from C, we write the following script:

#include < R.h >
#include < stdio.h >

void twoloop(int *startOne, int *stopOne, int *stepOne, int *startTwo, int *stopTwo, int *stepTwo)
{
int i,j;
for(i = *startOne; i < *stopOne+1; i = i + *stepOne)
for(j = *startTwo; j < *stopTwo+1; j = j + *stepTwo)
Rprintf("Parameter one is %d and Parameter two is %d\n", i, j);
}

Notice the pointer arguments. Also, notice we include R.h header file. Save this script as twoloop.c and then from command line, use the following instruction to compile it.

R CMD SHLIB twoloop.c

This creates a new file called twoloop.so that will be loaded into R with the following function inside of an R session.

dyn.load("twoloop.so")

Great, now we use the .C() function to call the function. But first we need to create some R objects. There are six variables so here is what we'll do.

a <- 10
b <- 50
c <- 1
p <- 180
q <- 240
r <- 4

Now we simply pass those objects in as integers. Remember, that is what the C function is expecting so let's not mess around. It is here to help us after all.

.C("twoloop", as.integer(a), as.integer(b), as.integer(c), as.integer(p), as.integer(q), as.integer(r))

Of course if you're like me, you run the function first because you're so excited to see if it actually works. It does and then you remember that you were supposed to time it. No worry here, hit the up arrow to prompt R to display the last command (the monstrosity above) and hit Ctrl-A to get to the beginning of the command. Insert system.time() like thus:

system.time(.C("twoloop", as.integer(a), as.integer(b), as.integer(c), as.integer(p), as.integer(q), as.integer(r)))

Here is the tail of the C function's output along with its performance.

Parameter one is 50 and Parameter two is 224
Parameter one is 50 and Parameter two is 228
Parameter one is 50 and Parameter two is 232
Parameter one is 50 and Parameter two is 236
Parameter one is 50 and Parameter two is 240
user system elapsed
0.002 0.002 0.017

Nice. Time to crack a beer on the new sofa. With our best friend C.

An Pre-Introduction to Brute Force Parameter Search in C

2011-06-06T15:52:00.000-07:00

It used to be that our culture was simple technology and sophisticated people. Now the technology is sophisticated and the people are simple. But you don't need to feel trapped. You can always get into a time machine (I know, sophisticated technology) and go back in time to when technology was not so complex. You know, back when people wrote programs in C. From command line. With vi. Ah yes, those were the days.

We shall travel there now, and find a way to create a series of permutations between two parameters. If this sounds crazy, well it just means you haven't been stress testing your favorite trading system, that's all. You know the system I'm talking about. The one where you place trades based on where the 50-day and 200-day moving averages are in relation to one another. You can backtest this strategy based on these simple rules, but the true curve-fitting fun starts when you backtest a whole bunch of parameters. For example, the 50-day and the 180-day. Or the 40-day and the 160-day. You get the idea.

I wrote one such program in R recently, but I must admit it was a bit ... yawn ... slow. It was actually painful to watch. I felt bad for R, really. Well not to worry, we can amp it up a bit with a little C program that takes the burden off R and busts out the parameters lickety-split. The first part of this process is to get the C code right. There is no better way than to fire up vi and start a typing.

$ vi parameter_cruncher.c

That opens up a new window where you can start typing in the following code. Remember that to actually type in vi you have to select the "i" key. That puts you in insert mode. Anyway, here is the code:

// takes 6 arguments



#include <stdio.h>



int main(int argc, char **argv) 

{

    int m = atoi(argv[1]);

    int n = atoi(argv[2]);

    int p = atoi(argv[3]);

    int q = atoi(argv[4]);

    int r = atoi(argv[5]);

    int s = atoi(argv[6]);

    int i;

    int j;

        

    for( i=m; i<n+1; i=i+p)

      for( j=q; j<r+1; j=j+s)

    printf("Parameter 1 is %d, Parameter 2 is %d\n", i, j);

    return 0;

}

Once you get that saved, you can compile it from command line with the following:

$ gcc parameter_cruncher.c -o parm

This makes our executable a bit more manageable to type. Then we type the following with 6 arguments (it's in the comments, but you can also count the variables that call the atoi (ASCII to Integer) function).

$ ./parm 10 50 10 150 250 50

This reads as from 10 to 50 in steps of 10 and from 150 to 250 in steps of 50. Basic permutation stuff, nothing complicated really. I wrote this to test that the code actually does what I want it to do before I implement it by calling it from R.

This list isn't too long (can you calculate the correct permutation count in your head? quick!). Here it is below:

Parameter 1 is 10, Parameter 2 is 150

Parameter 1 is 10, Parameter 2 is 200

Parameter 1 is 10, Parameter 2 is 250

Parameter 1 is 20, Parameter 2 is 150

Parameter 1 is 20, Parameter 2 is 200

Parameter 1 is 20, Parameter 2 is 250

Parameter 1 is 30, Parameter 2 is 150

Parameter 1 is 30, Parameter 2 is 200

Parameter 1 is 30, Parameter 2 is 250

Parameter 1 is 40, Parameter 2 is 150

Parameter 1 is 40, Parameter 2 is 200

Parameter 1 is 40, Parameter 2 is 250

Parameter 1 is 50, Parameter 2 is 150

Parameter 1 is 50, Parameter 2 is 200

Parameter 1 is 50, Parameter 2 is 250