It's All Greek to Me

Book Review: Rock Breaks Scissors by Poundstone

2016-02-23T08:38:00.002-08:00

http://www.amazon.com/Rock-Breaks-Scissors-Outguessing-Outwitting/dp/1491531185

This is the third book I have read recently that is about forecasting (see here and here for the previous two).

Poundstone did a pretty good job overall. The first chapter of the book is a general discussion on the human psychology of perceiving randomness. The following one half gives specific examples of how to outperform in various more light-hearted and trivial prediction exercises (e.g. multiple choice tests, rock-paper-scissors, passwords etc.), and the remaining one half does the same thing but with more serious topics (e.g. sports betting, web big data, stock market etc.).

Once again I'll try to distill the essence of the book here:

The human mind is handicapped in creating and detecting randomness
We can take advantage of the above in many situations for a statistically significant gain
In situations where randomness rules, the real pros do not care so much about views; they care more about odds

Book Review: Superforecasting by Tetlock and Gardner

2016-02-07T01:25:00.002-08:00

The theme of this book is very similar to Nate Silver's The Signal and the Noise. The first few chapters are worth reading, while as it drags on the final one-third of the book is less appealing, as it indulges in corny talks of team works etc.. The new ideas I got from this book, in addition to those already listed in the review above, include:

The use of Brier score for rating forecast, because 'good' forecast should be in probabilistic terms
As a corollary, one has to make enough forecasts in order to have a statistically significant score (=/= good score)
Update predictions as frequently as appropriate
There is a vicious cycle going on in the 'Mainstream' of our society: people are not aware of the importance/option to think probabilistic -> they crave for binary, determined answers -> the media are aware of this demand -> the media cultivate the guru/pundit culture -> the people get more addicted to this way of thinking

All in all, it is not as readable a book as Silver's, but I would still recommend it to anyone who is serious about improving and understanding the art of forecasting.

Book Review: The Signal and the Noise by Nate Silver

2015-03-19T05:37:00.002-07:00

An anecdotal account without much technical materials, nonetheless an enjoyable read. Through the case studies (weather forecast, earthquake prediction, economic forecast, sports betting, poker, etc.) the readers are gradually introduced to what quantitative prediction is/should be.

If I am to distill three most important points made in the book, they would be

Think probability; or even better, think CONDITIONAL probability, aka Bayesian.
Giving your prediction a context, or "having a story," would result in forecasts that are much better than blindly dive into datamining.
The evaluation of the performance of predictions should not be based (solely) on whether they got it right. Maintaining best practices is more important.

There may not be much originality in the materials, but the writing is decent and there are valuable lessons to take home. Recommended especially for people who work with quantitative data manipulation.

Minority Game, Majority Game, $ Game etc.

2014-07-01T01:58:00.001-07:00

Minority game (and other extensions) is the formalized study of the El Farol Bar problem. It was proposed by various researchers that the dynamics under such games resembles the financial market. This is a list (with some sparse comments) of literature that are related to this topic:

Martino et al, 2003, "Statistical mechanics of the mixed majority-minority game with random external information"

Namatame and Sato, "Localized minority games and emergence of efficient dynamic order"

Satinover and Sornette, 2007, "'Illusion of control' in time-horizon minority and Parrondo games"

Wiesinger et al, 2010, "Reverse engineering financial markets with majority and minority games using Genetic Algorithms"

Cherkashin et al, 2009, "The reality game"
This framework is particularly interesting because it allows for feedback. The size of the bets placed by the players would change the odds of winning.

Vitting Andersen and Sornette, 2002, "The $-game"
They introduce the $-Game, and explain why it (instead of the Minority Game) is a better model for the financial market.

Yeung et al, 2008, "Models of financial markets with extensive participation incentives"
They introduce the Wealth Game, and define the temperature of the grand canonical ensemble. More interestingly, they connect this theoretical framework to the real market by calibrating the model to market data, thus identifying where we are on the "phase diagram."

De Martino et al, "From perceived risk to collective behavior in minority games"

Volatility Smile and Volatility Models: A Short Note

2014-06-30T17:46:00.001-07:00

Local Volatility (LV) Model

LV preserves market completeness (is it a good/must-have thing?)
LV gives forward smiles that are too flat compared to market
In fact, LV doesn't really qualify as a model: it fits implied vol perfectly but trivially, with no capability/intention to consider the underlying dynamics
LV also produces poor smile dynamics prediction

More General Notes

The most sought-after quality for a model should be accurate hedging, not mathematical tractability etc.
In fact, there is no good reason to limit ourselves to vanillas only when calibrating a vol model; calibrating only to vanillas has been common practice, which is unfortunate
The information carried by vanillas is the unconditional (risk-neutral) probabilities that can be recovered by taking second derivatives w.r.t. strikes (i.e. the stock price is here at t=0, what is the probability that it will end up there at t=T?)
Conditional probabilities, on the other hand, is not contained in vanillas (i.e. conditional on the stock price being here at t=s
Another way of saying this is that vanillas are not sensitive to unconditional distributions
Therefore, one way to proceed is to include some exotics in the calibration: forward starting options are sensitive to conditional probabilities; while American digitals (or barriers) are sensitive to jumps
As a rule of thumb, jumps are "responsible" for short-dated smile, and stochastic volatility is "responsible" for longer-dated smile

Reference:
Ayache 2004, Can anyone solve the smile problem?
Ayache 2005, Can anyone solve the smile problem? A Post-Scriptum

Also, Rebonato's Volatility and Correlation: The Perfect Hedger and the Fox is relevant, and it's a great read, but that would be another episode.

How do you model the dynamics of asset prices? A self-reflection (Epilogue of 3)

2014-03-01T04:55:00.002-08:00

As the title suggests, this was supposed to be a three part series. But I feel like I have left out so much that I wanted to discuss, so although the first 3 parts have already provided an overview on what I would call the "classical" quant modeling approach (loosely speaking, one that focuses on describing the asset price time series with a random process), I would take this opportunity to talk about what lies beyond such approach.

We are interested particularly in capturing the tails of the distribution. What about putting the tail on the center stage? That is the spirit of extreme value theory. It has had wide application in natural disaster forecast. The basic idea is to posit a probability distribution that specifically fits the tail of the distribution. Contrast this with, say, the stochastic volatility approach in which we tried to fatten the tail, while still keeping our focus on the 'body' of the return distribution. Now we are directly and un-apologetically modeling the tail itself. The advantage is obvious: we have more freedom in fitting the tails closely by dedicating the calibration to it. The short coming is more subtle. In order to perform extreme value analysis, we have to specify the threshold at where the tail starts, so that we can attach a distribution to it. If you define daily return of -20% as the threshold, then you can use some distribution to catch the sub-sample of daily return less than -20%. But how do you choose the threshold? Choose it too deep into the body and you miss the point of using extreme value models. After all, we are trying to focus on the tail by not worrying about modeling the boring part of the distribution. On the other hand, choose it too far off into the tail and you suffer by not having enough historical data points for an accurate calibration. The other issue is that extreme value analysis would not solve the "you don't know what you don't know" problem. You pick a threshold, you do the fitting, and chances are sooner or later a much worse draw down would take you by surprise, proving to you that the tail is even fatter that you have assumed.

Now you might say, "Hey, isn't that a necessary evil of any perceivable model in finance? That one has to make assumptions and wait to be surprised?" And I tend to agree to the sad fact. Perhaps except for if we follow a completely different paradigm. Recently some researchers having been looking at crash forecasting using a wide range of unconventional tools. They range from fractal analysis that studies the (change in) scaling of time series; agent-based models that take a bottom-up approach trying to produce asset price dynamics by considering (grossly simplified) behaviors of traders; and critical point/phase change models that are inspired by Ising model in solid state physics. The common theme is the acknowledgement of market non-linearity and the lack of a single, forever stable market state. Unfortunately there is no straight forward way to connect these models to option prices, but with the advancements of numerical methods we may some day see wider applications of them.

Looping the Strings Problem

2014-02-28T20:27:00.002-08:00

This one is a classic that I believe is found in one of the popular interview books. You start with N pieces of strings that are mixed into a single pile. You start randomly choosing two free ends at a time and joining them together. After N rounds, you will be done as there will be no more free ends left. If the number of closed loops is k, what is F = E[k]?

If you go down the combinatorial path, you will soon see how hopeless it is to try figuring out the many different intermediate states. So perhaps combinatorial is not the right tool here. In fact, this problem can be readily solved with a closed-form solution using induction and recurrence. Before arriving at that, let's get some sense and inspiration by stating with a small N.

N = 1 is trivial and we see that k = 1 surely, so F = 1. Consider N = 2. During the first round of joining, when one free end is picked, it can be joined to any of the remaining 3 free ends. In one out of these three cases, we end up with k = 2 (namely, if the free end is joined to another end of itself); while in the remaining two cases, we end up with k = 1 (namely, if the free end is joined to any end of the other string). So $ F_2 = \frac 1 3 (1+F_1) + \frac 2 3 (F_1)=\frac 4 3 $. More generally, it's easy to show that $ F_N = \frac 1 {2N-1} (1+F_{N-1}) + \frac {2N-2} {2N-1} (F_{N-1})$

This recurring relation can be rearranged to give the difference equation $ F_N = F_{N-1} + \frac 1 {2N-1} $. With the initial condition of $ F_1 = 1$, we can alternatively express it as a finite sum. Note how slow it grows with N (~ log(N)).

How do you model the dynamics of asset prices? A self-reflection (3 of 3)

2014-01-30T01:18:00.000-08:00

Last time we added jumps to the recipe to enrich the original random walk dynamics. Jump diffusion model is able to fit to market implied vol much better than the diffusion only counterpart. Before moving on, just two reminders:

The numerical pricing procedure becomes more involved, especially for grid/finite differencing methods (because we end up with an integro-differential equation, as opposed to a PDE)
Although we usually speak of "the" jump diffusion model, we are free in specifying the probability distribution of the jump amplitude

How about good old Gaussian?
So is jump diffusion the ultimate holy grail? And is pure diffusion model done with? For the first question, we will see later in the series that there are many other approaches that supersede jump diffusion, but for now let's take a step back and address the second question.

It turns out that the use of mixtures is able to tackle at least some of the shortcomings of the original random walk model. A mixture, not surprisingly, is formed by combining more than one 'sub-distributions.' (usually, but not necessarily, of the same kind) Here we focus on Gaussian mixture. There are a number of possibilities regarding how the sub-distributions are combined, the use of Markov chain being a prominent one. Essentially, every time we need a new random variable to be generated, we first toss a coin to determine which sub-distribution out of the mixture to draw from. In the case of a mixture of two Gaussians, if one sub-distribution has a larger variance than the other, then the mixture would have fatter tail, although the individual sub-distributions do not.

Candidate: Mixture

Markovian
Most likely not tractable
Excess kurtosis
Market incomplete

The use of mixture models in option pricing is rare, possibly because not everyone likes the idea of the hidden Markov chain that comes with the same package. Justifying and interpreting it is easier in the context of econometric modeling that deals with longer time frames, in which case the state changes can be perceived as a change in regimes. For something like intra-day option pricing, the model may require that we draw from alternate sub-distributions every 5 minutes or so, and it's not trivial to make sense of what that means. (other than the fact that it fits the data well)

Another French Mathematician
If producing a fatter tail is all we care about, then what about substituting Gaussian process with something with excess kurtosis? The resulting model would likely not have analytic solution, but we can fall back on numerical methods as our last resort. For example, ask the computer to generate 10000 random numbers from a Student's t distribution and use those in a Monte Carlo simulation.

The idea sounds neat, but one has to be cautious about it. We may not be aware of it, but when we were modeling asset prices with Gaussian or Poisson processes, both of the processes are closed under linear transformation. In other words, aggregating multiple Gaussian (or Poisson) random variables would result in yet another Gaussian (or Poisson) variable. That means when we use Gaussian process to model asset prices, even though it may not be a very faithful description of reality, at least we are being consistent and we know what we are getting ourselves into: namely, if the one-day distribution is an N(0.01, 0.05), then the one-month distribution would be an N(0.25,0.25), etc. Other distributions in general do not have this property. If in our hypothetical Monte Carlo simulation we draw from a t distribution with a degree of freedom of 10 for daily return, how would the monthly return be distributed? How about quarterly?

These problems can be avoided by going for non-Gaussian processes while staying within the territory of Levy processes, which is a broad family of processes that possess the nice property of being closed under linear transformation. (obviously both Gaussian and Poisson processes are members of this family)

Short Notes on HMM (Hidden Markov Model)

2014-01-17T07:54:00.001-08:00

1. Regarding calibration

Numerical optimization is computatinoal intensive and prone to local max, but it is able to estimate any general model specification. On the other hand, EM algorithms such as Baum-Welch are more reliable in seeking global max, but they are only capable of backing out the parameters of models with distributions that are fully described by moments (Gaussian, Poisson, ...)

2. Problem with HMM, and extensions

As Soltan discusses in his paper, ordinary HMM is by definition prone to instability due to the exponentiated matrix structure. The instability dominates in the long run, meaning that state occupancy duration estimation of ordinary HMM is usually unreliable. There are two ways to go: 1) HSMM (a.k.a. explicit duration HMM, inhomogeneous HMM, etc.), or 2) Treat duration as an extra degree of state space (see Soltan).

References:

http://www.reza-soltan.com/papers/3.pdf

http://sist.sysu.edu.cn/~syu/HSMM_bibliography.htm

The Importance of Trading More

2013-12-13T07:48:00.000-08:00

This is by no means an attempt to promote high frequency trading (HFT). In fact, the HFT scene has become so crowded that even real pro shops are having a hard time, let alone retail traders with their home brew system (By the way, here is an interesting read. Judge for yourself how much to believe). So why should a trader trade more?

Because it's about how confident you can be in making a conclusion. If you make one trade in your life and it turns out winning, your rate of winning is 100% and yet the only legitimate conclusion you can make is...well, that you don't have enough data point. In the spirit of Bayesian statistics as was used here, we can formulate a simple analysis that hopefully would provide some insights.

Suppose that you are a manager about to hire a new trader for your team. There are two candidates, Andrew Flook who had 5 winning trades out of 5 last year; and Bob Luckson who had 70 winning trades out of 100 last year. Who should get the job? Since Andrew has a 100% winning rate and Bob has only 70%...are we missing something?

Let's cast this in the light of Bayesian statistics. Of course I would have to make some assumptions:

Winning probability of a fluke trader = p = 0.5
Winning probability of a skilled trader = q = 0.7
Total number of winning trades = W
Total number of losing trades = L
Unconditional probability that any trader is a skilled one = s = 0.2

The last assumption about a trader being a skilled one is based on the anecdotal evidence that only about 20% of all traders are profitable in the long run. So, if $R$ stands for the trading Results of last year, $S$($F$) stands for the trader being a Skilled(Fluke) one, we have

$$ P(R|S)P(S) = q^W (1-q)^L s $$
$$ P(R|F)P(F) = p^W (1-p)^L (1-s) $$
$$ P(S|R) = \frac {P(R|S)P(S) }{P(R|S)P(S) +P(R|F)P(F) } $$

Plugging in the numbers, we find that Andrew (the 100% guy) has a 57% chance of being a skilled trader, while Bob enjoys a 99.9% chance.

How do you model the dynamics of asset prices? A self-reflection (2 of 3)

2013-11-23T21:44:00.004-08:00

Last time we have set a stage for the discussion on modeling the dynamics of asset prices. Gaussian process is the "canonical" way to go, and we've seen the characteristics/limitations of it. To recap:

Candidate: Brownian motion

Markovian
"Tractable"(-ish, depending on how you transform it)
Excess kurtosis = 0
Market completeness (if number of hedging instruments >= number of sources of randomness)

This view is usually summed up and referred to as "random walk" (or diffusion, if you are from the natural science branch of the academia). But does the financial market really walk? Or does it prefer doing something else?

Poisson
We've already met Gauss last time. Now we'll introduce a contemporary of him, Siméon Denis Poisson. While Brownian motion is a diffusion stochastic process (a random walk down the street), the Poisson process is a counting process that is associated with jumps. When we add a Poisson counting process to the diffusion equation, the stochastic differential equation becomes a jump-diffusion equation that allows for jumps, or "gapping up/down". Why is this desirable or required? Just talk to any trader, and they'll tell you that price jump is a reality, especially for markets with substantial close hours (i.e. the majority of the markets other than FX, S&P futures...). So instead of a random walk down the street, perhaps after all it is more like parkour down the street?

You Jump, I Jump
What are the advantages of modeling asset prices with a jump-diffusion process over a diffusion only process?

Candidate: Jump-diffusion Process

Markovian
Not tractable (unless under restrictive assumptions)
Excess kurtosis > 0
Market incomplete (unless under restrictive assumptions)

So, only Markovian-ness stays. Derivative prices under jump-diffusion models are generally not tractable unless assumptions are made. For example, Merton's jump-diffusion model assumes that the jump size is lognormally distributed. The possibility of jumps gives the asset return distribution a desired fat tail feature, hence excess kurtosis. Finally, market completeness is in general lost (there are two sources of randomness - diffusion and jump, but only one hedging instrument - the asset itself), and one has to estimate a market price of risk of the jumps. Incidentally, in Merton's model he assumes that such a market price of risk is zero because of diversifiability.

The implementation of jump-diffusion model can be challenging. The numerical calibration is of course more computationally intensive, but the more subtle and fundamental issue is this: how do we identify jumps? When there is a "sudden" gap in the time series, how can one be sure that it is NOT just an extreme value, yet still being drawn from the good old Gaussian distribution?

Monty Hall, extended

2013-11-22T01:23:00.001-08:00

The original Monty Hall problem is so famous that pretty much every soul knows the solution by heart (in case you don't: change to the other still closed door and double your winning probability!). Despite (or because of) the fame of this problem, some would learn the answer without really knowing how to solve it. Let's go through the problem in its original setting:

Classic Three Door Monty Hall
There are three doors A, B and C, and only one leads to a prize. After you picked door A, the host (who knows where the prize is) opens door C, which turns out to be not the prize door. What is the winning probabilities of you sticking to door A versus switching to door B?

And of course, the toolkit to invoke is Bayesian statistics:
$$ P(A|C_o) = \frac { P(C_o|A)P(A)}{P(C_o)} $$
where $P(X)$ is the unconditional probability of the prize being behind door X, and $P(Y_o)$ is the probability of door Y being opened by the host. In this case, the entities on the RHS are straightforward, except perhaps for $ P(C_o)$:
$$ P(C_o|A) = 1/2 $$
$$ P(A) = 1/3 $$
$$ P(C_o) = P(C_o|A)P(A) + P(C_o|B)P(B) + P(C_o|C)P(C) $$
$$= \frac 1 2 \times \frac 1 3 + 1 \times \frac 1 3 + 0 = \frac 1 2$$
So $P(A|C_o) = 1/3$. One can in a similar fashion find that $P(B|C_o) = 2/3$. The reason $ P(C_o|B) = 1$ is of course that if the prize is really behind door B, the host has no choice but to open door C.

Extended Five Door Monty Hall
So far so good. Now we are invited to a "Monty Hall Deluxe" game with a bigger prize, but more doors and more rounds. In round one, you pick a door. The host would open one door, after which you may choose to switch to another. Then once you have decided and acted accordingly, we enter round two where the host would open one more door (so number of closed door drops to 3 at this point). Then you decide on whether to switch for a second time, and if so to which door. What should the optimal strategy be?

Suppose you first pick door A and the host opens door E. The first round of this game is really similar to the Classic version above. Namely,
$$ P(A|E_o) = \frac { P(E_o|A)P(A)}{P(E_o)} $$
with
$$ P(E_o|A) = 1/4$$
$$ P(A) = 1/5$$
$$ P(E_o) =\sum P(E_o|X)P(X) $$
$$= \frac 1 4 \times \frac 1 5 + \frac 1 3 \times \frac 1 5 + \frac 1 3 \times \frac 1 5+ \frac 1 3 \times \frac 1 5+0 = \frac 1 4$$
So $P(A|E_o) = 3/15$. Similarly, one can show that $P(B|E_o) = P(C|E_o) =P(D|E_o) =4/15$. So once again, it is better for you to switch (and due to symmetry, it doesn't matter to which).

Suppose you switch to door B. Now we are in round two. The host then open door D, which, surprise-surprise, turns out to have no prize behind it too. Now what do you do? What are the probabilities of staying versus switching?

One thing to be careful about is this: since at this point round one is already over, the conditional probabilities we calculated (i.e. $P(A|E_o)$, $P(B|E_o)$, ...) should now be considered unconditional:
$$ P(A) = 3/15$$
$$ P(B) = 4/15$$
$$ P(C) = 4/15$$
We also have the followings:
$$ P(D_o|A) = 1/2$$
$$ P(D_o|B) = 1/3$$
$$ P(D_o|C) = 1/2$$
$$ P(D_o) =\sum P(D_o|X)P(X) $$
$$= \frac 1 2 \times \frac 3 {15} + \frac 1 3 \times \frac 4 {15} + \frac 1 2 \times \frac 4 {15} +0 = \frac {29} {90}$$
Hence $P(A|D_o) = 9/29$, $P(B|D_o) = 8/29$ and $P(C|D_o) = 12/29$. So staying with door B is even worse than switching back to door A, and switching to the untouched door C is the best. Just keep switching!

How do you model the dynamics of asset prices? A self-reflection (1 of 3)

2013-11-17T19:54:00.003-08:00

Having been working on/studying mathematical finance for a while now, I feel like it would be useful to step back and look at one of the most important topics in this area: how does one model asset price dynamics?

Gauß
The canonical account would inevitably begin with Gauss. The Gaussian distribution (or the normal distribution, depending on your academic upbringing) is undeniable the most well-investigated probability distribution in human history. We don't need to go into all the details and properties of it, but it's good to be reminded of a number of facts regarding how the normal distribution is related to some other entities:

It is closely related to the phenomenon of diffusion;
The Central Limit Theorem says that (under some technical conditions) the sum of many i.i.d. random variables would converge to a normal distribution;
Brownian motion is mathematically described by Wiener process, which follows a Gaussian distribution.

I don't want to delve into historical investigation in whether Louis Bachelier is the first person to model financial time series with (arithmetic) Brownian motion, but it's fair to say that he is at least among the first ones who formalize it and apply it in derivative pricing. Later down the road, some would argue that arithmetic Brownian motion is prone to producing negative values and would propose other alternative processes (e.g. geometric Brownian motion for stock prices, O-U process for rates), but those would all be in the same spirit of arithmetic Brownian motion, save some transformations (e.g. taking the natural log as in geometric Brownian motion)

Our Checklist
Before proceeding further, let's compile a checklist that we will re-visit multiple times in this series:

Candidate: Brownian motion

Markovian
"Tractable"(-ish, depending on how you transform it)
Excess kurtosis = 0
Market completeness (if number of hedging instruments >= number of sources of randomness)

Since this is the first time the checklist is presented, we should go through each point:

A Markov process has no memory. It doesn't matter if the market went up 20%, up 3%, down 3%, or down 20% yesterday. Today's probability of movement is not affected in any way.
"Tractable" refers to a closed-form expression for the asset price process itself, or the price of derivatives with the asset as underlying.
Excess kurtosis is a measure of "tail effect"
Market completeness means a contingent claim can be fully hedged

The paradigm of thinking of asset price as following a Brownian motion is commonly known as the random walk hypothesis. It is well-known and widely used: it's the canonical/default view in the majority of quant finance textbooks; it is mentioned and discussed in "pop-finance" books such as A Random Walk Down Wall Street. But how realistic is it and what are the caveats?

The Balanced 1-0 Matrix Problem

2013-11-07T02:15:00.000-08:00

This post is about the Balanced 1-0 Matrix problem. Namely, for an n-by-n matrix (can be relaxed to n-by-m, but we will stick to square matrix here), how many legitimate assignments are there such that each row as well as column contains exactly n/2 zeros and n/2 ones?

You will find different approaches to solve this problem in the link, and one thing to note is how quickly the sequence grows: the numbers of possible assignments are 1, 2, 90, 297200... for 1-by-1, 2-by-2, 3-by-3, 4-by-4 matrices. The brute force approach is not very interesting, so we will only discuss the dynamic programming approach. The Matlab script can be found at the end.

The idea is first to have an easy way to store the arrangement. That is what stateArr does. As a 2-by-n array, it stores how many zeros and ones are still available to be allocated for each of the n columns. Then we have the systematic sweeping through the rows, meaning that we do a tree search starting from the first row downward. We iterate all the possible combinations (of course, they have to have n/2 zeros and n/2 ones) for a row, and compare these possible combinations against the stateArr to see if there are enough zeros and ones to be assigned. This is what the for loop within the DP_recursion is for.

Finally, the algorithm described above would have to visit every and all solutions unless we use a memoization trick. All partial solutions, as well as the corresponding stateArr, are stored in stateArrTable, and for every step the script would first check whether partial solution already exists. If so, use the stateArr as a signature/key to retrieve it; if not, and only if not, will it jump into recursion.

Unfortunately, even with dynamic programming, the computation is too intensive for Matlab for any n > 4.

function output = ZeroOneMat_DP(numCol)
% ((1, 2) (2, 1) (1, 2) (2, 1)) would be stored as
% [1, 2, 1, 2; 2, 1, 2, 1]
% First row is # zeros, second row is # ones
stateArr = numCol/2*ones(2,numCol);
stateArrTable = [];
partialAnsArr = [];
[output stateArrTable partialAnsArr] = DP_recursion(numCol, stateArr, stateArrTable, partialAnsArr, 0);
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [output stateArrTable partialAnsArr] = DP_recursion(numCol, stateArr, stateArrTable, partialAnsArr, numSol)
% Iterate all combinations
locationMat = nchoosek(1:numCol,numCol/2);
tempOutput = numSol;
for i = 1:size(locationMat,1)
    thisRow = zeros(1,numCol);
    thisRow(locationMat(i,:)) = 1;
    % Change stateArr
    stateArr_temp = stateArr;
    for j = 1:numCol
        if thisRow(j) == 0
            stateArr_temp(1,j) = stateArr_temp(1,j) - 1;
        elseif thisRow(j) == 1
            stateArr_temp(2,j) = stateArr_temp(2,j) - 1;
        end
    end
    % Check if there is negative values
    if min(min(stateArr_temp)) < 0

    elseif min(min(stateArr_temp)) >= 0 & max(max(stateArr_temp)) > 0
        % See if already in table
        [foundBool theAnswer] = findInTable(stateArr_temp, stateArrTable, partialAnsArr);
        if foundBool == 0 % not
            [tempOutput2 stateArrTable partialAnsArr] = DP_recursion(numCol, stateArr_temp, stateArrTable, partialAnsArr, tempOutput);
            % Save to partial answers
            stateArrTable = [stateArrTable; stateArr_temp];
            partialAnsArr = [partialAnsArr; tempOutput2 - tempOutput];
            tempOutput = tempOutput2;
        elseif foundBool == 1
            % find in table
            tempOutput = tempOutput + theAnswer;
        end
    elseif max(max(abs(stateArr_temp))) == 0 % successfully placed all numbers through the last row
        tempOutput = tempOutput + 1;
    end
end
output = tempOutput;
end

function [foundBool theAnswer] = findInTable(stateArr, stateArrTable, partialAnsArr)
foundBool = 0;
theAnswer = -1;
for i = 1:size(stateArrTable,1)/2
    if max(max(abs(stateArrTable(2*i-1:2*i,:)-stateArr))) == 0
        theAnswer = partialAnsArr(i);
        foundBool = 1;
        break
    end
end
end

Those damn cards...

2013-11-05T01:33:00.000-08:00

We are quickly approaching the end of the combo interview question bank series. So far we have discussed a variety of coin and dice problems. Yet another popular theme would be card games, specifically with poker decks. As compared to coins and dice, a poker deck contains two colours and four suits, giving rise to some interesting and complicated games and problems.

CF01
A deck consists of 3 red and 3 black cards. They are turned over one by one, and at any point you can make the call of the following card being red. If it turns out to be the case, you win. What is the optimal strategy?

CF01- Answer:
This is closely related to option pricing with binomial tree. Being able to call 'red' as you wish is equivalent to early exercise. It's trivial to show that without early exercise, the probability tree reads:
0.5000
0.3125 0.6875
0.1250 0.5000 0.8750
0.0000 0.2500 0.7500 1.0000
0.0000 0.5000 1.0000
1.0000 0.0000
corresponding to #Black card shown : #Red card shown
0:0
0:1 1:0
0:2 1:1 2:0
0:3 1:2 2:1 3:0
1:3 2:2 3:1
2:3 3:2
Remember, this is when there is no early exercise, meaning that you can only passively look at cards being flipped open and wait to see if the last card is red. To take early exercise into account, some nodes on the probability tree will be over-ridden, depending on the numbers of remaining cards on the node. Take the (1:2) node as an example. When 1 Black card and 2 Red cards are already shown, there are 2 Black cards and 1 Red card remaining in the deck. Hence the winning probability of early calling at this point is 1/3 = 0.333..., which is greater than 0.25. Hence the optimal strategy would require early calling at this node. Similar argument can be made regarding other nodes on the tree.

CF02
You draw a card from a deck and player B draws another. You win only if your draw has a higher number. What is the probability that you will win?

CF02- Answer:
P(numbers equal) = 3/51 = 1/17. Due to symmetry, the two parties have the same winning probability. Hence P(you win) = (1 - 1/17) / 2 = 8/17

See here for more interview questions/brainteasers

Those damn dice... (Part II)

2013-10-30T05:32:00.000-07:00

Last time we investigated questions that ask about the probability of a variety of dice rolling situations. This time we focus on games that involve dice.

DR04
You roll a six-sided die and a ten-sided die. If you guess the sum correctly you win an amount that equals the sum. What is the optimal guess?

DR04 - Answer:
First of all, think about what "optimal guess" means. It should maximize the expected payoff. Hence we can iterate a table:
Sum | 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# combination | 1 2 3 4 5 6 6 6 6 6 5 4 3 2 1
The product is maximized at sum = 11, which is the answer.

DR05
What is the expectation value of the product of rolling a six-sided die and a eight-sided die?

DR05- Answer:
Since the two rolls are independent events, E[A & B] = E[A] * E[B] = 15.75

DR06
The game is as follow: Roll a 12-sided die. You can either get the amount of the roll, or choose to roll two 6-sided dice, at which point you must receive the sum of the two dice. What is the game worth?

DR06- Answer:
Use backward induction. The expected payoff of the second round is 7. Hence you would go for round 2 only if the first round was less than or equals 6. Thus the payoff is
(7 + 8 + 9 + 10 + 11 + 12) / 12 + 0.5 * 7 = 8.25

DR06
What is the expectation value of the absolute difference of the points on two 6-sided dice?

DR06- Answer:
By simple state space counting, the answer is
(0 * 6 + 1 * 10 + 2 * 8 + 3 * 6 + 4 * 4 + 5 * 2) / 36 = 1.94

See here for more interview questions/brainteasers

Brainteaser: Expected Number of Trials

2013-10-11T23:27:00.002-07:00

The first part is the classic coin flipping question: For a fair coin, what is the expected number of trials in order to get the first Head? And the answer is 2 (see here for discussion and related problems).

Now, put the coin aside and consider instead a deck of 52 cards. What is the expected number of trials in order to get the first red card? Would it be less than, equal to, or larger than 2? In the coin case, one way to get the answer is to do the infinite sum

$$ E[H] = \sum_{i=1}^{\infty} \frac{1}{2^i} i $$

Now, along the same line we can have a finite series

$$ E[Red] = \sum_{i=1}^{52} P_i i$$

The only problem is that for this case the $P_i$'s are uglier. Just write down the first few terms and we will see

$$ E[Red] = \frac {26}{52}1 + \frac {26}{52} \frac {26}{51}2 + \frac {26}{52} \frac {25}{51} \frac {26}{50}3 + \sum_{i=4}^{52} P_i i$$

As you can see, some of the fractions are greater than 1/2 and some are less. Can we easily conclude if the finite sum is less than or greater than 2?

See here for more interview questions/brainteasers

Correlation Trading

2013-09-17T23:14:00.001-07:00

Correlation trading is something that hasn't yet been discussed on this blog. In principle, one can bet on the correlation(s) among n assets of one's choice. However most commonly correlation trading is done on an index. In other words, you are betting on the correlation of the constituents of the index. You can approach a bank and ask them to quote the strike of a correlation swap for you. Then you can sit back and let the bank worry about all the dirty hedging works.

There are sound reasons not to go down this route. First, what if you want to speculate on a basket that is not a common index? That may not be readily quotable at the bank. Secondly, the strike of the correlation swap quoted by the bank is likely to have a large spread buffer to compensate for the risk they take.

An alternative is to proxy the correlation swap. The technique is generally known as dispersion trading. It's a fancy name of long-shorting an index versus its components, or vice versa.

Straddles
The most primitive way of dispersion trading is to long the index straddle and short the constituents straddles (this would be betting on the correlation among the constituents to go up). In this case, each of the straddles proxy the specific variances (of the index or of the constituents), and the long-short portfolio of straddles in turn proxies the correlation. Such a simplistic scheme would of course leave a lot to be desired (e.g. the need to rehedge, see here).

Variance Swaps
Can we do better? Absolutely. It's almost a no-brainer: replace the straddles with variance swaps! So instead of longing the index straddle and shorting the constituents straddles, you long the index variance swap and short the constituents variance swaps to bet the correlation going up. Thanks to the characteristics of variance swaps, you don't have to worry about rehedging anymore. Sounds good, doesn't it? But there is still room for improvement. Although hedging is not necessary, the weights of the variance swaps have to be dynamically maintained. As an example, suppose our (value weighted) index I has only two components A and B. Initially the the index is made up of 50% each of A and B. You try to trade correlation by longing $100 notional of variance swap on I and shorting $50 notional of variance swaps on A and B each. Over time, the underlyings move and the weights of A and B become, say, 43% and 57% respectively. You'd have to dynamically re-balance your variance swap portfolio to keep in line with the changing weights.

Note: The above discussion assumes a value weighted index.

As a side note, the strike of the long index variance swap, short constituents variance swap portfolio would in general differ from a correlation swap quote. The reason is non-zero vol-of-vol. This paper gives an in-depth investigation into the matter.

Gamma Swaps
Knowing the shortcoming of dispersion trading using variance swaps (i.e. the need for reallocation), what alternative do we have? We can use a kind of swap that 'scales' itself according to the underlying level, a feature that is provided by gamma swaps. You long the index gamma swap and short the constituents gamma swaps to bet the correlation going up. Due to the structure of a gamma swap, your exposure to each index component would in fact automatically match the desired weightings as the underlyings change.

Common Option Strategies: The Greeks

2013-09-16T02:39:00.005-07:00

Naked, single option trading is not only risky, but also it is difficult to single out specific exposures. Hence option combo strategies are more common. But what strikes to choose? Hopefully this post would give you some hints.

Assumptions:
- Volatility smile = $0.15 - 0.1\times (K-K_{ATM}) + 0.1 \times (K-K_{ATM})^2 $
- Sticky strike
- Flat volatility term structure

Since $\Theta$ and Vega are similar to $\Gamma$, we skip presenting those. We are showing:
- Price
- Delta
- Gamma
- Vanna
- Volga
- Charm
- $ \frac {\partial}{\partial t} Vega$

Note:
$Vanna = \frac{\partial}{\partial S} \frac{\partial P}{\partial \sigma}$
$Volga = \frac{\partial^2 P}{\partial \sigma^2}$
$Charm = \frac{\partial}{\partial S} \frac{\partial P}{\partial T}$

Strangle

Short Bear Spread

Back Spread

Calendar Spread

Diagonal Spread

Butterfly

Broken Wing Butterfly

Iron Condor

Short Note: VIX Derivatives Pricing

2013-09-10T02:50:00.004-07:00

- Lest it has not been emphasized enough in the previous post, note that there is no straightforward no-arbitrage relationship to get VIX future price from sopt VIX.

- Perhaps one day, the market would be so well-developed and liquidly-traded that we could have a "Volatility Market Model." Until then, we will have to rely on more modest approaches.

- Why do people say that VIX options are priced off VIX future instead of spot? Well, for options on, say, FX or stock, there is no need to distinguish between pricing options off futures or spots because spot and future are strictly connected by no-arbitrage; for VIX, however, such connection is absent. If you start of with VIX spot, you first have to price the future contract before you can price the option. So it is better to directly use VIX future price as input when pricing option on VIX, so as to avoid "an extra layer of error."

- Currently the different approaches of pricing VIX derivatives are:
1. Modeling the instantaneous spot variance as a stochastic process
This is similar to modeling the short rate in interest rate derivatives pricing. Instantaneous spot variance, much like short rate, is neither observed nor tradable. Yet, they are rather well-studied and people are quite familiar with them, at least in the context of vanilla and exotic stock options. The idea is to pick your favorite model that describes the evolution of the instantaneous variance (say, Heston or 3/2 model). Calibrate it to the current volatility surface (considering VIX is a 30-day measure, you may want to calibrate only to the front and forthcoming month). Then use the calibrated parameters to price the VIX derivatives with numerical integration.

2. Modeling the instantaneous forward variance as a stochastic process
This is similar to the HJM framework in interest rate derivatives pricing. One example is the model proposed by Bergomi (see, for example, Bergomi's Smile Dynamics series).

3. Modeling VIX itself as a stochastic process
This differs from the previous two cases in the sense that VIX itself, which is observed, is modeled as a stochastic process.

Bottom line:
In addition to calibration efficiency, one important desirable feature is for the model to fit to both the SPX option prices and the VIX option prices.

Reference
Mencía and Sentana, 2012, Valuation of VIX Derivatives
http://www.bde.es/f/webbde/SES/Secciones/Publicaciones/PublicacionesSeriadas/DocumentosTrabajo/12/Fich/dt1232e.pdf
Wang and Daigler, 2008, The Performance of VIX Options Pricing Models: Empirical Evidence Beyond Simulation
http://www2.fiu.edu/~zwang001/Research/papers/VIX_Option_Pricing_FMA.pdf

VIX Futures Pricing

2013-09-05T23:57:00.002-07:00

Recently I came across a blog called macroption, which seems to carry some interesting materials.

The blogger specifically explains the difference between the term structure and futures curve of VIX, which can sometimes be confusing. To eliminate the ambiguity, we can use the full notation, with three arguments, to denote forward volatility: $ V(t,S,T) $ (which is similar to the full notation of LIBOR). $ V(t,S,T) $ is, of course, the forward $T-S$ day volatility starting at a future time $S$ as observed at time $t$. With this in mind, the term structure of VIX would be a graph of $V(0,0,T)$ against $T$; while the futures curve of VIX would be a graph of $V(0,S,S+\delta)$ against $S$ ($\delta$ in this case would be 30 days).

Now that we have clarified the notation, let's talk about VIX futures pricing and how convexity plays a role. $VIX^2$ is the fair strike of a variance swap, which can be replicated using vanillas in a model-independent manner (recap here). To express it in terms of expectation, we can write

$$ VIX_0^2 = E^Q_0[V(0,0,30)] $$

However, the VIX index and hence its related derivatives (futures and options) are quoted not in $VIX^2$ terms, but simply in $VIX$ itself (in other words, as we see that VIX is at 14.5, the fair strike of a variance swap is 0.145 * 0.145 = 0.021). If we consider a future contract, in general if the underlying is commodities, FX or stocks we can simply apply the no-arbitrage rule and arrive at the future price using the appropriate funding/dividend/storage/convenience yields. However, these simply don't exist for VIX, which is not a tradable entity. So, we have to rely on

$$ F(0,T) = E^Q_0[\sqrt{VIX_T^2}] \neq \sqrt{E^Q_0[VIX_T^2]}$$

This is where the convexity adjustment comes in (in fact it should be a 'concavity adjustment', as the square root function $\sqrt{x}$ is concave in $x$). We can approximate the future price as

$$ F(0,T) \approx \sqrt{E^Q_0[VIX_T^2]} - \frac{var^Q_0[VIX_T^2]}{8[E^Q_0[VIX_T^2]]^{2/3}}$$

(see, for example, Zhu and Lian, An Analytical Formula for VIX Futures and its Applications, 2010) The negative sign of the higher order correction term clearly shows concavity.

Life on the Trading Floor

2013-08-29T01:14:00.002-07:00

Interesting read for those who want to learn more about how things are on the trading floor (even though a bit dated).

http://www.trade2win.com/articles/752-day-life-forex-spot-desk-trader-part-1-a
http://www.trade2win.com/articles/754-day-life-forex-spot-desk-trader-part-2-a

MathJax

2013-08-01T05:09:00.001-07:00

This is just to try out the cool things that are made possible by MathJax. To enable it in Blogger, follow the instructions here and here.

Schrodinger Equation
$$ i \hbar \frac {\partial } {\partial t} \Psi = \hat{H} \Psi $$

Black-Scholes Call Price
$$ C = e^{-d \tau} S N (d_1) - e^{-r \tau} K N(d_2)$$

Fourier Transform
$$ \hat{f} (k) = \int^{+\infty}_{-\infty} f(x) e^{-2 \pi i k x} dx $$

It also supports inline equations (e.g. $ E = m c ^2$). No more excuse to avoid typing up some nice formulae!

A Follow-up on "A Volatiltiy Trading Strategy That is (Almost) Too Good to be True"

2013-07-31T00:15:00.001-07:00

This is a follow-up on the previous post. As I mentioned, backtest shows that the VIX term structure based strategy has faltered since late 2010. The strategy has seemingly lost its magic between late 2010 and late 2012, as reflected in the fluctuating backtest balance. What is interesting is perhaps that starting from late 2012, "doing the opposite" seems to be a working strategy:

Recall that in the original strategy, we hold VXX when recent realized vol is greater than VIX; hold XIV when recent realized vol is less than VIX. So the opposite strategy would be to hold XIV when recent realized vol is greater than VIX; hold VXX when recent realized vol is less than VIX. Of course, looking at backtest results in this manner is bordering data-snooping, but it does get us thinking: what kind of regime change has occurred in the volatility market since the end of 2012? When would it flip back again?