Econometrics Beat: Dave Giles' Blog

Variance Estimators That Minimize MSE

2013-05-21T16:08:00.002-07:00

In this post I'm going to look at alternative estimators for the variance of a population. The following discussion builds on a recent post, and once again it's really directed at students. Well, for the most part.

Actually, some of the results relating to populations that are non-Normal probably won't be familiar to a lot of readers. In fact, I can't think of a reference for where these results have been assembled in this way previously. So, I think there's some novelty here. But we'll get to that in due course.

I can just imagine you smacking your lips in anticipation!
To get things started, let's suppose that we're using simple random sampling to get our n data-points, and that this sample is being drawn from a population that's Normal, with a (finite) mean of μ and a (finite) variance of σ². Let x* denote the sample average: x* = (1 / n)Σx_i, where the range of summation here (and everywhere below) is from 1 to n.

Let's consider the family of estimators of σ²:

s_k² = (1 / k)Σ[(x_i - x*)²],

where "k" is a positive number. Now, to be very clear, I'm not suggesting that we should necessarily restrict our attention to estimators that happen to be in this family - especially when we move away from the case where the population is Normal. I'll come back to this point towards the end of this post.

Clearly, if k = (n - 1), we just have the usual unbiased estimator for σ², which for simplicity we'll call s². If k = n, we have the mean squared deviation of the sample, s_n², which is a downward-biased estimator of σ².

However, we all know that unbiasedness isn't everything!

Often, we look at our potential estimators and evaluate them in the context of some sort of loss function. If this loss function is quadratic, then the expected loss (or "risk") of an estimator is its Mean Squared Error (MSE). You'll recall that the MSE of an estimator is just the sum of its variance and the square of its bias.

Let's compare the unbiased estimator, s², and the biased estimator, s_n², in terms of MSE.

Because we're using simple random sampling from a Normal population, we know that the statistic c = [(n - 1)s² / σ²] follows a Chi-square distribution with (n - 1) degrees of freedom. So, s² = (σ² / (n - 1))χ²_{(n - 1)}. We also know that the mean of a Chi-square random variable equals its degrees of freedom; and its variance is twice those degrees of freedom. So, E[s²] = σ², and Var.(s²) = 2σ⁴ / (n - 1).

The first of these two results also holds if the population is non-Normal, but the second result doesn't hold, as I discussed in this earlier post.

Next, noting that s_n² = (n - 1)s² / n, it follows that;

E[s_n²] = [(n - 1) / n]σ² ;
Bias[s_n²] = E[s_n²] - σ² = - (σ²/ n)

and
Var.[s_n²] = 2σ⁴(n - 1) / n².

So, the MSE of s_n² is given by the expression,

MSE(s_n²) = Var.[s_n²] + (Bias[s_n²])² = σ⁴ (2n - 1) / n².

Because s² is unbiased, its MSE is just its variance, so MSE(s²) = 2σ⁴ / (n - 1). Noting that MSE(s_n²) = [(n - 1) / n] MSE(s²) - (σ⁴ / n²), we see immediately that MSE(s_n²) < MSE(s²), for any finite sample size, n. This can be seen in the following chart, drawn for σ²= 1. (Of course, the two estimators, and their MSEs coincide when the sample size is infinitely large.)

Although s_n² dominates s², in terms of having smaller MSE for all possible sample values, and all (finite) sample sizes, it's still not the best we can do within the family of estimators we're considering! Let's go back to this class of estimators and ask, "what value of k will lead to the estimator with the smallest possible MSE for all members of this class?"

We can answer this question by using a bit of simple calculus. We'll write out the expression for the MSE of s_k², and it will be some function of "k". Then we'll differentiate this function with respect to "k", set the derivative to zero, and then solve for the value of k (say k*). We should then check the sign of the second derivative to make sure that k* actually minimizes the MSE, rather than maximizes it!

So, here goes ........

Notice that we can write a typical member of our family of estimators as

s_k² = (1 / k)Σ[(x_i - x*)²] = [(n - 1) / k]s² .

So, using the results that E[s²] = σ², and Var.(s²) = 2σ⁴ / (n - 1), we get:

E[s_k²] = [(n - 1) / k]σ² ;
Bias[s_k²] = E[s_k²] - σ² = [(n - 1 - k) / k]σ²;
and
Var.[s_k²] = 2σ⁴(n - 1) / k².

The MSE of s_k² is given by the expression,

M = MSE(s_k²) = Var.[s_k²] + (Bias[s_k²])² = (σ⁴ /k²)[2(n - 1) + (n - 1 - k)²].

Differentiating M with respect to "k", and setting this derivative to zero, yields the solution, k* = (n + 1). You can easily check that k* minimizes (not maximizes) M.

So, within this family that we've been considering, the minimum MSE (MMSE) estimator of σ² is the estimator,

s_n+1² = (1 / (n + 1))Σ[(x_i - x*)²] .

This is certainly a well-known result. It also extends naturally to the situation where we're estimating the variance of the (Normally distributed) error term in a linear regression model. In that case the MMSE of this variance is (1 / (n - p + 2))Σe_i², where e_i is the i^th OLS residual, and p is the number of coefficients in the model.

So far, so good!

Now, let's connect with the earlier post that I mentioned above, and see how all of this works out if we have a population that's non-Normal. We'll retain the simple random sampling, though. As was discussed in that post, in general the variance of s² is given by:

Var.[s²] = (1 / n)[μ₄ - (n - 3)μ₂² / (n - 1)] = V, say.

Here, μ₂ and μ₄ are the second and fourth central moments of the population distribution. Recall that μ₂ is the population variance, and for the result immediately above to hold the first four moments of the distribution must exist.

Let's extend this variance expression to members of the family, s_k². Then we'll work out the expression for the MSE of such estimators for a non-normal population. Finally, this will allow us to derive the MMSE estimator in this family for any population distribution - not just for the Normal population that we dealt with earlier in this post.

Once again, we'll begin by using the fact that we can write:

s_k² = (1 / k)Σ[(x_i - x*)²] = [(n - 1) / k]s² .

The estimator, s², is still unbiased for σ² even in the non-Normal case, so we still have the results:

E[s_k²] = [(n - 1) / k]σ² ; and Bias[s_k²] = [(n - 1 - k) / k]σ²= [(n - 1 - k) / k]μ₂

as before.

Also,

Var.[s_k²] = [(n - 1) / k]² Var.[s²] = [(n - 1) / k]²(1 / n)[μ₄ - (n - 3)μ₂² / (n - 1)] ,

and so the MSE of s_k² is given by:

MSE(s_k²) = [(n - 1 - k) / k]²μ₂² + V[(n - 1) / k]²,

where V = Var.[s²] is defined above.

Going through the calculus once again, it's easy to show (I used to hate that statement in textbooks) that the value of "k" for which the MSE is minimized is:

k** = (n - 1)(V + μ₂²) / μ₂²,

where V = (1 / n)[μ₄ - (n - 3)μ₂² / (n - 1)].

Check: For the Normal distribution, μ₄ = 3μ₂², and so k** = (n + 1) = k*, as before.

Having gone to all of this effort, let's finish up by illustrating the optimal k** values for a small selection of other population distributions:

Uniform, continuous on [a , b]

μ₂ = (b - a)²/ 12 ; μ₄ = (9 / 5)
k** = (n - 2) + (3 / n) +1296(n - 1) / [5n(b - a)⁴]

Standard Student's-t, with v degrees of freedom

μ₂ = v / (v - 2) ; μ₄ = 3v² / [(v - 2)(v - 4)]
k** = (n - 2) + (3 / n) +3(n - 1)(v - 2) /[n(v - 4)] ; for v > 4

χ², with v degrees of freedom

μ₂ = 2v ; μ₄ = 3(v + 4) / v
k** = (n - 2) + (3 / n) +3(n - 1)(v + 4)] / (4nv³)

Exponential, with mean θ

μ₂ = θ² ; μ₄ = 9θ⁴
k** = (n² + 7n - 6) / n

Poisson, with parameter λ

μ₂ = λ ; μ₄ = λ(3λ + 1)
k** = (n - 2) + (3 / n) +(n - 1)(3λ + 1) / (nλ³)

Now, a final word of caution.

Yes, setting k = k** in the case of each of these non-Normal populations, and then estimating the variance by using the statistic, s_k² = (1 / k)Σ[(x_i - x*)²], will ensure that your "estimator" is MMSE within this particular family of estimators. However, this doesn't mean to say that it's the "best", or even a feasible, estimator to use.

For instance, consider the last example where the population is Poisson. In that case, the population mean and variance are both λ. The MLE for λ is the sample average, x*. Therefore, x* is also the MLE for the population variance. Actually, x* is the "minimum variance unbiased" (MVUE) estimator of λ. The statistic s² is also an unbiased estimator of λ, but it is inefficient relative to x*. So x* dominates s² in terms of MSE. See here for a nice discussion.

If we were to try and implement our MMSE estimator of the variance in this case, we'd be trying to estimate λ. However, k** is a function of λ. We'd need to know the population variance in order to obtain the MMSE estimator of that parameter! You can see that the same issue applies to the Student's-t and χ² examples given above but it's not an issue with the other two examples.

Sometimes, MMSE estimators simply aren't "feasible". They're functions of the unknown parameters we're trying to estimate. They're not really estimators, at all!

Camp(s) Econometrics

2013-05-19T14:09:00.003-07:00

The New York Camp Econometrics VIII was held in Bolton Landing, NY, last month. I recall Badi Baltagi (one of the Camp Econometrics organisers) telling me about this great annual event a few years ago. The Texas Econometrics 2013 was held in Lost Pines back in February. This was the 18^th Camp for the group in Texas.

I also seem to recall that there used to be another regular Camp Econometrics in Southern California some years ago. If my neurons are still firing in the right order, I believe that Denis Aigner was one of the leaders of that venture.

Back to the NY Camp:

"This event is a gathering of econometricians and empirical economists whose successful goal is to: (1) Bring together a group of econometricians/empirical economists and guests of host universities to discuss issues in econometrics, both applied and theoretical; (2) Present papers for comments by participants; (3) Stimulate student interest in econometrics; (4) Help students develop their technical presentation skills by encouraging the students of host universities to participate in the meetings and present papers."

Events like the Camp(s) Econometrics, and The Econometric Game, in the Netherlands, really are great ventures!

Cookbook Econometrics - Reprise

2013-05-18T20:28:00.001-07:00

A few days ago I was looking at my copy of Econometric Foundations, written by Ron Mittelhammer, George Judge, and Doug Miller. It's an excellent book, by the way.

I noticed, for the first time, that on p.xxviii of the Preface they have the following to say, under the heading of "A Comment":

"Many view econometrics as a potpourri or bag of tricks, and the cookbook metaphor for econometrics textbooks has become commonplace. Unfortunately, this philosophy can produce analysts who know a list of econometric recipes but who have insufficient understanding of which techniques to apply in a given situation or how to interpret the results of an application properly. As the inventory of econometric procedures has grown, the importance of understanding when it is appropriate to apply each econometric procedure, as well as knowing the appropriate interpretation of the results, has grown more than proportionately. The number of reference works that describe the growing inventory has expanded pari passu. These reference works will be accessible to the well-trained analyst who has mastered the basic philosophy and principles on which econometrics is founded. However, analysts who have done little more than memorize the recipes in conceptual econometric cookbooks will find the growing literature on new econometric methods impenetrable. Our goal is for you to be able to determine or create the econometric procedures that are applicable to your problem and then be able to apply them empirically and interpret the results appropriately. This is what this book is about, and this is what we think modern graduate econometrics instruction should be about."

I don't know Doug Miller personally, but I do know and respect Ron and George, and so I wasn't surprised to find myself punching the air and ~~shouting~~ saying, "YES!"

I had similar things to say in a post couple of years ago, and I won't repeat them all here.

Yes, I enjoy helping readers by providing "how-to-do-it" posts (such as this one on testing for Granger causality). Judging by the feedback, these posts appear to be widely appreciated - thanks! However, I'm just as committed to fostering a clear understanding of:

The assumptions that are needed to get our standard econometric results
The connections between many of the "tools" that we use
How to approach problems we haven't met before - in a sensible way
Why some econometric tools work better than others
The history and intuition that provide the foundation for much of what we do in econometrics
~~Motherhood and apple pie~~
The crucial importance of data quality

O.K. - enough harping on for now!

I Know What You Did Last Summer!

2013-05-18T07:50:00.002-07:00

O.K., I know that I stole that title! It was absolutely blatant.

The other day, some colleagues and I were discussing the issue of students (including our own offspring), and their summer jobs - or no jobs, as the case may be. I'm not passing judgement here in what follows, by the way.

When I was a student I had to earn enough money over the summer to live off for the rest of the year. That's just the way it was. Period! My parents were great - I could live at home over the summer at no cost to me. But that was it. They lived in a very small rural town in New Zealand, a long way from where I attended university. The upside of this was that, being a rural area (in the mid/late 1960's through to the early 1970's), there were some seriously good work opportunities.

So, what did I do each summer?

Well, I did lots of different things, actually. They ranged from career-related, to "I don't care how tough this is - I just need the bucks!"

Let's look at some of these jobs. They were interesting, to say the least:

Summers of '66, '67 & '68: Sheep-shearing gang & harvesting hay and grain
Summer of '69: Actuarial office of a major life insurance company
Summer of '70: Research Section of the N.Z. central bank - the Reserve Bank of New Zealand

In retrospect, I was very lucky. I got to experience some work situations that were way outside what most students get to see.

Didn't get to Woodstock - not because I was making hay, but because I was in class - remember, I was in the Southern Hemisphere. The "Summer of Love" was my "Winter of Woe".
Decided that an actuarial life wasn't for me - financially, a bad decision!
Got to be one of 3 people working on the construction of the first version of the Reserve Bank's macroeconometric model. You'd be right if you guessed that I found that really exciting!

That last summer job led to a scholarship for my Masters-level studies, and ultimately to a very rewarding period at the RBNZ before I opted for an academic life in Australia.

You just never know where that summer job is going to lead to!

What's the Variance of a Sample Variance?

2013-05-17T09:44:00.001-07:00

This post is really pitched at students who are taking a course or two in introductory economic statistics. It relates to a couple of estimators of the variance of a population that we all meet in such courses - plus another one that you might not have met. In addition, I'll be emphasising the fact that some "standard" results depend crucially on certain assumptions. Not surprisingly - but not always made clear by instructors and text books.

To begin with, let's consider a standard problem. We have a population that is Normal, with a mean of μ and a variance of σ². We take a sample of size n, using simple random sampling. Then we form the simple arithmetic mean of the sample values: x* = Σx_i , where the range of summation (here and everywhere below) is from 1 to n.

Under my assumptions, we know that the sampling distribution of x* is N[μ , (σ² / n)]. The normality of the sampling distribution follows from the Normality of the population, and the fact that x* is a linear function of the data. The variance of the sampling distribution stated above is correct only because simple random sampling has been used.

Now, let's get to what I'm really interested in here - estimating σ². We all learn that the mean squared deviation of the sample, σ^*2 = (1 / n)Σ[(x_i - x*)²], is a (downward-) biased estimator of σ². If we allow for the fact that we've actually lost one degree of freedom by estimating μ using x*, then an unbiased estimator of σ² is s² = (1 / (n - 1))Σ[(x_i - x*)²].

O.K., now what does the sampling distribution of s² look like?

Well, under the assumptions I've made, including the Normality of the population, s² has a sampling distribution that is proportional to a Chi-square distribution. More specifically, the statistic, c = [(n - 1)s² / σ²] is Chi-square with (n - 1) degrees of freedom.

[As an aside, s² and x* are independently distributed if and only if the population is Normal. The "only if part" of the latter statement is due to the Irish statistician, Geary - see here.]

So, we now know something, indirectly, about the sampling distribution of s², and we know that E[s²] = σ². What is the variance of σ²?

Because we're assuming a Normal population, implying that the statistic I've called "c" follows a Chi-square distribution, we can use the result that the variance of a Chi-square random variable equals twice its degrees of freedom.

Re-arranging the formula for "c", we can write: s² = cσ² / (n - 1).

Then, Var.(s²) = {[σ² / (n - 1)]²Var.(c)} = {[σ⁴ / (n - 1)²]2(n - 1)} = 2σ⁴ / (n -1).

[As another aside, the mean of a Chi-square random variable equals its degrees of freedom, so applying this result to "c" and re-arranging, we immediately get the result that E[s²] = σ². However, we know this already, and this result holds even if the data are non-Normal.]

Now, this is as far as things usually go in an introductory economic statistics course. To sum up:

E[s²] = σ²
c = [(n - 1)s² / σ²] ~ χ²_{(n - 1)}
Var.[s²] = 2σ⁴ / (n -1)

Notice that Var.(s²) vanishes when n grows very large. This, together with the first above, implies that s² is a (mean-square) consistent estimator of σ².

Unfortunately, students often don't realize that the second and third of these results rely on both simple random sampling and the Normality of the population.

A thoughtful student will notice that the first result holds even if the data are non-Normal, and will ask, "what's the variance of s² if the population isn't Normal?" That's a good question!

To answer it, let's introduce an important concept - the "moments" of a probability distribution. Let X be a random variable. Then E[X^k] is called the k^th "raw moment" (or, moment about zero) of the distribution of X. (Here, "k" is a positive integer, but more generally we can allow k to be negative, or a fraction.) Let's denote the k^th such moment by μ'_k. So, the first raw moment is just the population mean. That is, μ'₁ = μ.

Then consider the quantities, μ_k = E[(X - μ)^k], for k = 1, 2, 3,.......... We call these the "centered moments" of the distribution of X. You'll notice that μ₂ is just the population variance. The third and fourth centered moments are used (together with μ₂) to construct measures of skewness and kurtosis, but that's another story.

By the way, there's an important detail. The expectations involved in the construction of the moments require forming an integral. If that integral diverges, the corresponding moment isn't defined. or instance, the k^th moment for a Student's-t distribution with v degrees of freedom exists only if v > k. In the case of the Cauchy distribution (which is just a Student's-t distribution with v = 1), none of the moments exist!

Alright - back to the question in hand! What is the variance of s² if the population is non-Normal? The answer, in the case of simple random sampling, is:

Var.(s²) = (1 / n)[μ₄ - μ₂²(n - 3) / (n -1)] .

If the population is Normal, then μ₄ = 3σ⁴, and μ₂² = σ⁴. So, we get Var.(s²) = 2σ⁴ / (n - 1), in this case.

Notice that this more general expression for Var.(s²) also vanishes as n grows. So, a pair of sufficient conditions for the mean-square consistency of s² (as an estimator of σ²) is:

The data are obtained using simple random sampling; &
At least the first 4 moments of the population distribution exist.

We can easily work out the expressions for Var.(s²) in the case where the population follows some other distributions that you may have heard about. Here are just a few illustrative results:

Uniform, continuous on [a , b]

μ₂ = (b - a)²/ 12 ; μ₄ = (9 / 5)
Var.(s²) = [1296(n - 1) - 5(n - 3)(b - a)⁴] / [720n(n - 1)]

Standard Student's-t, with v degrees of freedom

μ₂ = v / (v - 2) ; μ₄ = 3v² / [(v - 2)(v - 4)]
Var.(s²) = [2v²(2v - 5)] / [n(v - 2)²(v - 4)] ; for v > 4

χ², with v degrees of freedom

μ₂ = 2v ; μ₄ = 3(v + 4) / v
Var.(s²) = [3(n - 1)(v + 4) - 4(n - 3)v³] / [n(n - 1)v]

Exponential, with mean θ

μ₂ = θ² ; μ₄ = 9θ⁴
Var.(s²) = [2(4n - 3)θ] / [n(n - 1)]

Poisson, with parameter λ

μ₂ = λ ; μ₄ = λ(3λ + 1)
Var.(s²) = 2λ² / (n - 1) + (λ / n)

Keep in mind that in each of the cases, the sampling distribution of c = [(n - 1)s² / σ²] will no longer be a χ² distribution! Given our assumption of simple random sampling, you should be able to convince yourself that the asymptotic sampling distribution of "c" will be Normal.

References

Cho, E. & M. J. Cho, 2008. Variance of sample variance. Proceedings of the 2008 Joint Statistical Meetings, Section on Survey Research Methods, American Statistical Association, Washington DC,1291-1293.

Geary, R. C. (1936). The distribution of the Student's ratio for the non-normal samples. Supplement to the Journal of the Royal Statistical Society, 3, 178-184.

Top Economics Blogs

2013-05-15T19:41:00.001-07:00

Nice to be on the list - thanks, INOMICS!

What's Your Favourite Estimator?

2013-05-12T14:57:00.000-07:00

It's interesting to dwell on the popularity of different estimators that econometricians use. Some estimators are "in vogue" for a period, and then give way to others as new developments come along. Different topics have captured the attention of theoreticians and practitioners alike at different times in history.

Here's a Google Ngram showing the extent to which some familiar estimators for simultaneous equations models have been mentioned in books since 1960:

Not too surprisingly, good old OLS just goes on and on:

I was going to include the GMM estimator in these plots, but this acronym has meanings other than the obvious one that comes to mind. So, the results would have been misleading. To be safe, let's use the full phrase Generalized Method of Moments and allow for case sensitivity:

Interestingly, the phrase appeared in some books before the publication of Hansen's classic 1982 paper.

Flowers for Mom - From Quandl

2013-05-12T09:58:00.000-07:00

Today being Mothers' Day in many parts of the world, I thought that flowers would be appropriate. Well, a price index for (Gardens, Plants, and) Flowers. Specifically, a harmonized price index for these goods for 27 European Union countries.

I retrieved the monthly data for the period January 2006 to March 2013 from Quandl.com - a really nice resource that I posted about recently. As well as downloading the data in various formats, reading the data from R, etc., you can also embed an interactive chart of the data directly into a document such as this one, and make the data visible to viewers. (Just click on the graph.) For instance:

As this price is not seasonally adjusted, but obviously has a pronounced seasonal pattern, I thought I'd use this post to illustrate the basics of seasonal adjustment. What I'm going to do is take you through the steps associated with a bare-bones version of the ratio-to-moving-average method. This is something that I show students if I'm teaching an introductory descriptive economic statistics course.

Then, I'll show you how close the results are to the ones that you get if you seasonally adjust the data using the full-blown Census X-13 method that is employed by most statistical agencies world-wide.

Let the price index at time 't' be denoted by P_t, and let's assume that this series is made up of the product of trend (T_t), cycle (C_t), and irregular(I_t) components. (In fact, when I use the X13 method and allow it to decide whether the series is multiplicative or additive in its components, it selects the multiplicative model I'm using here.)

So, taking natural logarithms, we have:

ln(P_t) = ln(T_t) + ln(C_t) + ln(I_t) .

The rudimentary ratio-to-moving-average method involves the following steps. The column labels refer to the associated Excel workbook and csv file that are available on the data page of this blog:

First, we take an unweighted 12-month arithmetic moving average of the ln(P_t) data. (Column D.) Averaging the data smooths the series. A 12-month average should smooth out the seasonal movements, as well as any irregular movements.
There's a slight problem - it's to do with the "timing" of the observations in Column D. It arises because there's an even number (12) of months in a year. Conceptually, the first figures in Column D ( 4.48255) should be located half way between the June and July months if it's to be at the middle of the year. Right now, it's half a month out of alignment. This problem also arises if we're seasonally adjusting quarterly time-series data, because 4 is an even number too. (There's also an even number of weeks in a year - but I digress!)
To re-align the data, we now take an unweighted 2-period arithmetic moving average of the numbers in Column D. The results appear in Column E. Averaging two "out-of-alignment" numbers shifts them by half a month, and bingo, they're now lined up with the appropriate dates! We call the resulting series the "Centered Moving Average".
If we've done things properly there should be 6 values "missing" at the start of the series in Column E, and 6 missing at the end. (Two and two, if we had quarterly data.)
What have we achieved by this? Well, the data in Column E represent what is left of the ln(P_t) series after we've smoothed away the Seasonal and Irregular components. That is, they represent the combined ln(Trend) and ln(Cycle) components.
Next, we subtract the Centered Moving Average series (Column E) from the ln(P_t) data in Column C. This gives us (in Column F) the ln(Seasonal) and ln(Irregular) components.
Then, we take the arithmetic mean of all of the July month values in Column F. This gives us a single, common "seasonal factor" for that month. We then do the same with all of the August month values in that column, and so on. The results appear in Column G. Notice that we're implicitly assuming that the seasonal factors are going to be stable over time.
We're nearly there! If we add up the 12 seasonal factors they should sum to zero over the full year. Seasonality, by definition, is an intra-year phenomenon. Let's CHECK if we have this result. You can see in the workbook that they actually sum to 0.00178098. Not bad, but not good enough!
We apportion this discrepancy across the 12 seasonal factors by subtracting (0.00178098/12) from each of the numbers in Column G. The results, which are the final seasonal factors, appear in Column H. Notice that these factors are repeated, year after year.
We can now seasonally adjust the ln(P_t) series. We subtract these seasonal factors from the data in Column C. The results appear in Column I.
Taking the exponential of the Column I series gives us the seasonally adjusted series for P_t itself, as in Column J.

geometric averages

arithmetic

averages

dividing

subtracting

Ratio

moving

average

Next, I used Eviews to apply the X-13 seasonal adjustment method to P_t. Once you are viewing the series, you select "Proc", and then "Seasonal Adjustment", and go from there. Here's the original price index and its seasonally adjusted counterpart:

The EViews workfile that I used is on the code page for this blog. In that file there are actually three seasonally adjusted versions of the price index. The series called "X13" will be self-explanatory; the series called "Manual" was obtained using the basic steps outlined above, as shown in the Excel workbook; and the series called "RMA" was obtained using the "moving average" option under the "seasonal adjustment" procedure in EViews. The latter series should be almost identical to "Manual", and indeed it is. More on this below.

Now, how similar is my rudimentary seasonally adjusted series, "Manual", to the one obtained using X-13? Here's a scatter-plot of the two series - it's virtually a 45-degree line.

This is confirmed by the following OLS regression, and simple correlations:

You can see that there is almost a perfect correlation between my seasonally adjusted series, and both "RMA" and "X13":

The rudimentary ratio-to-moving average seasonal adjustment procedure that I went through above isn't always going to provide results that are this close to the ones that you get when you use X-13. There are several reasons for this:

X-13 can allow for outliers in the data. Our basic method ignores this possibility.
X-13 can allow for a seasonal pattern that "evolves" over the cycle, or over time. Our basic method assumes a stable seasonal pattern.
X-13 can allow for the fact that different months have different numbers of "trading days", and for "holiday effects", such as the moving dates for Easter. These effects are ignored in our basic method.
X-13 can deal with "end-point" effects that arise at the beginning and end of the sample, where values can't be computed for the centered moving averages. Our basic method doesn't take this into account.

However, in many cases, very similar seasonally adjusted series are obtained. This is very comforting for those of us who teach this material. You can take students through the rudimentary steps that I've outlined, and they can generate very convincing seasonally adjusted time-series.

Oh yes - don't forget those flowers for Mothers' Day!

New Paper Published

2013-05-10T14:12:00.000-07:00

A paper of mine appears in the latest issue of the Chilean Journal of Statistics. The paper is titled, "Exact asymptotic goodness-of-fit testing for discrete circular data with applications.

I've posted previously about this general topic, here, here and here.

R is His Friend

2013-05-09T10:14:00.000-07:00

Marcus Beck has a nice (& relatively new) blog called R is My Friend. You can guess that his posts relate to the use of R.

I particularly liked his piece on the use of the XML package in R to mine data from the internet; and his post on using the integrate function in R, even when the anti derivative has no closed-form solution.

Grad. student readers will also like his post, How Long is the Average Dissertation.

My own take on a related question can be found here.

Robust Standard Errors for Nonlinear Models

2013-05-08T11:52:00.002-07:00

André Richter wrote to me from Germany, commenting on the reporting of robust standard errors in the context of nonlinear models such as Logit and Probit. He said he 'd been led to believe that this doesn't make much sense. I told him that I agree, and that this is another of my "pet peeves"!

Yes, I do get grumpy about some of the things I see so-called "applied econometricians" doing all of the time. For instance, see my Gripe of the Day post back in 2011. Sometimes I feel as if I could produce a post with title almost every day!

Anyway, let's get back to André's point.

The following facts are widely known (e.g., check any recent edition of Greene' text) and it's hard to believe that anyone could get through a grad. level course in econometrics and not be aware of them:

In the case of a linear regression model, heteroskedastic errors render the OLS estimator, b, of the coefficient vector, beta, inefficient. However, this estimator is still unbiased and weakly consistent.
In this same linear model, and still using OLS, the usual estimator of the covariance matrix of b is an inconsistent estimators of the true covariance matrix of b. Consequently, if the standard errors of the elements of b are computed in the usual way, they will inconsistent estimators of the true standard deviations of the elements of b.
For this reason,we often use White's "heteroskedasticity consistent" estimator for the covariance matrix of b, if the presence of heteroskedastic errors is suspected.
This covariance estimator is still consistent, even if the errors are actually homoskedastic.
In the case of the linear regression model, this makes sense. Whether the errors are homoskedastic or heteroskedastic, both the OLS coefficient estimators and White's standard errors are consistent.

Moreover, in the case of a model that is nonlinear in the parameters:

The MLE of the parameter vector is biased and inconsistent if the errors are heteroskedastic (unless the likelihood function is modified to correctly take into account the precise form of heteroskedasticity).
This stands in stark contrast to the situation above, for the linear model.
The MLE of the asymptotic covariance matrix of the MLE of the parameter vector is also inconsistent, as in the case of the linear model.
Obvious examples of this are Logit and Probit models, which are nonlinear in the parameters, and are usually estimated by MLE.

I've made this point in at least one previous post. The results relating to nonlinear models are really well-known, and this is why it's extremely important to test for model mis-specification (such as heteroskedasticity) when estimating models such as Logit, Probit, Tobit, etc. Then, if need be, the model can be modified to take the heteroskedasticity into account before we estimate the parameters. For more information on such tests, and the associated references, see this page on my professional website.

Unfortunately, it's unusual to see "applied econometricians" pay any attention to this! They tend to just do one of two things. They either

use Logit or Probit, but report the "heteroskedasticity-consistent" standard errors that their favourite econometrics package conveniently (but misleading) computes for them. This involves a covariance estimator along the lines of White's "sandwich estimator". Or, they
estimate a "linear probability model" (i.e., just use OLS, even though the dependent variable is a binary dummy variable, and report the "het.-consistent standard errors".

If they follow approach 2, these folks defend themselves by saying that "you get essentially the same estimated marginal effects if you use OLS as opposed to Probit or Logit." I've said my piece about this attitude previously (here, here, here, and here), and I won't go over it again here.

My concern right now is with approach 1 above.

The "robust" standard errors are being reported to cover the possibility that the model's errors may be heteroskedastic. But if that's the case, the parameter estimates are inconsistent. What use is a consistent standard error when the point estimate is inconsistent? Not much!!

This point is laid out pretty clearly in Greene (2012, pp. 692-693), for example. Here's what he has to say:

"...the probit (Q-) maximum likelihood estimator is not consistent in the presence of any form of heteroscedasticity, unmeasured heterogeneity, omitted variables (even if they are orthogonal to the included ones), nonlinearity of the form of the index, or an error in the distributional assumption [ with some narrow exceptions as described by Ruud (198)]. Thus, in almost any case, the sandwich estimator provides an appropriate asymptotic covariance matrix for an estimator that is biased in an unknown direction." (My underlining; DG.) "White raises this issue explicitly, although it seems to receive very little attention in the literature.".........."His very useful result is that if the QMLE converges to a probability limit, then the sandwich estimator can, under certain circumstances, be used to estimate the asymptotic covariance matrix of that estimator. But there is no guarantee the the QMLE will converge to anything interesting or useful. Simply computing a robust covariance matrix for an otherwise inconsistent estimator does not give it redemption. Consequently, the virtue of a robust covariance matrix in this setting is unclear."

Back on July 2006, on the R Help feed, Robert Duval had this to say:

"This discussion leads to another point which is more subtle, but more important...

You can always get Huber-White (a.k.a robust) estimators of the standard errors even in non-linear models like the logistic regression. However, if you believe your errors do not satisfy the standard assumptions of the model, then you should not be running that model as this might lead to biased parameter estimates.

For instance, in the linear regression model you have consistent parameter estimates independently of whether the errors are heteroskedastic or not. However, in the case of non-linear models it is usually the case that heteroskedasticity will lead to biased parameter estimates (unless you fix it explicitly somehow).

Stata is famous for providing Huber-White std. errors in most of their regression estimates, whether linear or non-linear. But this is nonsensical in the non-linear models since in these cases you would be consistently estimating the standard errors of inconsistent parameters.

This point and potential solutions to this problem is nicely discussed in Wooldrige's Econometric Analysis of Cross Section and Panel Data."

Amen to that!

Regrettably, it's not just STATA that encourages questionable practices in this respect. These same options are also available in EViews, for example.

Reference

Greene, W. H., 2012. Econometric Analysis. Prentice Hall, Upper Saddle River, NJ.

Turn on the Economy

2013-05-07T23:22:00.000-07:00

"Turn on the economy". That's one of the invitations issued to (web) visitors to the museum of New Zealand's central bank - The Reserve Bank of New Zealand. Accepting this invitation will allow you to see a virtual version of Bill Phillips' famous MONIAC computer at work, and to "play" with the economy yourself.

Doesn't that appeal to you?

Bill Phillips - "the Indiana Jones of Economics" - gave us "the Phillips Curve", of course. However, the MONIAC computer was a revolutionary device that Bill used to demonstrate economic stabilization policy.

If you happen to be visiting New Zealand's capital city - Wellington - you can visit the Bank's (physical) museum, and see the MONIAC "in the flesh".

The Indiana Jones of Economics

2013-05-07T20:59:00.002-07:00

All students of economics have heard about The Phillips Curve in one of its forms or another. The Phillips Curve is named after A. W. H. (Bill) Phillips, a remarkable New Zealander who made a number of fundamental contributions. His work, undertaken largely at the London School of Economics, dealt with stabilization policy, and modelling in continuous time, to name just two topics.

Bill Phillips was quite a character, and his varied life has been amply documented in various places. His entry in Wikipedia is a useful starting point, and the memorial piece written in 1978 by one of my former teachers, Brian Easton, is also a "must read" item.

Some time ago, I wrote about Bill in a post titled, "A Moniacal Economist", in reference to his famous MONIAC machines. These were hydraulic analogue computers that could be used to demonstrate the workings of the macro-economy.

In February of this year, the BBC Radio aired a piece about Bill Phillips, titled "The India Jones of Economics". In this 14-minute broadcast, Tim Hartford provides an interesting commentary of Bill's life and contributions to our discipline. You can download the broadcast - it's Episode 4, 6 February 2013 - from here.

More about Bill Phillips at a later date..............

My Recent Reading

2013-05-06T16:10:00.000-07:00

Here are some of the papers that I have been reading in the past few days:

Majid M. Al-Sadoon, 2013. Geometric and long run aspects of Granger causality. Discussion Paper, Barcelona Graduate School of Economics.
David Ardia & Lennart Hoogerheide, 2013. GARCH models for daily stock returns: Impact of estimation frequency on value-at-risk and expected shortfall forecasts. Tinbergen Institute Discussion Paper.
Otilia Boldia & Alastair R. Hall, 2013. Estimation and inference in unstable nonlinear least squares models. Journal of Econometrics, 172, 158-167.
Kazuhito Higa, 2013. Estimating upward bias in the Japanese CPI using Engel's law. Working Paper, Hitotsubashi University.
Anna Mikusheva, 2013. Survey on statistical inferences in weakly identified instrumental variables models. Applied Econometrics, 29, 117-131.

Econometrics Lectures on YouTube

2013-05-06T15:42:00.005-07:00

I'm always keeping my eyes open for new or different resources that I can integrate into my Economic Statistics and Econometrics courses. For example, as I've mentioned before in previous posts (here and here), I've been really pleased with what I've been able to achieve with Wolfram's cdf files.

In my undergrad. Statistical Inference course I also refer the students to some of the excellent mini-lectures by Keith Bower. I find his presentations to be clear and (very importantly) accurate.

If you check out YouTube you'll find a number of video presentations relating to the teaching of econometrics. To be honest, many of them don't particularly impress me. Maybe I;m just hard to please!

There are some exceptions to this, though, including David Hendry's 20111 lecture on Teaching Undergraduate Economics at Oxford, and the great series of videos of Mark Thoma in action in the classroom.

Burgernomics

2013-05-06T14:32:00.001-07:00

Looking through the papers that are "in press" at Economics Letters today, I came across a paper by Anthony Landry, titled "Borders and Big Macs". (The link is to the working paper version.) Here's the abstract:

"I provide new estimates of border frictions for 14 countries using local, national, and international Big Mac prices. I ﬁnd that borders generally introduce only small price wedges, far smaller than those observed across New York City neighboring locations."

This led me to wonder just how many academic papers have been written using the well-known "Big Mac Index" (BMI) that's published annually by The Economist magazine. I don't know the exact answer, but there are 39 listed on RePEc's IDEAS site.

That's a lot of burgers!

A Visual Proof That OLS is BLU

2013-05-06T09:48:00.000-07:00

Back in the day (as they say), we had monochrome monitors on our P.C.'s. Do you remember the ghastly green or weird amber colours? Then, one bright day everything became multi-coloured! This is not just me reminiscing - this is leading up to an innovative proof of the Gauss-Markhov Theorem. Honestly!

In a post yesterday, I mentioned Ken White's sense of humour - that's Ken White, "The SHAZAM Man", as my kids used to affectionately call him. On one of his many visits in the late 1980's, Ken offered to give a talk to a group of students about using the SHAZAM econometrics package. (We had no money for software at the time, but thanks to Ken's outstanding generosity we always had the latest version of his package for everyone to use.)

There were (and probably still are) some hidden tricks in SHAZAM. For instance, if you entered the command "USER KEN", then all sorts of extra things were available to you. For instance, the subsequent commands became case-sensitive. Another command that wasn't widely known was "SET COLOR", which could be rescinded with "SET NOCOLOR". With this option toggled on, the background colour on the monitor changed as certain SHAZAM commands were executed.

I must admit, this got a little hard on the eyes after a while, which might account for my declining vision! However, you have to remember that we didn't have the internet to amuse us, and it really didn't take too much to get us excited - technologically speaking! So, if you issued the command, "READ X", to indicate that a series of data was to be read, the background colour on the monitor turned red.

So, Ken got started on his promo-talk about SHAZAM. The students got right into it. Then he said, "You know, the great thing about SHAZAM is that unlike the competing packages, you can actually use it to prove some important theorems." Now the class got visibly excited!

"For instance", said Ken. "Let's prove the Gauss-Markhov Theorem". He keyed in the command OLS Y X. As the results began to appear on the screen, the background colour of the monitor turned very bright blue. in response to the use of the OLS command. The colour was so bad that you could hardly see the results.

Dead-pan, Ken said, "And that, my friends, is how you prove that the OLS estimator is BLU!"

The Frequent Regressor Club

2013-05-05T12:07:00.002-07:00

My friend, Ken White, developed the SHAZAM econometrics package in 1977. Ken's a funny guy - that's to say, he has a great sense of humour.

On one of his many visits to Christchurch, New Zealand (when I was living there, many years ago) he gave me a wooden die that he'd had an artisan carve at the local Arts Centre in Chistchurch. On each of the six faces he'd had the guy put the name of an econometrics/statistics package - TSP, LIMDEP, GAUSS, RATS, and SHAZAM. Yes, I know that's only 5 names. The thing was, SHAZAM appeared on two of the faces! The idea was to roll the die to decide which package to use in your lab. class. I still have the die - much to the occasional bemusement of students who see it on my desk.

Ken also kindly gave me a very early edition of the Captain Marvel comic - the one where the meaning of the acronym, SHAZAM, is revealed to the readers. That comic is still in my office too. There's going to be a heck of a garage sale in the department when I retire!

Some of you will recall the manuals that went with earlier versions of SHAZAM. There was an "Appendix" at the back, titled Frequent Regressor Club. At that time, Ken travelled A LOT. He was an avid collector of frequent flyer points with United Airlines. He had so many that he hardly knew what to do with them. This was the inspiration for the Frequent Regressor Club.

Readers of the manual were told that they could collect points for every OLS regression that they ran. They just had to mail a copy of the printout to Ken. (This was pre-PC days.) Nonlinear regressions earned extra points; and step-wise regressions weren't allowed!

If you earned enough points you could apparently get a free upgrade of SHAZAM. It was just a spoof, but Ken was constantly stunned by the number of people who sent him their printouts!

Those were the days!

Granger Causality Testing Done Properly

2013-05-04T08:31:00.002-07:00

I enjoy following David Stern's Stochastic Trend blog. David is Research Director at the Crawford School of Public Policy at the Australian National University. He's an energy and environmental economist who does some really interesting work - not my field at all, but I always enjoy reading what he has to say.

In his latest blog post, David links to a recent paper that he's co-authored with Robert Kaufman, from Boston University. The paper is titled, "Robust Granger Causality Testing of the Effect of Natural and Anthropogenic Radiative Forcings on Global Temperature".

As I said, this isn't my field. However, if you want to see an example of Granger causality testing done well, you should take a look at this well-written paper.

Nice one!

When Will the Adjusted R-Squared Increase?

2013-05-03T20:37:00.000-07:00

The coefficient of determination (R²) and t-statistics have been the subjects of two of my posts in recent days (here and here). There's another related result that a lot of students don't seem to get taught. This one is to do with the behaviour of the "adjusted" R² when variables are added to or deleted from an OLS regression model.

We all know, and it's trivial to prove, that the addition of any variable to such a regression model cannot decrease the R² value. In fact, R² will increase with such an addition to the model in general, and it will stay the same if the added variable is totally uncorrelated with the other variables in the model. Conversely, deleting any regressor from an OLS regression model will reduce the value of R².

Indeed, this is precisely why various "adjusted" R² measures have been suggested over the years. You can boost the goodness-of-fit of the model by throwing anything into the regression, whether it makes economic sense or not.

The adjusted R² that we typically use involves "correcting" both the numerator and denominator sums of squares (in the usual R² formula) for the appropriate degrees of freedom. If the sample size is "n", and the model includes "k" regressors (including the intercept) this adjusted R² can be expressed as:

R_A² = 1 - [(n - 1) / (n - k)][1 - R²] ,

and it can be shown that R_A² ≤ R² ≤ 1. The adjusted R² can take negative values, and this will occur if and only if R² ≤ [(k - 1) / (n - 1)].

Now, what can we say about the behaviour of R_A² if we add regressors or delete them from the model? The adjusted R² may increase or decrease (or stay the same) when we do this, and there are some simple conditions that determine which will occur.

The first result is that adding a regressor will increase (decrease) R_A² depending on whether the absolute value of the t-statistic associated with that regressor is greater (less) than one in value. R_A² is unchanged if that absolute t-statistic is exactly equal to one.

If you drop a regressor from the model, the converse of the above result applies. Dropping a regressor amounts to imposing a (zero) restriction on its coefficient. If you square the t-statistic, you get an F-statistic, and it's exactly the F-statistic for testing if the single linear restriction is valid. Not surprisingly, then, there's a more general result than the one given above - one that applies to a situation where several regressors are simultaneously added to or dropped from the model.

Adding a group of regressors to the model will increase (decrease) R_A² depending on whether the F-statistic for testing that there coefficients are all zero is greater (less) than one in value. R_A² is unchanged if that F-statistic is exactly equal to one.

So, you can increase the adjusted coefficient of determination by adding regressors that are statistically insignifcant, but the situation isn't quite as bad as with the usual (uadjusted) R².

Finally, the second result given above generalizes to the case where we are considering any set of linear restrictions on the regression coefficients - not just zero restrictions.

In summary, and not too surprisingly, the behaviour of the adjusted coefficient of determination as we add or delete regressors is quite systematic. If you're a student who's hoping that deleting a regressor with a t-statistic of 1.1 will increase the value of R_A², think again!

Mark Thoma on "Replication"

2013-05-03T10:25:00.000-07:00

Yesterday, in his Economist's View blog, Mark Thoma discussed the importance of replicating results in empirical economics. He's absolutely right, of course.

I'll leave you to read what had to say, but I especially liked his closing passage:

"One place where replication occurs regularly is assignments in graduate classes. I routinely ask students to replicate papers as part of their coursework. Even if they don't find explicit errors (and most of the time they don't), it almost always raises good questions about the research (why this choice, this model, what if you relax this assumption, there's a better way to do this,here's the next question to ask, etc., etc.). So replication does occur routinely in economics, and it is very valuable, but it is not a formal part of the profession the way it should be, and much of the replication is done by people (students) who generally assume that if they can't replicate something, it is probably their error. We have a lot of work to do on the replication front, and I want to encourage efforts like this."

At least one of my colleagues also assigns replication exercises in this way, and I really should do the same. Fortunately, more journals are either recommending or requiring that data-sets be made available as a condition of publication. The Journal of Applied Econometrics is one such journal, and we've recently been pushing in that direction with the Journal of International Trade & Economic Development.

This should become part of our culture.

When Can Regression Coefficients Change Sign?

2013-05-03T09:51:00.003-07:00

Let's suppose that you've been running regressions happily all morning. It's sunny day, but what could be better than enjoying some honest-to-goodness econometrics? Suddenly, you notice that one of the estimated coefficients in your model has a sign that's the opposite to what you were expecting (from your vast knowledge of the underlying economics). Shock! Horror!

Well. it's really good that you're on the look-out for that sort of thing. Congratulations! However, something has to be done about this problem.

Being young, with good eyesight, you also happen to spot something else that's interesting. One of the other estimated coefficients has a very low t-statistic. You have a brilliant idea! If you delete the variable associated with the very small t-value, maybe the "wrong" sign on the first coefficient will be reversed. Is this possible?

Sadly, no, it's not going to be that simple.

Let's make sure that what I'm talking about is quite clear. Here are the results for a hypothetical regression, estimated by OLS, and with t-statistics in parentheses:

y = 0.43 + 1.45X₁ - 0.89X₂ + residual ; n = 34 ; R² = 0.89
(1.45) (0.66) (-1.83)

You were expecting a positive coefficient for X₂. If you drop X₁ from the regression, and re-estimate the model by OLS, could the sign of the coefficient of X₂ become positive? No - that's impossible.

Leamer (1975) proved that such a change in sign cannot occur if the absolute value of the t-statistic for the variable you're deleting is less than the absolute value of the t-statistic for the variable whose sign you're interested in. That's the situation we have in the example above.

On the other hand, if we were to delete X₂ from the above model, its possible for the sign of the coefficient for X₁ to change.

A number of extensions/generalizations of this result are also available, including:

Leamer's necessary condition, stated above, was extended to include a sufficient condition by Visco (1978).
Leamer's result was re-stated in a somewhat simpler form by Oksanen (1987).
Leamer's result was generalized by McAleer et al. (1986) to apply to cases where the deleted variables are combined in arbitrary linear combinations. See, also, Visco (1988).
All of the above results were shown by Gikles (1989) to hold if the model is estimated by any Instrumental Variables estimator, rather than by OLS.

This last extension is based on the algebraic relationship between the OLS and (generalized) I.V. estimators, and the results on restricted I.V. estimation given by Giles (1982).

By the way - HT to a former student of mine, Darren Gibbs, who asked some interesting questions that led me to write the 1989 paper referenced below. I never teach a course without learning something new!

References

Giles, D. E. A., 1982. Instrumental variables estimation with linear restrictions. Sankhya: The Indian Journal of Statistics, B, 44, 343-350.

Giles, D.E.A., 1989. Coefficient sign changes when restricting regression models under instrumental variables Estimation. Oxford Bulletin of Economics and Statistics, 51, 465-467.

Leamer, E. E., 1975. A result on the sign of restricted least-squares estimates. Journal of Econometrics, 3, 387-390.

McAleer, M., A. Pagan, & I. Visco, 1986. A further result on the sign of restricted least-squares estimates. Journal of Econometrics, 32, 287-290.

Oksanen, E. H., 1987. On sign changes upon deletion of a variable in linear regression analysis. Oxford Bulletin of Economics and Statistics, 49, 227-229.

Visco, I., 1978. On obtaining the right sign of a coefficient estimate by omitting a variable from the regression. Journal of Econometrics, 7, 115-117.

Visco, I., 1988. Again on sign changes upon deletion of a variable from a linear regression. Oxford Bulletin of Economics and Statistics, 50, 225-227.

All About Spherically Distributed Regression Errors

2013-05-02T19:45:00.000-07:00

This post is based on a handout that I use for one of my courses, and it relates to the usual linear regression model,

y = Xβ + ε

In our list of standard assumptions about the error term in this linear multiple regression model, we include one that incorporates both homoskedasticity and the absence of autocorrelation. That is, the individual values of the errors are assumed to be generated by a random process whose variance (σ²) is constant, and all possible distinct pairs of these values are uncorrelated. This implies that the full error vector, ε, has a scalar covariance matrix, σ²I_n.

We refer to this overall situation as one in which the values of the error term follow a “Spherical Distribution”. Let's take a look at the origin of this terminology.

The following discussion is quite general, so you'll realize that it applies to any random variables, not just the error term in our regression model. Further, so that we can look at some diagrams, let’s consider the special case of two dimensions, rather than three, so that what would be a (3-dimensional) sphere becomes a (2-dimensional) circle.

So, consider the pair of random values ε_i and ε_j, which we’ll generically denote x and y. (This latter terminology has nothing to do with X and y in the regression model.) The values of these two random variables are plotted in the directions of the x and y axes in the graphs which follow.

In the three-dimensional plots we will see the joint probability density function, p(x, y) in the direction of the z axis. All of these three-dimensional plots are for values in the range -3 ≤ x, y ≤ 3. Scales are given on the associated two-dimensional “contour” plots. The latter plots show “isolines” – that is, lines that join up (x, y) points that yield the same value (height) for p(x, y). These contours are exactly analogous to the contour lines that you see on a topographic map to depict the nature of the terrain. They reflect what you see when you look down vertically on to the three-dimensional (bivariate) density plots.

The appearances of the density plots, and the shape of the associated contour plots, depend upon the variances of x and y, and the covariance (and hence correlation) between these two random variables.

If x and y have the same variance (i.e., if ε_i and ε_j are homoskedastic), and if they are uncorrelated (i.e., if ε_i and ε_j are not autocorrelated), then the contours will form circles. If there were three random variables we would need a four-dimensional density graph, and the contours would form a sphere. Hence the term “Spherical Distribution”. If there were four or more random variables the sphere would become a “hyper-sphere”.

If x and y have different variances, the joint density surface is no longer symmetrical in the x and y directions, and then the coutour plot takes the form of an ellipse, rather than a circle. The same thing happens if x and y are correlated, even if they have the same variance. In this case, the slope of the primary axis of the ellipse is determined by the sign of the correlation between x and y.

Some examples follow, all for the case where x and y follow a bivariate normal distribution with zero means for x and y. The plots were all done using R (of course).

E(x , y)' = (0 , 0)' ; var.(x) = var.(y) = 1 ; cov. (x , y) = 0

E(x , y)' = (0 , 0)' ; var.(x) = 1; var.(y) = 9 ; cov. (x , y) = 0

E(x , y)' = (0 , 0)' ; var.(x) = var.(y) = 1 ; cov. (x , y) = 0.5

E(x , y)' = (0 , 0)' ; var.(x) = var.(y) = 1 ; cov. (x , y) = -0.75

E(x , y)' = (0 , 0)' ; var.(x) = var.(y) = 1 ; cov. (x , y) = 0.99

E(x , y)' = (0 , 0)' ; var.(x) = 1 ; var.(y) = 9 ; cov. (x , y) = 0.7

Now, consider a final case:

Do x and y have the same means?

Do they have the same variances?

Are they correlated - if so, positively or negatively?

Good Old R-Squared!

2013-05-02T13:53:00.001-07:00

My students are often horrified when I tell them, truthfully, that one of the last pieces of information that I look at when evaluating the results of an OLS regression, is the coefficient of determination (R²), or its "adjusted" counterpart. Fortunately, it doesn't take long to change their perspective!

After all, we all know that with time-series data, it's really easy to get a "high" R² value, because of the trend components in the data. With cross-section data, really low R² values are really common. For most of us, the signs, magnitudes, and significance of the estimated parameters are of primary interest. Then we worry about testing the assumptions underlying our analysis. R² is at the bottom of the list of priorities.

Our students learn that R² represents the proportion of the sample variation in the data for the dependent variable that's "explained" by the regression model. Even so, they don't always think of R² as a statistic - a random variable that has a distribution.

That's what it is, of course. If we define the coefficient of determination as R² = [ESS/TSS], where ESS is the "explained sum of squares", and TSS is the "total sum of squares", then clearly the denominator is a function of the random "y" data. Moreover, the numerator is a function of the OLS estimator for the coefficient vector, β, so it's a random quantity too. Yes, R² is indeed a statistic. So is the familiar "adjusted" R², of course.

This implies that it has a sampling distribution. How often do students think about this?

We're going to focus on the sampling distribution of the coefficient of determination for a linear regression model

y = Xβ + u ; u ~ N[0 , &sigma:²I_n] ,

estimated by OLS.

If we recall the relationship between R² and the F-statistic that used in the OLS context for testing the hypothesis that all of the "slope coefficients" are zero, namely:

F = [R² /(1 - R²)][(n - k) / (k - 1)] ,

then we start to see something about the possible form of this sampling distribution. If you know something about the connections between random variables that are F-distributed, and ones that follow a Beta distribution, you won't be surprised if I tell you that Cramer (1987) showed that the density function for R² can be expressed as a messy infinite weighted sum of Beta densities, with Poisson weights. It's also a function of X data, and the unknown parameters, β and σ.

Yuk! Let's try and keep things a little simpler.

Koerts and Abrahamse (1971) explored various aspects of the behaviour of the statistic, R².

Let E = I - (i i'), where i is a column vector with every element equal to one in value. It is easily shown that E is idempotent, and that it is an operator that transforms a series into deviations about the sample mean. Similarly (because of the idempotency), the matrix (X'EX) is the "deviations about means" counterpart to the (X'X) matrix.

Let's assume that (X'EX) is well-behaved when n becomes infinitely large. Specifically, let's make the the standard assumption that

Limit[n^-1(X'EX)] = Q,

where Q is finite and non-singular. Then, Koerts and Abrahamse (1971, pp. 133-136) show that

plim(R²) = 1 - σ² / [β'Qβ + σ²] = φ , say.

The quantity, φ, is usually regarded as the "population R²" in this context. The sample R² is a consistent estimator of the population R².

Also, asymptotically, R² converges to a value less than one. Moreover, its large-sample behaviour depends on the unknown values of the parameters, and also on the (unobservable) matrix, Q. The nature of the latter matrix - the degree of multicollinearity in the X data (asymptotically) - plays a role in the probability limit of R².

Now, let's put the large-n asymptotic case behind us, and let's focus on the sampling distribution of R² in finite samples. First, what can be said about the first two moments of the sampling distribution for the coefficient of determination?

Following earlier work by Barten (1962), Cramer (1987) derived the mean and the standard deviation of R², and also for its "adjusted" counterpart. As you'd guess from my earlier description of his result for the p.d.f. of R², the results are pretty messy, and they depend on the X data and the unobserved parameters, β and σ. However, there are some pretty clear results that emerge:

R² is an upward-biased estimator of the "population R²", φ.
The bias vanishes quite quickly as n grows.
Whenever the bias of R² is noticeable, its standard deviation is several times larger than this bias.
"Adjusting" R² (for degrees of freedom) in the usual way virtually eliminates the positive bias in R² itself.
However, the adjusted R² has even more variability than R² itself.

Koerts and Abrahamse (1971) used Imhof's (1961) algorithm to numerically evaluate the full sampling distribution of R², for various choices of the X matrix, and different parameter values. They found that:

This sampling distribution is sensitive to the form of the X matrix.
If the disturbances in the regression model follow a positive (negative) AR(1) process, then this shifts the sampling distribution of R² to the right (left) - that is, towards one (zero). In other words, if the errors follow a positive AR(1) process, the probability of observing a large sample R² value is increased, ceteris paribus.

However, the effects of autocorrelation on the sampling distribution of R² were studied more extensively by Carrodus and Giles (1992). We considered a range of X matrices, and both AR(1) and MA(1) processes for the regression errors, and obtained results that did not fully support Koerts and Abrahamse. Using Davies (1980) algorithm, we computed the c.d.f.'s and p.d.f.'s for R², and found that:

Almost without exception, negative AR(1) errors shift the distribution of R² to the left.
Positive AR(1) errors can shift the distribution of R² either to the left or to the right, depending on the X data.
The direction of the shift in the distribution with either positive or negative MA(1) errors is very "mixed", and exhibits no clear pattern.

So, there are few things for students to think about when they look at the value of R² in their regression results.

First, the coefficient of determination is a sample statistic, and as such it is a random variable with a sampling distribution. Second, the form of this sampling distribution depends on the X data, and on the unknown beta and sigma parameters. Third, this sampling distribution gets distorted if the regression errors are autocorrelated. Finally, even if we have a very large sample of data, R² converges in probability to a value less than one, regardless of the data values or the values of the unknown parameters.

The bottom line - it's just another statistic!

References

Barten, A.P., 1962,. Note on the unbiased estimation of the squared multiple correlation coefficient. Statistica Neerlandica, 16, 151-163.

Carrodus, M. L. and D. E. A. Giles, 1992. The exact distribution of R² when the regression disturbances are autocorrelated. Economics Letters, 38, 375-380.

Cramer, J. S., 1987. Mean and variance of R² in small and moderate samples. Journal of Econometrics, 35, 253-266.

Davies, R. B., 1980. Algorithm AS155. The distribution of a linear combination of χ² random variables. Journal of the Royal Statistical Society. Series C (Applied Statistics), 29, 323-333.

Imhof, J.P., 1961. Computing the distribution of quadratic forms in normal variables. Biometrika, 48, 419- 426.

Koerts, J and A. P. J. Abrhamse, 1971. On the Theory and Application of the General Linear Model. Rotterdam University Press, Rotterdam.

Finite Sample Properties of GMM

2013-05-01T16:29:00.000-07:00

In a comment on a post earlier today, Stephen Gordon quite rightly questioned the use of GMM estimation with relatively small sample sizes. The GMM estimator is weakly consistent, the "t-test" statistics associated with the estimated parameters are asymptotically standard normal, and the J-test statistic is asymptotically chi-square distributed under the null. But what can be said in finite samples?

Of course, this question applies to almost all of the estimators that we use in practice - IV, MLE, GMM, etc. Indeed, lots of work has been done to explore the finite-sample properties of such estimators. For instance, consider my own work on bias corrections for MLEs (see here, here, and here). So, I'm more than sympathetic to the general point that Stephen made.

The example that I had provided, which was just a teaching example, used 175 quarterly observations. Is this enough for the asymptotics to "kick in" for GMM estimation of an Euler equation of the type I was considering?

My first reaction was: "let's just bootstrap this thing and find out." My second reaction was: "there has to be plenty of evidence out there already, so let's not re-invent the wheel." Indeed, this is the case. Several studies have examined the performance of GMM in precisely the context that I was using it in my own example.

Three relevant studies are those of Tauchen (1986), Kocherlakota (1990), and Hansen et al. (1996). There are probably others as well.

1. Tauchen considers sample sizes of 50 and 75 - much smaller than I was using. Among his conclusions (p.397):

"There is a variance/bias trade-off regarding the number of lags used to form instruments: with short lags, the estimates of utility function parameters are nearly asymptotically optimal, but with longer lags the estimates concentrate around biased values and confidence intervals become misleading."
"The test of the overidentifying restrictions performs well in small samples; if anything, the test is biased toward acceptance of the null hypothesis."

2. Kocherlakota considers a sample of T = 90 observations. Among his conclusions:

The J-test exhibits minimal size distortion in four of the seven experimental designs considered, and is is biased toward over-rejection of the null in the other three cases. These outcomes depend very much on the choice of instruments.
In the three cases of over-rejection on the part of the J-test, the estimates of the parameters are downward median-biased. However, they are essentially median-unbiased in the other four cases considered.

3. Hansen et al. consider a sample size of 100 for the part of their study most relevant here. Among their conclusions:

The finite-sample properties of the GMM estimator depend very much on the way in which the moment conditions are weighted.
"Continuous updating in conjunction with criterion-function-based inference often performed better than other methods for annual data; however, the large-sample approximations are still not very reliable." (p.278)
"The continuous-updating estimator typically had less median bias than the other estimators, but the Monte Carlo sample distributions for this estimator sometimes had much fatter tails." (p.278) [This has implications for the coverage probability of confidence intervals; DG.]
"The test for overidentifying restrictions are, by construction, more conservative when the weighting matrix is continuously updated, and in many cases this led to a more reliable test statistic."

So, where does that leave us? I'm glad that I used the continuous-updating version of the GMM estimator in my illustration. With T = 175, I'm in a somewhat better position than those considered in the simulation studies I've just cited. However, the results should be treated very cautiously.

On the other hand, what are you going to do in practice? I think that Tauchen (1986, p.415) had the right idea: bootstrap the GMM estimates to compensate for small-sample bias, and to obtain appropriate confidence intervals and critical values for the J-test.

References

Hansen, L. O., J. Heaton, & A. Yaron, 1996. Finite-sample properties of some GMM models. Journal of Business and Economic Statistics, 14, 262-280.

Kocherlakota, N. R., 1990. On tests of representative consumer asset pricing models. Journal of Monetary Economics, 26, 285-304.

Tauchen, G., 1986. Statistical properties of generalized method-of-moments estimators of structural parameters obtained from financial market data. Journal of Business and Economic Statistics, 4, 397-416 (plus discussion and response).