When I was a graduate student, my mentor Jim Poterba told me more than once that my research should “always be about the economy and never about economists.” I took those words to heart and consistently resisted the temptation to do self-referential research about the economics profession.
I finally violated Poterba’s rule at this year’s American Economic Association meetings. Doubly ironic, one of the co-authors on the paper was Jim Poterba himself! We both got involved because we thought the project had some special merit for the profession.
Alan Krueger spearheaded an ambitious project to bring together parallel data on students from Princeton, Chicago, MIT, Stanford, and Harvard graduate programs in economics with the hopes that we could better understand the graduate admissions and training process. Susan Athey (who was at Stanford at the time) and Larry Katz were the other co-authors. You can read the paper here, and read about the paper at insidehighered.com.
I can’t really take any credit or blame for the paper — Alan Krueger did all the work and had all the ideas. It is a stretch to even consider me a co-author. (It would be more appropriate to make Julie Less, who worked tirelessly to assemble the U of C data, the U of C co-author.)
Our results thus far, unfortunately, are not particularly earth-shattering. Admissions committees are good at predicting who will do well in first-year classes, but not good at predicting who will find good jobs after graduate school. The first-year course grades are strong predictors of job placement. Once you take into account first-year grades, it turns out that GRE scores, gender, and doing a foreign undergraduate degree do not predict who will get a good job.
There were three findings I think are most interesting. First, the GRE analytical score turns out to be more important than the GRE math score in predicting first-year grades, despite the tremendous emphasis that is put on the GRE math score in the admissions process. (In contrast, ETS discontinued the analytical part of the test!). Second, women do very badly in first-year courses given their background characteristics, but they do fine on the job market. Third, after controlling for other factors, students who went to top undergraduate institutions do not do much better in classes, but they do a lot better on the job market.

January 9, 2007, 8:41 am

So that means if you put a grazing cow on the hood of your car, you could save some money on a G.P.S. satellite location system. ”


2007
9:23 am
The last sentence of your penultimate paragraph is confusing. I suggest that you use something other than a comma to separate between “first year grades” and the list of factors that are not good predictors.
— Posted by Isaac
2007
9:59 am
Maybe this is an obvious point, so if so ignore it. But in my experience, people who do well in first year courses are those people who are practically blue collar mathematicians. Those skills have high payoffs in the first 18 months - where problem sets, exams and prelims are all that matters. But the job market doesn’t reward straight A’s as much as it rewards talent which is signaled through things like graduate school publications and grants, faculty letters of recommendation, and job market papers. It would be interesting to link other job market outcomes, therefore, to graduate school experiences - citations of published papers, downloads of papers at SSRN, publications at various journals, for instance. My sense is that good economists are very curious, and that is orthogonal (or weakly correlated) to excellent mathematical skills - especially as the profession has moved more towards applied econometrics and away from pure theory.
— Posted by scott cunningham
2007
10:09 am
This article was emailed to me by a friend a few months ago. It was on the WSJ online.
http://online.wsj.com/public/article/SB1158538187476658 42-he7lCpgXm8N2NkVl6PvPNthMh_0_20061017.html?mod=tff_ma in_tff_top
— Posted by wesleyb41
2007
10:21 am
Thanks commenter #1. I was trying to make sense of that sentence. Now I get it.
— Posted by egretman
2007
10:56 am
wesleyb41 - wow. I totally missed that article. Thanks for posting it.
— Posted by scott cunningham
2007
11:13 am
Scott - I found it very encouraging that I don’t HAVE to go to Harvard in order to be considered for Senior Management in business. I just have to work like crazy to get there… no big deal really.
— Posted by wesleyb41
2007
11:15 am
I interpreted that article as more speaking to the nature of superstar markets. Unobserved traits have a lot to play than investments in schooling. Plus, a lot of luck.
— Posted by scott cunningham
2007
11:24 am
The 5th paragraph (2nd to last) seems contradictory. If “Admissions committees are good at predicting who will do well in first-year classes, but not good at predicting who will find good jobs after graduate school.” and “The first-year course grades are strong predictors of job placement.”, then predicting how well someone will do in 1st year classes must predict job placement. Therefore, the 1st sentence must be wrong (if A predicts B and they are good at predicting A, then they are good at predicting B).
— Posted by pkimelma
2007
12:32 pm
Third, after controlling for other factors, students who went to top undergraduate institutions do not do much better in classes, but they do a lot better on the job market.
Damn that second tier southern university alma mater of mine. It’s all about Ivies. Just like they say. Dang it.
— Posted by egretman
2007
12:37 pm
Egretman - click on the WSJ link - you will see that the gap between Ivy League school and all the rest is shrinking. At least the author thinks it is.
— Posted by wesleyb41
2007
12:49 pm
I did, wesleyb41, but the WSJ article is just anecdotal. It’s not the heavily statistified double-secret scientific article that the above post is.
The truth is not to be denied. The south has no Ivies, no Chicago, no Standford. We are second rate. That’s just the facts, maam. Let’s face it, we are not going to get the good jobs.
Rather than deny it. We should embrace it. For instance, Rice is good for something, right? ( I mean the school not the food product)
— Posted by egretman
2007
12:52 pm
wesleyb41 for someone that claims to be senior management, you sure do have alot of time to imply to other people that other authors suck. You suck cause you have nothing original, creative or constructive to add. Your not even funny or encouraging.
— Posted by mike.wong
2007
1:00 pm
Yeah, I guess you are right. Vanderbilt, Sewanee, Duke, Emory and the other southern schoosl just couldn’t hold a candle to those others. We should tell them to just go ahead and shut the doors.
— Posted by wesleyb41
2007
1:18 pm
Mike - I never said I was in senior management. I linked the article and I agreed with the author who wrote it. I am glad is not necessary to have a degree from Harvard to be a CEO, CFO, etc.
I find it very encouraging that it matters more what you accomplish than what name is on the MBA. I think you may have misinterpreted what I said. Sorry you think I suck.
— Posted by wesleyb41
2007
2:07 pm
Do you guys have data on what fields the grad students you analyze were in? Conventional wisdom is that conditional on first year scores, students with theory papers do much worse on the job market than students with empirical papers. Might be interesting to show how true this is; also including this might effect the importance of first year scores. Lastly, do theory students love theory so much to justify the lower outcomes on the job market, or is this an overconfidence bias?
— Posted by sanagol
2007
5:28 pm
About the GRE math vs. analytical scores: this makes some sense since the GRE math section only tests mathematical ability up through high school geometry. The math skills needed to do well in graduate school econ classes (proofs and algebraic trickery) aren’t well-tested by this section, so I’ve got to think that there is a lot of compression in GRE math scores among students in these schools. Whatever variation you observe below 800 is probably measurement error (and last I checked, getting a math 800 still only got you around a 93rd percentile since so many test-takers do well in that section).
The GRE analytical is some measure of reasoning ability, which is pretty helpful in grad school of any sort. Since GRE/ETS abolished that section, maybe you could get whoever runs LSAT to use their analytical reasoning section.
— Posted by your t.a.
2007
9:16 pm
I am curious, when you make the statement “contol for other factors”, how is that done? Can someone point me to a tutorial on the web?
Thanks.
— Posted by doherjo1
2007
12:00 am
doherjo1 - Probably, anything you read on the web will ultimately be confusing if you haven’t heard of regression analysis before, but check this out anyway.
— Posted by scott cunningham
2007
4:48 pm
Well you said women do better than men at the job market and not at the first year courses, thats also a lot to do with few women being enrolled in PhD courses and the affirmative action by firms.
As for Ivy leagues mattering more in job market its simply since its a signalling factor a bit of the old boys network effect.
Some one from 2 big name schools would appeal to a recruiter much more. Also an MIT Alumnus would have some positive bias about MIT undergrad and not so much about State University of some southern state.
It goes on to show the recruiters biases (and may be the fact that the first year course grades are not relevant in recruitment)
— Posted by 4nirav
2007
8:37 pm
The obvious follow-up paper is to track the same students through first 6+ years of their academic career. How successful were they? How many top-tier publications did they get? Did they receive tenure at their initial placement? Etc.
One prediction: the students who went to top undergrad schools and who did not do so well in their first-year course, but still placed well, end up not being very successful tenure-track academics.
— Posted by msp
2007
1:26 pm
I went to the ASSA Conference also, but academic economists seem less and less relevant every year. One reason is that academia is closer to a caste system than a meritocracy. For example, compare the New Orleans Saints player roster (http://neworleanssaints.com/roster.cfm) with the rosters of traditionally “highly rated” economics departments (JR Hasselback, Economics Faculty Directory 2006-2007, Prentice Hall, 2007). In the NFL meritocracy, good players from mediocre football programs can find a place on the roster. Such is generally not the case in the academic economist caste system. Perhaps this issue has been explored before, but I am not aware of any publications that address the apparent problem.
— Posted by hkilpatrick
2007
8:45 am
pkimelma wrote: “The 5th paragraph (2nd to last) seems contradictory. If “Admissions committees are good at predicting who will do well in first-year classes, but not good at predicting who will find good jobs after graduate school.” and “The first-year course grades are strong predictors of job placement.”, then predicting how well someone will do in 1st year classes must predict job placement. Therefore, the 1st sentence must be wrong (if A predicts B and they are good at predicting A, then they are good at predicting B).”
You’re assuming equal distribution. Example, ad com can predict who gets good course grades (let’s define good as top half) but ad coms predicteds were are all in the 11-50th percentile. Good grades predict who gets good job placement (lets say everyone who got a good job was in top 10% (and by def part of good grades [top 50%]), and no one from bottom 50% of class got a good job). So ad com could predict virtually everyone who would be top half of the class but no one who would get a good job.
— Posted by Eco