Race to the Top

What did the Carnegie report say anyway?

2009-10-04T19:25:16Z

In the last few years, there’s been a fair amount of talk in the legal academy about how legal education ought to change, in part in response to the 2007 Carnegie Foundation report. Many assume that the report is a repeat of the familiar “law school should be more practical,” and to a certain extent that’s true. But it’s more nuanced than that.

One way to get the gist is to head to the website of the new law school at UC-Irvine, founded by noted constitutional law scholar Erwin Chemerinsky. The site summarizes its curriculum this way: “A cutting-edge, strongly interdisciplinary curriculum will prepare UC Irvine School of Law graduates not only to think like a lawyer but also to actually practice law.” UCI’s website also includes a key quote from the Carnegie report: “Most law schools give only casual attention to teaching students how to use legal thinking in the complexity of actual law practice. Unlike other professional education, most notably medical school, legal education typically pays relatively little attention to direct training in professional practice. The result is to prolong and reinforce the habits of thinking like a student rather than an apprentice practitioner, conveying the impression that lawyers are more like competitive scholars than attorneys engaged with the problems of clients.”

You can also read the executive summary of the Carnegie report here, but the highlights are:

(1) Lawyers are best taught through a curriculum that integrates the three pillars of doctrine, skills, and professional identity, rather than having a curriculum that focuses on doctrine, and treats the other pillars as “add ons.”

Ideally, these three pillars (or “apprenticeships”) can be integrated in the same course. For example, rather than having a course called “Torts” that focuses on doctrine, a separate course called “Factual Investigation” that focuses on a skill that any torts lawyer needs, and a course on “Professional Responsibility” that covers ethical dilemmas facing such lawyers, these things can all be included in one course, even if not all are covered in depth. One might call such a course “The Torts Process,” which happens to be the name of a well-established course book from Aspen (Henderson et al) that uses this approach, and which I use to teach 1L Torts.

(2) The combination lecture/Socratic method is far overused, as is the Langdellian approach of learning the law through the dissection of appellate opinions. These problems are particularly pronounced after the first year.

(3) Law schools are far behind other educational institutions in how they assess student learning, and the extent to which they provide feedback that improves learning outcomes.

(4) Problems 1-3 are all related.

What’s Happening in Legal Education?

2009-10-04T19:38:58Z

With the beginning of a new school year, and a new law professor hiring season, one might want to know the answer to this question. Specifically, I can imagine law-professor job candidates being asked the question: “Are you familiar with the Carnegie report on legal education, and how would it affect your approach to teaching?” So I’d recommend you at least read the executive summary. Over at Concurring Opinions, Kathleen Boozang says that the report was greeted by a “big yawn” by most law professors, but she speculates that the collapse of the legal market could change that.

A recent National Law Journal article quotes Rod Smolla, the dean of Washington and Lee and architect of its new Carnegie-esque third-year curriculum, as saying “we are at a moment of historical change in legal education…When we look back at this period in five to 10 years, we will mark it as the time when the whole mission of law schools made a fundamental turn.” In W and L’s third year, students learn “substantive” areas like family law or employment law through real and simulated cases where students act as lawyers and try to solve problems, not by reading appellate opinions, taking notes in class, and mushing it all into an outline at the end for a 3-hour typing race. More students opted into the new curriculum than expected, and applications there were up 33% this past year, with a survey indicating that the new curriculum played a role in many students’ decisions to go there.

Meanwhile, Martha Minow, the new dean of Harvard Law School, where the Langdellian method of teaching from appellate opinions was developed, has called for “another case method” closer to the one used in business and public-policy schools, and consistent with W and L’s approach and Carnegie.

So will there be significant change or not much at all? My own view, for what it’s worth, is that much depends on the success or failure of innovations like Washington and Lee’s third-year curriculum, the more practice-oriented curriculum at Northwestern, and the new law school at UC-Irvine. If they succeed, I would imagine others will follow. What do you think?

New U of Chicago Dean on Educating Lawyers

2009-10-04T19:40:36Z

In this op-ed for the National Law Journal last year, “Legal Education Must Look Beyond the First Year,” UCLA Dean Michael Schill, soon to move to University of Chicago, talks about a theme I agree with: the consistency of schools doing sophisticated interdisciplinary research, and training lawyers in a rigorous way. He says:

For the first 20 years of my career as a legal academic, the pendulum of legal education swung strongly away from the “trade school” model of legal education toward interdisciplinary education and theory. The Carnegie Report suggests that, perhaps in our embrace of abstract theory, many law schools have neglected their principal obligation — teaching our students to be lawyers. As schools adjust their curricula, UCLA’s experience suggests that we be careful not to overreact. Deep interdisciplinary knowledge and mastery of theory can co-exist very well with increased specialization and practical skills development. Indeed, both sets of skills training reinforce each other, and only by embracing both will we produce the best possible legal professionals.

He and his team also made a terrific case for the strength of UCLA in educating lawyers in a piece, “How UCLA Law Trains Lawyers,” you can find here.

I have no reason to believe this orientation had much of anything to do with his hiring or his plans at U of C, but think it’s interesting that a school with a great reputation in interdisciplinary work (at least with one discipline), but less of one in clinical and skills education, has hired a dean who embraces both.

About Race to the Top

2009-10-06T00:26:03Z

Each year, hundreds of law professors, lawyers, and judges complete the survey that makes up 40% of the total score — no other category comes close — of the U.S. News rankings that dominate the institutional incentives for most law schools.

U.S. News used to call this a “reputation” survey but it does not anymore; it asks respondents to assess the quality of each J.D. program. But with no information on educational quality, research shows that respondents simply replicate the previous year’s U.S. News rankings, and the scores remain fairly static over time.

Given the question U.S. News asks, we need to think more about evaluating schools based on the quality of education that they provide for students, or the “value added” by the institution, in filling out the survey.

If schools can rise and fall based on the education they provide, they will have the incentive to improve, and less of an incentive to “game” the system. We might just get a race to the top in legal education. This project aims to provide some information to help make this possible.

Distributive Justice in Law Schools

2009-10-04T19:43:09Z

Sometimes I feel like it’s Bill Henderson’s world, and I’m just living in it and trying to help connect the dots. So when John McCain talks about “spreading the wealth,” I start thinking about distributive justice, specifically who gets what in the law schools that employ some of us, collect tuition from others, and ask still others for money every once in a while.

Our current system of no competition on educational quality among law schools, which I’m trying to help address with the Race to the Top project (download the Guide to the U.S. News survey), has serious consequences for distributive justice in law schools.

The first is how we allocate the scarce resources of admission slots and financial aid. I talked a bit about this yesterday, but the basic answer is LSAT scores, as Henderson recently demonstrated. Financial aid and fundraising priorities goes to buying LSAT scores to move up in the rankings, when it could be going to expanding loan repayment programs for public interest or government jobs, or any number of other priorities.

Now, competing for the best students through merit-based aid doesn’t sound so bad — a bit of a waste of money from a public-good perspective — but not terrible. Until you think about how merit is defined: test-taking speed, which is what the LSAT is about in large part.

And, as Bill Henderson has demonstrated in what has to be one of the most important law review article of the past twenty years, the only reason why the LSAT is a good predictor of law school grades is because most law school grades are determined by these time-pressured exams, having little to do with analytic ability or other skills relevant to quality lawyering, and everything to do with speed.

Which brings us to distributive justice problem #2: the next scarce resource we allocate is access to top jobs — at most law schools, they’re only accessible to the top of the class.

And what Henderson and others have demonstrated is that who falls where on the curve is different depending on the assessment method professors choose. That is, if you use a few short memo assignments instead of a time-pressured final exam to determine the grades, different people will get “As” and access to the top jobs. Doing memo assignments — or final exams with word limits and no heavy time pressure (take-home, 6-8 hr, etc.)– does a much better job of sorting people by analytic ability and work ethic than the time-pressured final exam. Mike Madison (Pittsburgh) and others have ably discussed the virtue of memo assignments — one benefit for professors is that doing most of the grading during the semester frees up valuable time for uninterrupted writing at the end. I’ve found this to be a huge benefit, and on take-home exams or memo assignments, I’ve never had a problem doing a curve.

So the question for those law professors (until this year, myself included) who continue to use time-pressured exams to determine most of the grades is: why are you choosing speedocracy over meritocracy?

In the status quo, the speedy high-scoring LSAT folks get the best grades on the first-year time-pressured exams that determine grades, giving them the access to the top jobs that generally pay the most, making them the least in need of significant financial aid, while continuing to receive the most aid.

I’m not the first to criticize legal education on such grounds, I realize, but anyone else think it’s time for a change? Law professors can act locally, of course, but competition on quality, through the U.S. News rankings, is the easiest way to do this globally and bring about the change we need. In the Voters’ Guide we published earlier this week, we highlighted a set of schools that use “best practices” in legal education such as multiple assessments, feedback during the semester, and less reliance on time-pressured exams — if U.S. News voters would award these schools high marks, we could have a race to the top that would help students learn more and better, and make law schools more meritocratic.

You can contact us at rtttlaw@gmail.com if you want to help; we could use it. In the meantime, I’ll shut up for a while until after this other election. Thanks for listening.

Law Schools Competing on Quality

2009-10-04T19:45:23Z

Why should anyone care about the stupid U.S. News survey anyway? According to a commonly held view, the rankings are silly, and the thing to do is ignore them. But I think this view is quite misguided.

It turns out — and this is the basic premise of the Race to the Top project that I helped start recently — that a major obstacle to the improvement of legal education generally is the lack of competition on quality among peer institutions, and that this lack of competition also leads to other bad consequences for law schools like spending lots of money on buying LSAT scores and shifting full-time students into “part-time” programs. And the easiest way to address both sets of problems is by taking the U.S. News rankings more seriously, not less, and focusing on this survey.

What would such competition look like? In the Voter’s Guide we sent out earlier this week to U.S. News voters, we said: “For example, take Penn and Northwestern, two national schools that compete for students and are close in the overall rankings. Both have very high student satisfaction and bar passage rates. But consider the curricular differences in areas particularly important in preparing students for practice: Northwestern has top-10 (or close) legal writing, clinical, dispute resolution and trial advocacy programs in last year’s U.S. News surveys of faculty in these fields. Penn is not ranked in any of these areas, and is one of the few remaining law schools that uses third-year law students to teach 1Ls legal research and writing. Northwestern is also moving towards an increasingly innovative, practice-oriented curriculum, all of which suggests that Northwestern has a higher-quality J.D. program than Penn.”

This kind of head-to-head comparison is completely lacking — there’s been no information out there on the relative quality of the education provided at different schools — and as a result, U.S. News voters simply replicate the previous year’s overall US News rankings when filling out the surveys. Glossy brochures notwithstanding, these quality assessment ratings rarely change from year to year, and when they do change over time, it is in response to a shift in a school’s overall ranking (driven by higher LSAT scores, for example), not any underlying shift — of reality or perception — on the quality of the JD program. By the way, if you don’t like the criteria used above to compare schools, would love to hear what existing data you would look to instead in assessing the relative quality of a school’s JD program.

To understand why the lack of competition on quality has other bad consequences, recall there are four basic components of the U.S. News formula:

40%: Quality Assessment, from surveys of law professors (25%) and lawyers/judges (15%)

25%: Student Selectivity, from LSAT Scores (12.5%), UGPAs (10%), and Acceptance Rate (2.5%)

20%: Placement Success, from Emp rates at graduation (4%), 9 months out (14%), and Bar Passage (2%)

15%: Faculty Resources, from Expenditures per student (11.25%), Student-Faculty Ratio (3%), and Volumes in Library (.75%)

So since schools can’t move up on the quality factor (40%) in the rankings, what do they do? They start competing on the next biggest category in the U.S. News formula — LSAT scores and undergraduate GPAs — by emphasizing these things more in admissions, and throwing money at (buying) higher credentials. Bill Henderson provides evidence of this trend here. How much money is your school spending on “merit-based” financial aid, and how is merit determined? I’m guessing it’s not based on valuable graduate training in another discipline, interesting work experience that indicates potential excellence as a lawyer, or being the first in the family to go to a professional school.

Are we really any better than Baylor, which literally paid people to retake the SAT? I’m not so sure. Here’s our deal: take that Kaplan course if you can afford it, work really hard studying for the LSAT, and if you’re speedy enough, we’ll give you a full ride. Sounds like paying for LSAT scores to me; we’re only a tad more subtle.

The good news is we can fix this if we want to. It’s actually not this pesky magazine controlling our priorities — we (law professors and lawyers) control the U.S. News rankings, 40% of it, the largest category by far. If we have real competition on quality, there will be less need for schools to compete on other things. We just need to get enough information flowing to make competition on quality possible, and then start filling out the survey accordingly. I hope those voting this month and next will start now.

The Lowest Top-20 Schools on Student Satisfaction

2009-10-04T19:48:26Z

Over at TaxProf Blog this week, Paul Caron is doing a great series of posts unpacking the data from the new The Princeton Review’s Best 174 Law Schools. One post is on the “academic experience” rating, which Princeton Review describes this way:

Academic Experience Rating: The quality of the learning environment, on a scale of 60 to 99. The rating incorporates the Admissions Selectivity Rating and the average responses of law students at the school to several questions on our law student survey. In addition to the Admissions Selectivity Rating, factors considered include how students rate the quality of teaching and the accessibility of their professors, the school’s research resources, the range of available courses, the balance of legal theory and practical lawyering skills stressed in the curriculum, the tolerance for diverse opinions in the classroom, and how intellectually challenging the course work is.

Because it incorporates admissions selectivity, the elite schools should all do quite well here, and indeed 17 out of 20 score above 90. The three that fell below 90: Southern Cal (88); Yale (87); Cornell (63).

Judging from the narrative sections in the book, USC’s relatively low score (95 for UCLA) may be due in part to too much theory/not enough practical from some professors. Yale: indifference to teaching among some faculty seems to be the culprit. Yikes on Cornell. Sure, discount it a bit for weather/location and not being first-choice school of many, but still, that’s awfully low. Students seem to complain about range of courses offered, small size of faculty. May also be they’re working harder (5.5 hrs a day outside class) than peers at other places.

I’ve said before why I think these Princeton Review ratings ought to be a factor for U.S. News voters — student satisfaction is a very good indicator, compared to available alternatives, in assessing the academic quality of J.D. programs, and commonly used in other rankings schemes like Business Week’s for business schools, for example. And we look to consumer satisfaction as a proxy for quality with all kinds of services — not clear why legal education is so different.

If we only knew the response rates (or at least a minimum for each school), I’d say U.S. News voters ought to use these as the major factor, and all prospective students ought to go out, buy the book, and use it instead of U.S. News in deciding among JD programs. But Princeton Review doesn’t release that information. So I’d use it as a “bump up” if unusually high, “bump down” if unusually low.

Guide to U.S. News Survey

2009-10-04T20:00:13Z

It’s the final days of the election for many law professors: that is, the U.S. News survey asking voters to assess the quality of each school’s JD program is due on Thursday. The Race to the Top project, which Dave Fagundes and I started a few months ago, just put out our first guide in time for the final voting from the academy, and in advance of the lawyers/judges survey next month. We emailed it yesterday to all the law professor voters, but if you didn’t get it by chance, you can download it at our website.

For another indicator of the quality of the JD program, look at the student satisfaction data extracted from The Princeton Review by Paul Caron over at TaxProf Blog — the key is to compare peer institutions in order to give different ratings to competitors, and thereby promote competition on quality. If everyone in the top 20 gets the same score — a “5,” for example — then they just end up competing on who can throw the most money at students with high LSAT scores. This is essentially the status quo.

Over the next few days, I’ll say more on why people who care about things like justice and meritocracy should care about this U.S. News survey — yes, there’s a presidential election, a global economic crisis, a quite-possibly innocent man about to be executed here in Georgia, and a few other things — but this isn’t just navel-gazing. It matters.

Penn's Rankings Problem

2009-10-04T20:01:42Z

As U.S. News voters figure out what rating to give each school, and start focusing more on educational quality, Penn Law seems to be quite well-positioned — sky-high student satisfaction (”academic experience” rating of 96 in Princeton Review), great bar passage rates, on curriculum, we’ll have to see what they submit for their “Best Practices” survey today (thanks to all who have submitted so far!).

But Penn faces a real ceiling on these “quality assessment” surveys: its legal writing program. Like Yale, it’s taught by 3Ls. This ceiling prevents Penn from having an “outstanding” JD program (”5″), instead, I’m inclined to think they should get a “4″ (”strong”). Particularly where one of its chief competitors, Northwestern, has top-10 legal writing, clinical, dispute-resolution and trial advocacy programs (Penn’s nowhere on any of these lists, from last year’s U.S. News surveys) and an increasingly innovative, practice-oriented curriculum, all of which point to a “5″ in the survey, Penn needs to fix this soon.

Here’s what the recently released Princeton Review “Best 174 Law Schools” says:

The only gripe that many Penn students express is with the first-year legal writing program. While some report positive experiences, many complain that the program is of poor quality and ‘instructed by third-year law students that often don’t have a lot of real-world experience outside of the summer clerking opportunities.

Remember, the question in the U.S. News survey is to rate 1-5 the quality of the school’s J.D. program, and so some relevant indicators include: bar passage rates relative to entering credentials; levels of student engagement and satisfaction from the recently released Princeton Review law school rankings and implementation of findings from the Law School Survey of Student Engagement; and the strength of the curriculum, particularly in critical areas like legal writing and clinical offerings.

Why give so much weight to legal writing? Three reasons: (1) one of the most important skills for lawyering; (2) arguably the most important class in law school (I think so); and (3) frequent complaints from lawyers about new graduates’ ability to communicate effectively in various forms.

So Penn, it’s time to spend some money on real legal writing professors. The people who head Penn and Yale’s progams may be terrific, but there’s only so much one person can do. The law student instructors may be doing a good job given what they know, but… they’re law students. Georgetown has moved away from this model in last few years — are there any more schools out there that still do this? My colleague Hillel Levin’s excellent and ongoing series of posts on legal research and writing didn’t even mention this 3L model — I assume, because it’s so rare these days.

Last year, Penn’s quality assessment # was 4.4 from lawyers/judges, and 4.3 from law professors. I would have been inclined to recommend giving Penn a “5″, and still want to see what they submit on Best Practices of course — but until Penn beefs up legal writing, I’m inclined to give Penn a “4″ and hope you do the same.

Bar Passage: A Key Factor to Look To in USN Voting

2009-10-04T20:03:01Z

For those filling out the U.S. News survey rating the academic quality of J.D. programs across the country, one logical question is what kind of information one ought to look at to make such determinations. Here’s one key piece of data: bar passage rates relative to entering credentials.

So if we look at schools that have students with not-great entering credentials, but high bar passage rates in recent years — that’s a good signal that the quality of the JD program is relatively strong.

Two possible objections (and others welcome) on this as a metric: first, this encourages and rewards “teaching to the bar.” My response is: well, the school that has pulled off one of the biggest bar-passage miracles of recent times, New York Law School, raised bar passage rates — 57% to 90% — primarily by teaching struggling students analytic skills. See Dean Matasar’s description of how they did it here (p. 3 of pdf). Intensive training in analytic skills for struggling students? Sounds good to me.

Second objection is: bar passage is already included in the U.S. News formula — why double count it? The response is: bar passage counts for next to nothing (2%) in the US news formula, and it’s considered on an absolute, not relative, basis. So Yale gets essentially the same credit for achieving a 90% bar passage rate in New York as New York Law School does, working with students with far lower entering credentials.

Below is the list we have so far, and thanks to Bill Henderson for pointing us in the direction of some of these schools.

I’m quite sure we’re missing some, and we’re working on finalizing the list for that Voters’ Guide out next week — so please let us know other schools that might be considered to be in this category.

Schools that Achieve High Bar Passage Rates Relative to Entering Credentials:
Campbell (NC)
Cardozo
Duquesne (PA)
Florida Coastal
Florida International
Mercer (GA)
New York Law School
North Carolina Central
Northeastern
Texas Tech
University of Memphis
University of San Francisco