eduwonkette

eduwonkette hangs up her cape

Mon, 26 Jan 2009 20:31:09 -0500

Today, dear readers, I'm hanging up my cape. I'm joining the Sociology department at NYU this fall, and have a dissertation to wrap up that needs my undivided attention.

Last night, I peeked back at my initial post in September 2007:

Rather than stepping into [education policy's] ideological boxing ring, this blog takes a different approach. eduwonkette will consider some of the most contentious education policy debates from a different perspective....My hope is that this blog provides a forum for a broader discussion about the possibilities for, and barriers to, improving schools for American children. But eduwonkette is not all about being wonky. We'll have some fun, too.

I never imagined that this blog would turn out to be the amazingly dynamic forum that it has become, and have all of you to thank for that. It's been a heckuva lot of fun.

I'm grateful to Education Week for hosting me over the last year, and especially thank Ed Week's Jeanne McCann, who has been incredibly supportive of this blog in all matters big and small.

And I couldn't have wished for a better blogging partner than Aaron Pallas, aka skoolboy. His wit and wisdom will live on over at Gotham Schools, where he will be occasionally blogging.

So I guess this is goodbye, at least for now. I'll leave you with some classic superhero fare:

Harvey Dent: We've received a letter from Batman this morning. "Please inform the citizens of Gotham that Gotham City has earned a rest from crime. But if the forces of evil should rise again to cast a shadow on the heart of the city, call me."

Wish #1: Taking Kids' Out of School Time Seriously

Mon, 26 Jan 2009 12:15:18 -0500

Sexy sound bite solutions dominate education policy airtime – paying kids for test scores, paying teachers for test scores, and eliminating teacher certification are some of the latest examples.

My final wish for the next four years is much more banal. Kids spent the overwhelming majority of their 0-18 years outside of school – the average American has spent 87% of his or her waking hours outside of school by age 18. If we want to reduce inequality, we need to start thinking seriously about how kids are spending the majority of their time, and support interventions like pre-school and an extended school day and year.

Even if all kids attended schools of identical quality, we would still see inequality in educational outcomes by socioeconomic status because of the 87% conundrum. Home learning environments, it turns out, are much more unequal than school environments. Below, this figure in a terrific paper by Doug Downey and colleagues makes this very clear. To be sure, schools offer unequal learning opportunities, but there is even more inequality in learning opportunities between families.

In other words, the difference between the best and worst schools is smaller than the difference between the most and least enriched home learning environments. As Downey put it, “As a result, a disadvantaged child attending a low-quality school can still enjoy a larger ‘school boost’ than an advantaged child attending a high quality school. In this way schools can favor advantaged students, but still serve as equalizers.”

This is not to say that we shouldn’t also focus on that 13%, but that increasing the amount of time that poor kids spend in school is a promising strategy for reducing inequality. While it is an expensive intervention, keeping poor kids in school longer – both for preschool and during the school year and summer - is a policy option that deserves more attention.

(For those who are interested in the research here, Karl Alexander found that two-thirds of the reading achievement gap between poor and rich 9th graders in Baltimore is explained by how much they learned during their elementary school summers: Karl Alexander's recent work - written up here in Ed Week - or Doug Downey and colleagues paper using data from the Early Childhood Longitudinal Study. )

Wish #2: The End of Proficiency Only Accountability Systems

Fri, 23 Jan 2009 17:55:24 -0500

The No Child Left Behind Act may represent the largest threshold-based government accountability system in the country. Schools are evaluated not by how much progress students make, but by their success in pushing students over the proficiency bar. By now, you’re probably familiar with the discontents of this system: states can game the system by setting that proficiency bar low; some schools have triaged their students, essentially reallocating resources to the kids most likely to become proficient in the very short-term; and policymakers can misleadingly make claims about declining racial achievement gaps based on proficiency rates, even as these gaps are unchanged or growing.

Proficiency-based accountability systems leave us in a terrible spot. On the one hand, we want to push kids and raise the bar for proficiency. But on the other hand, we want to make sure that the lowest performing students aren’t kicked to the curb. The higher you raise that bar, the more likely you are to have a significant proportion of students in any given school below proficiency. And those are precisely the conditions under which it makes sense for educators to allocate their time and attention strategically.

All of this, of course, should have been expected in a system focused on proficiency rather than growth. And contrary to popular belief, NCLB's growth model pilot doesn't allow true value-added models, but is instead based on a "projection model” which requires all students to reach a fixed proficiency target regardless of their initial achievement levels.

What am I suggesting? The new Department of Education would do well to let states experiment with a few different accountability systems: 1) dump proficiency altogether and identify schools as in need of improvement based on whether they are making less growth than expected. In other words, drop NCLB’s arbitrary targets and evaluate schools based on how they are doing compared to the schools we already have, or 2) keep proficiency around, but focus improvement efforts on schools that are both low-growth and low-proficiency – not relative to an arbitrary standard, but perhaps those in the bottom 15% of both categories. (That number should be set based on the number of schools to which states can provide targeted support.)

Either of those options would require significant new investments in better tests that are designed to measure growth, and careful attention to building a value-added model that is both valid and reliable. New Yorkers know well that a poorly designed value-added model at the center of the Progress Reports wreaks more havoc than no value-added model at all.

My recommendations will surely fail to impress the “no excuses” crowd (or more aptly, the “nuke the system” crowd—my belated entry into Elizabeth’s Green’s name-the-reformer contest) who see anything short of “100% proficiency” as not radical enough. “No excuses” is great rhetoric, but in the end it’s just that. So my wish #2 is that we move past this bravado in the next four years and develop a more reasonable and effective way of identifying and supporting low-performing schools in getting better.

PS: Check out Richard Rothstein's related op-ed, Getting Accountability Right, which speaks back to Wish #4 (integrating a broad set of goals of public schooling into accountability systems).

Wish #3: Asking More "Why?" Questions

Thu, 22 Jan 2009 10:37:11 -0500

Earlier this month, a team of researchers at MIT and Harvard released a report contrasting the impact of charter schools, “pilot” schools, and traditional public schools on student achievement. The finding of charter school effects on achievement, using a random assignment research design, fueled the rhetoric of charter school advocates, some of whom saw the findings as a license for unlimited expansion of charter schools.

The researchers themselves were more cautious. They acknowledged that the study was not designed to discern why the effects were found. In fact, if the study had found that students in charter schools had shown less growth in achievement than students in traditional public schools, they wouldn’t have known why either.

Good public policy depends on compelling answers to “why” questions about both the observed effects and non-effects of policies and programs. And these “why” questions pertain both to the inner workings of policies and programs as well as the context in which the policies and programs are situated. Borrowing policies that have been found to be effective in one setting and expecting the same results in another setting makes sense only if we know why the policies were effective in that first setting. A research study showing that a policy or program “worked” in a particular setting doesn’t tell us that.

Our wish, then, is for asking “why?” more loudly, and earlier in the lifecycle of a policy or program. Why might achievement be higher in charter schools? Why do children learn more in smaller classes? Why are some teachers more successful in teaching low-achieving students than high-achieving students? Why don’t school expenditures have a stronger association with student outcomes? In skoolboy’s view, the real leverage in education policy comes from good answers to the “why?” questions. To paraphrase Jim March, research that addresses “why?” questions is more useful than research that addresses “what works?” questions because it has so many more applications.

One challenge posed by our wish is that the researchers who are skilled at addressing “what works?” questions are not necessarily the ones who are good at addressing “why?” questions. Even in large federal evaluations, there typically is a division of labor in which the study of implementation and context is segregated from the study of program impacts, and different research organizations or researchers are responsible for differing parts of the overall enterprise. Asking “why?” more often will require some hard thinking about research training and the infrastructure for education research in the U.S.

Wish #4: Better Alignment of Accountability Systems to School Outcomes

Wed, 21 Jan 2009 07:00:41 -0500

Here’s a little thought experiment: Suppose that, in addition to adequate yearly progress in literacy and mathematics, high schools had to demonstrate progress in students’ ethical behavior. Would the graduates of Far Rockaway High School in Queens in New York City be as proficient in their treatment of others as they are in math and literacy?

Victims of Bernard Madoff’s $50 billion Ponzi scheme might wish that Far Rockaway had spent more time on the development of its students’ non-cognitive skills as their ability to read, write and figure. Of course, we cannot tell what led Madoff astray, and his experience at Far Rockaway probably had little to do with it. But the thought experiment opens the door to a wish for accountability systems in education that are better-aligned with the diverse school outcomes we think are important.

What skills do employers value in their workers? A 2008 survey of members of the Society for Human Resources Management found that human resources professionals reported that some skills and practices were more important for experienced workers in 2008 than two years before. More than a third of the respondents reported that adaptability/flexibility; critical thinking/problem-solving; leadership; professionalism/work ethic; teamwork/collaboration; and information technology application had increased in importance in the recent past.

The story is not that different for the general public. Asked to allocate a total of 100 points across eight goals of public education, a sample of adults divided them up relatively evenly: basic academic skills (19%); critical thinking (15%); social skills and work ethic (14%); physical health (12%); preparation for skilled work (11%); emotional health (11%); citizenship (10%); and the arts and literature (8%).

Why, if the public and employers think that these are the most important goals of public education, have we constructed accountability systems that focus on a narrow subset of these goals – basic proficiency in literacy and mathematics? Part of the answer is that we had an existing technology for measuring literacy and mathematics proficiency – standardized tests of academic performance.

Richard Rothstein, Rebecca Jacobsen, and Tamara Wilder, in Grading Education: Getting Accountability Right, argue that if these broad goals are important – and skoolboy thinks they are—then we should develop measures of these goals, and incorporate them into accountability systems. One of the things we’ve learned about education accountability systems that rely on rewards and punishments is that educators respond to incentives, doing what they can to avoid punishments and to achieve rewards associated with a particular pattern of outcomes. Particularly when the inducements are high-stakes, we are liable to get precisely the outcomes that are to be rewarded and punished – no more, and no less.

Literacy and mathematical proficiency are extremely important skills for schools to cultivate, and it’s appropriate that accountability systems monitor students’ literacy and math performance and provide incentives for educators to help students achieve challenging performance standards. But it’s also critically important for U.S. children and youth to prepare to assume the responsibilities of citizenship in a democracy that depends on a tacit social contract which binds us together, and we count on schools to do this and much more. Our wish is for accountability systems in education that are designed to measure and promote genuine growth and development in children and youth.

Wish #5: Education Policy Based on Averages, Not Outliers

Tue, 20 Jan 2009 01:25:44 -0500

To celebrate this week’s historic inauguration, skoolboy and I are going to take a step back and make some big wishes – or at least predictions - for education policy in the next four years.

And I want to start this wish in an unlikely place - with a man in a Speedo. Let’s talk about Michael Phelps. Who wasn’t astonished by his eight golds, including that insane race where he touched out his competitor by a hundredth of a second? All over the country, there are age group swimmers, probably some of them putting laps in as you read this, who want to swim as fast as Phelps. Perhaps some of them will emulate his breakfast diet after practice today: three fried-egg sandwiches loaded with cheese, lettuce, tomatoes, fried onions and mayonnaise, two cups of coffee, one five-egg omelet, one bowl of grits, three slices of French toast topped with powdered sugar, and three chocolate-chip pancakes.

Americans love outliers. The quick, the talented, the beautiful – we like to read stories about them, watch them on TV, and aspire to be like them. Up to a point, that’s all for the good.

But education policy has taken this fawning adoration to a new level. Rather than appreciating the good work outlier schools do but realizing that public policy must be crafted for mere mortals, some hold up outliers as evidence that all schools can reach a particular standard. Consider Joel Klein and Al Sharpton’s remarks in a recent Wall Street Journal piece:

Dismissing the potential of schools to substantially boost minority achievement, as is now fashionable in some Democratic circles, is ultimately little more than a recipe for defeatism….High-performing urban charter schools such as the KIPP (Knowledge Is Power Program) schools are showing that minority students can close the achievement gap if given access to high-quality instruction.

So what are Klein and Sharpton saying here? Their argument is essentially: 1) Some schools with high concentrations of minority and poor students are getting exceptional results, 2) If some schools with high concentrations of minority and poor students are getting good results, poverty must not affect academic achievement - at least not in ways that can't be overcome by good schools, and 3) If some schools can get exceptional results in spite of the challenges their students face, all schools should be able to.

I won’t belabor how flawed this logic is - because at the end of the day, it is just crazy talk. We don’t expect the other 99.9% of swimmers to be able to do what Michael Phelps can, and a swim coach that set out to reach that goal for his swimmers would be sorely disappointed. And we don’t infer that disabilities like Phelps’ ADHD can be overcome by all because one man did so. Few would disagree with the previous two sentences. But it seems that when we step into the education policy arena, we too often check our brains at the door.

So Wish #5 is a simple one: Please, dear policymakers, don’t craft your policies – or your expectations about the effects of policies – based on outlier schools or teachers.

The State of "State of the City" Speeches

Fri, 16 Jan 2009 07:09:34 -0500

While we await President-Elect Obama's Inauguration speech, here's a look at the rhetoric in ten mayors' "State of the City" speeches over the past year. Can you match the mayor with the quote?

1. We're going to demonstrate how a school community comes together when you give teachers, parents and principals real authority to make decisions in the classroom. We’re going to show how the atmosphere transforms with uniforms and parent contracts, when you instill a culture of discipline and respect. We’re going to show what happens when we set tall goals and raise expectations, when we publish clear benchmarks and hold ourselves accountable. We are going show what is possible when we make our children believe they can do anything.

2. In our schools, we have decreased the achievement gap and increased learning.

3. And on this 80th anniversary of Dr. King's birth and on the eve of the inauguration of our first African-American president, we can all be proud that African American and Latino students are leading the way in the rate of improvement.

4. No one wants to see their school closed, and there was controversy, but the leaders of our schools held their ground because they knew the change would help our kids, especially those with the greatest learning gap. And a year later we are beginning to see the positive results.

5. As adults, we have a responsibility to create hope in the lives of our children.

6. We’re continuing to work with nonprofits like the Bill and Melinda Gates Foundation to double the number of Class of 2010 students graduating from college -- and triple the number for the Class of 2013.

7. Our graduation rate remains dangerously low -- and while they talk about a world class education system, our Legislature slashes nearly a billion dollars in funding for our children. We refuse to accept the growing technology gap between children who will compete in the global economy, and those who, not by their choosing, will watch the world pass them by.

8. In 2007, 55 percent of seniors graduated -- the highest percent age since 1995. This year, we are working towards a goal of 60 percent. Each year, we will be working to increase graduation rates.

9. Anyone who believes in libraries also knows their importance to a major city -- not as monuments to civic pride, but as doors to education and opportunity.

10. All over our City we are seeing educational excellence in public and private schools -- beacons of light, illuminating the way forward.

a. R.T. Rybak, Minneapolis
b, Jerry Sanders, San Diego
c. Thomas Menino, Boston
d. Tom Barrett, Milwaukee
e. Manuel Diaz, Miami
f. Antonio Villaraigosa, Los Angeles
g. Adrian Fenty, Washington, D.C.
h. Cory Booker, Newark
i. Frank Jackson, Cleveland
j. Michael Bloomberg, New York

(Answers in the comments at the end of the day.)

Lies, Damned Lies, and Bush Administration Accomplishments

Tue, 13 Jan 2009 07:02:07 -0500

Yesterday, President George W. Bush, as part of his swan song, released a compendium entitled “Policies of the Bush Administration 2001-2009.” Not surprisingly, No Child Left Behind is the centerpiece of administration’s accomplishments in K-12 education, and the fact sheets detail the administration’s claims about progress.

Skoolboy’s favorite section is the one on Reading First. You remember Reading First, don't you? The program whose interim impact evaluation, sponsored by the Institute of Education Sciences, found no evidence of effects on reading comprehension test scores in grades 1 to 3? The text reads:

NCLB established the principle that Federal funding should be invested in programs that have rigorous research demonstrating their effectiveness. Reading First has provided more than $6 billion to fund scientifically-based instructional programs, valid and reliable diagnostic assessments, and professional development for teachers. State data shows that Reading First students from nearly every grade and subgroup have made impressive gains in reading proficiency. For first grade, 44 of 50 States reported increases in the percentage of students proficient in reading comprehension; for second grade, 39 of 52 States reported improvement; and for third grade, 27 of 35 States reported improvement.

“52 States”? Maybe we should have invested a bit more in Math First.

Okay, cheap shot, there are 54 state education agencies (SEA’s) that received funds under Reading First, including American Samoa, the Bureau of Indian Education, the District of Columbia, and the Virgin Islands.

But seriously: How did “44 of 50 States” report increases in the percentage of first grade students proficient in reading comprehension when, according to the American Institutes for Research compilation of Reading First Annual Performance Reports from 2003-2007, only 40 of the 54 SEA’s even reported reading comprehension proficiency for first grade students for two or more years?

Professor Bonuses Based on Course Evaluations?

Mon, 12 Jan 2009 09:58:11 -0500

Forget a PhD - profs might do better with improv training and Botox (seriously, attractive professors - especially attractive men - get better evaluations!). Reported this morning in the Chronicle:

The chancellor of the Texas A&M University system wants to give faculty members bonuses of up to $10,000, based on student evaluations, but some professors are raising concerns about the plan, saying it could become a popularity contest....Though details are preliminary, officials said, the goal is to offer awards starting at $2,500 to the top 15 percent of participating instructors.

To be sure, good teaching is insufficiently rewarded at most colleges and universities. But according to the Chancellor, “This is customer satisfaction....It doesn’t have to do with tenure, promotion, status. It has to do with students’ having the opportunity to recognize good teachers and reward them with some money.”

But is good teaching the same thing as "customer satisfaction?" Is there a better way to recognize and reward effective college teaching? Or, as one commenter at the Chronicle augured, are we left with this bargain?

Wow. What an easy way to make up to 10,000 dollars! Just water-down my courses, make no demands on students, make sure to bring donuts or some sort of snacks on the day of evaluations, and make sure my students know I think they are incredibly bright and they will get an A. Cha-ching!

Private Practice, Public Health, and the Autism/Vaccine Debate

Fri, 09 Jan 2009 10:54:25 -0500

"Private Practice" - the "Grey's Anatomy" spinoff starring Kate Walsh - may be one of the most poorly written, bad excuse for soft porn shows on TV. But big props to the show for last night's episode/public service announcement, which has already ignited a firestorm on the autism blogs.

The episode featured a mother with three sons, and the oldest was autistic. Blaming the MMR vaccine for her child's autism, she chose not to vaccinate her two other sons. An unvaccinated son picks up measles while the family is in Switzerland seeking autism treatment, exposes the entire practice when he comes in, and dies at the end of the episode.

Think this is implausible? Check out this graph of the growth of measles cases in the UK over the last 12 years, courtesy of the BBC:

And it's not just the UK. During the first seven months of 2008, 131 measles cases were reported to CDC, compared with an average of 63 cases per year during 2000--2007. (On a more local level, in the first four months of 2008, there were 22 measles cases in NYC.)

Parents have the right to make choices for their children, but they don't have an inalienable right to expose other people's children to deadly diseases. And if it takes a bunch of hot faux doctors to get that message out, so be it.

The Skillful Publicist

Thu, 08 Jan 2009 10:22:44 -0500

When a school district makes a big to-do about the use of "evidence to make decisions about how to help students learn, where to put our resources and how to manage our staff," is it fair to criticize it for implementing unproven and experimental programs? skoolboy supports modest experimental innovations, as long as they are evaluated carefully before expansion to a scale that would encompass an entire population. After all, students and teachers aren’t guinea pigs. The fact that schools are failing is not a justification to do any old thing, on the assumption that any innovation will be better than the status quo.

Speaking of any old thing … The Washington Post reported earlier this week that the Washington, DC Public Schools are abandoning support for National Board certification as a means of teacher professional development, shifting instead to, among other things, the Skillful Teacher program marketed by Research for Better Teaching, Inc. (RBT), founded by Jon Saphier in 1979. The program consists of a series of six one-day workshops; you can buy the book, which approaches 600 pages, for $70 at Amazon.

You might think that an organization that’s been peddling professional development for 30 years, with a book in its sixth edition, would have some compelling evidence of the effects of the jewel in its crown on teaching practice and student learning. If professional development doesn’t result in improvements in teaching and learning, what’s the point? But the RBT website doesn’t point to much evidence, emphasizing testimonials and brief "stories." skoolboy’s favorite is the account of Fairfax County, VA’s implementation, "Making Teacher Evaluation Substantive and Growth-Oriented." "In the first year of implementation, 162 teachers were dismissed or resigned compared to single digits the previous year," the website crows. Now, is that growth, or is it development? Sometimes skoolboy gets confused by the difference.

The blurb for Montgomery County, MD, on which the DC plan is based, touts an independent evaluation by Dr. Julia Koppich of the program’s effects on teachers and administrators, and claims that "in 2001 grade 2 students scored in the 68th percentile in math computation. In 2003 scores were in the 83rd percentile." The inference is that the Skillful Teacher program produced this change, but any reader of this blog knows that demonstrating program impact requires a careful design to rule out alternative explanations of changes over time in outcomes. (It also helps to have a good theory of how a program might plausibly produce particular changes.)

Montgomery County’s own internal evaluations of Studying Skillful Teaching aren’t as positive. Although 3rd grade teachers and Alegebra I teachers who took the course are "more likely to teach mastery lessons and less likely to miss opportunities to positively impact student learning" than comparable teachers who did not (Merchlinsky, 2006, 2007), there were no effects on elementary reading and math test scores or algebra performance.

And lest anyone think that people who live in glass houses shouldn’t throw stones, The Skillful Teacher isn’t the only professional development initiative that skoolboy, who teaches at the home of the Teachers College Reading and Writing Project, thinks could benefit from more rigorous evaluations before scaling up.

The Boston Pilot/Charter School Study: Some Good News, and Some Cautions

Wed, 07 Jan 2009 11:46:41 -0500

When Robert Pondiscio pages, I answer. Yesterday, the Boston Foundation released a study on the efficacy of charter and pilot schools, which had the advantage of including both observational estimates of these schools effectiveness (comparing the performance of students in these schools with those in traditional public schools, net of some control variables) as well as lottery-based estimates of their effectiveness (comparing those who won lotteries with those who did not).

Kudos to the research teams (Atila Abdulkadiroglu, Josh Angrist, Sarah Cohodes, Susan Dynarski, Jon Fullerton, Tom Kane, and Parag Pathak) - this is a well-done, careful study that provides us with a range of estimates of charter and pilot school performance. There is certainly enough positive evidence here to support the creation of more charter schools in Boston, but I want to offer two cautions.

1) Beware of extrapolating lottery-based estimates when only the most successful charter schools had oversubscribed lotteries to being with.The lottery-based estimates of the effects of charters on middle school math performance are simply huge - half of a standard deviation! But only a quarter of charter schools are part of the lottery sample. Zero of the 5 charter elementary schools, 4 of 13 charter middle schools, and 3 of 11 charter high schools are included in the lottery part of the study because other schools were not oversubscribed enough to require a lottery. To be sure, these lottery estimates are important to understanding what the most successful charter schools can do. But I think we will all be disappointed if we expect a charter school expansion to replicate the effects of these middle schools (Academy of the Pacific Rim, Boston Collegiate, Boston Prep, and Roxbury Prep), which are among the highest performing charter schools in the country.

2) Pay careful attention to what elements of these schools can and can't scale. Harvard graduates don't grow on trees. These schools have faculties drawn from the most selective colleges - faculties that are highly atypical of the public school teachers.

LDH, IES and the Reign of Frogs

Tue, 06 Jan 2009 08:32:38 -0500

Okay, barring the bad karma that seems to hang over the state of Illinois, Arne Duncan is now firmly ensconced as President Barack Obama’s nominee as Secretary of Education, thereby forestalling the Apocalypse predicted by the detractors of Linda Darling-Hammond. But eternal vigilance is the price of freedom, or something like that, and the fears have shifted to the future of the Institute of Education Sciences (IES), the federal government’s arm for education research and evaluation. Founding Director Grover “Russ” Whitehurst has moved on from his six-year term, and there are researchers lying awake at night in fear that President Obama might choose LDH, a one-time colleague of skoolboy at Teachers College, Columbia University, as his successor. I’m not entirely sure what they are afraid of, but clearly the dismantling of the research infrastructure built up over the past six years is near the top of the list.

If Linda Darling-Hammond is to serve in the Obama administration, skoolboy is not sure that the post of Director of IES would be the best use of her talents. But even if she were to be appointed to this post, I don’t think it would start raining frogs. Checks and balances on the actions of the IES Director abound, including the National Board for Education Sciences, which has the responsibility of approving the research priorities proposed by IES. Members of the National Board for Education Sciences are nominated by the President and confirmed by the Senate. The current chair of this board is Eric Hanushek, Senior Fellow at the Hoover Institution at Stanford University, and the vice-chair is Jon Baron of the Coalition of Evidence-Based Policy. And let’s not understate the difficulty of rapid change in a complex bureaucracy where most of the work is done by career civil servants, not by political appointees. (At least, that was my experience many years ago in a prior incarnation of IES. Working under Checker Finn, by the by.)

But the real problem is not with the organization of IES, which has made great strides in setting out criteria for fundable research, and implementing a rigorous peer review system that pushes ideological predispositions to the sidelines. (I serve on one of the IES standing review panels, along with a couple of hundred other researchers.) Rather, the problem is the failure of the education research community to develop a stockpile of effective educational interventions shown to work in multiple contexts. A quick overview of the kind of research that IES has been funding over the past four years tells the tale.

IES has adopted a progressive strategy, modeled on clinical trials in medical research, for funding education research to improve academic outcomes. Identification studies are intended to use existing data to identify existing programs and practices that are associated with better academic outcomes. A successful identification study will lead to a development project, in which a new education intervention (e.g., a new curriculum, a new instructional approach or program) is developed, and some preliminary data on effectiveness are gathered. Promising interventions are then evaluated in local settings using rigorous experimental and quasi-experimental methods designed to discern the efficacy of the intervention in those settings. Finally, those interventions demonstrating practically and statistically significant impacts on participants are “scaled up” – implemented more broadly in multiple settings, with multiple groups of participants, and without the direct involvement of the intervention developers in the replication sites.

Of the 275 regular education and special education research grants awarded by IES in research competitions between 2004 and 2008, only 7 – less than 3% – were scale-up projects, and an additional 26% were efficacy projects. The vast majority were development and identification projects that lacked prior evidence of program impact. Such projects are the raw material that might eventually lead to interventions that work at scale; but many will not, and even for the promising interventions, it may be many years before we are confident that they work as intended. That will be true no matter who inherits the IES Directorship.

Cool Teachers You Should Know: Remembering Bob Kiessling

Mon, 05 Jan 2009 00:41:46 -0500

This blog has a history of profiling "cool teachers you should know," and I wanted to start off 2009 by celebrating the life and work of Bob Kiessling, a legendary New Jersey math teacher and cross country/track coach who passed away over the holiday break.

Throughout my junior and senior years of high school, Bob Kiessling and I met in the dark at 5:30am and put in 5-7 miles. He wasn't the coach at my school, but my friend ran for his team and convinced me to tag along over the summer. Because of an arcane NJ sports rule, I couldn't run with his team during the school year. So Kiessling - in his inimitable, gruff Clint Eastwood style - decided we would run together in the mornings. He didn't want to be paid, patted on the back, or even thanked for coaching me. There were only two conditions: I had to sport a geeky reflector vest so we didn't end up as roadkill, and if it was at least 45 degrees, I had to wear shorts.

It is easy to point to the eight state championships that Kiessling's cross country and track teams won, multiple "coach of the year" awards from New Jersey track associations, his acclaim among fellow math teachers, or where his runners went on to run in college as evidence of his impact. But what struck me at his funeral on Saturday was the range of 500 former students and runners who came to pay their respects. Alongside his current students at Haddonfield Memorial High School were former students in their 30s and 40s - students he taught decades ago. Most of us had stayed in touch with Kiessling all this time. 10, 20, even 30 years later, so many of us remembered him as the teacher who made the biggest difference in our lives.

As a researcher, I think a lot about teaching - how we measure its impact, what it means for a teacher to be good, and how we motivate and support the right people to enter and stay in the classroom. Bob Kiessling has always been part of the answer to those questions for me. If you have a few minutes today, take a look at some of the testimonials from former students, parents, and colleagues that flooded in online. He will be missed.

Survivor: The TFA Edition, II

Wed, 24 Dec 2008 07:37:00 -0500

Yesterday, I wrote about Morgaen Donaldson’s research on the survival rates of three cohorts of Teach for America teachers in their initial placement schools and in teaching overall. Today, I’ll describe one of her analyses of why TFA teachers leave their schools, focusing on the complexity of the teaching assignment and the corps member’s academic preparation for the subject(s) that she or he taught.

For this analysis, a complex teaching assignment for an elementary school teacher is one in which the teacher teaches more than one grade in a given year. Similarly, a complex teaching assignment for a secondary teacher is one in which the teacher was assigned to teach more than one subject in a given year. Many TFA recruits had complex teaching assignments during the years of observation. Between 16 and 20% of elementary TFA teachers were assigned to teach more than one grade in a given year, and 35% to 50% of secondary TFA teachers were assigned to teach more than one subject in a given year. (Note that this is different than teaching one grade in 2003 and a different grade in 2004, or one subject in 2004 and a different subject in 2005. These too might make teaching more complicated, but it’s across years rather than within them.)

In the 2000, 2001 and 2002 TFA cohorts, the vast majority of corps members majored in the social sciences or humanities in college. 52% were social science majors, 20% were English majors, 3% majored in the arts, and 6% majored in a foreign language. In contrast, about 5% were math majors, and 15% majored in science, computer science or engineering. Just 2% were education majors, and 4% majored in other subjects. (I think the numbers can exceed 100% due to double majors, but the text isn’t entirely clear on this.)

But their teaching assignments often differed dramatically from their formal academic preparation. One-half to three-quarters of the TFA recruits teaching secondary math were not math majors, and 38% to 50% of science teachers lacked a science major. Even in social studies, 16% to 31% of the TFA teachers were teaching out of their major field. Donaldson reports that out-of-field teaching diminished the longer a TFA teacher stayed in teaching.

Among elementary TFA teachers, a multiple-grade assignment increased the odds of leaving the initial school during or at the end of the first year of teaching by a factor of 3.29 (a probability of 19.1% for multiple-grade teachers, and 6.7% for single-grade teachers.) For the most part, this type of complexity did not influence retention in subsequent years, with the singular finding that in year 4, multi-grade teachers were significantly less likely to leave their initial schools than single-grade teachers. Many of the multi-grade teachers leaving their initial placement school in the first year transferred to another school, but multi-grade teachers also were more likely to leave teaching altogether in the first year than single-grade teachers.

At the secondary level, TFA recruits teaching multiple subjects were more likely than single-subject teachers to leave their initial placement schools and the field of teaching altogether in their first year of teaching. Beyond this first year, however, there were no significant differences in the likelihood of leaving the initial placement school, but multiple-subject teachers had a greater chance of leaving teaching altogether.

The out-of-field teaching story is complicated, with TFA math teachers teaching out of field more likely to leave their initial placement schools and the occupation of teaching altogether, and social studies teachers teaching out of field more likely to leave teaching. Oddly, science teachers lacking a science major were less likely to leave teaching than science teachers with a science major.

These patterns suggest that, at least in the years 2000, 2001 and 2002, TFA teachers often faced very complex teaching assignments for which they were not well-prepared academically, and the complexity of these assignments heightened the risk of leaving the initial placement school or of leaving teaching altogether. As I noted yesterday, there’s no comparison group, so we don’t know if novice teachers in these schools arriving via the traditional route had similarly complex assignments. Nor do we know if this pattern holds for more recent cohorts of TFA recruits, as there have been six cohorts since the three that Morgaen Donaldson studied.

One thing seems clear, however. If we want novice teachers to stay in their initial schools and to stay in teaching, they need adequate support as they learn their craft in the first years of teaching. Asking teachers to teach multiple grades, multiple subjects and/or subjects out of their college major fields is a peculiar way of supporting them.

A program note: We're going to take a break here for the next 10 days or so. eduwonkette and I wish you happy holidays!