eduwonkette

Cool People You Should Know: Marta Tienda

Fri, 18 Jul 2008 08:26:12 -0500

When I read about the University of California's proposed changes to their admissions standards, which would deemphasize test scores in favor of class rank (Hat tip: Education Optimists), I realized that proposal is a partial outgrowth of a decade of work on higher education access by Marta Tienda. Among educational researchers, Tienda, a demographer and sociologist who teaches at Princeton, stands out for her record of doing research that informs public policy debates about educational opportunity for disadvantaged kids, and for the passion and flair with which she does this work.

After the Hopwood case temporarily ended affirmative action in Texas, the state adopted a plan that gave the top 10% of each graduating high school automatic admission to the two flagship campuses. Tienda mounted a major study of the policy change under the auspices of the Texas Higher Education Opportunity Project, and has since produced dozens of articles on the policy's impact. Her recent paper with colleague Sigal Alon, Diversity, Opportunity, and the Shifting Meritocracy in Higher Education, is a real tour de force for the ground that it covers. They analyzed four datasets covering high school classes graduating in the 1980s and 1990s in order to determine how the relative weights put on grades and test scores in the admissions process have changed over time. Their results support the "shifting meritocracy" hypothesis; selective colleges have increasingly relied on test scores to screen students, which, because of persistent test score achievement gaps, has made it difficult to admit a diverse group of students to these colleges in the absence of race-sensitive preferences. Alon and Tienda, through a series of simulations, show that deemphasizing test scores would allow institutions to admit diverse classes without lowering graduation rates.

Their conclusion is far-reaching, but can be summed up in just one sentence: "The apparent tension between merit and diversity exists only when merit is narrowly defined by test scores."

Tienda's personal story is also quite remarkable. As one of five children and the daughter of illegal Mexican migrant laborers, she planned to become a hairdresser until the 7th grade, when a teacher noticed her talent. As she explains in an interview, "It was such a riveting moment for me that I even remember what the teacher was wearing that day. Until then, I thought that college was only for rich people and I was from a working class family. But when my teacher suggested college and told me that there were scholarships to help good students like me get to college, that was it." Since then, she has been awarded honorary degrees from multiple universities, and has served on the boards of RAND, the Carnegie Foundation, the Sloan Foundation, Brown University, TIAA, the Federal Reserve Bank of New York, and a dozen other major organizations.

But she's never forgotten where she came from, and continues to do outreach work to encourage disadvantaged kids to go to college, and to make them feel comfortable once they're there. In this profile, you'll see her take no prisoners attitude in full effect. As she exhorted a group of students to carry the torch, "Let me tell you something. If you were admitted, you belong. Your job is to do the very best you can and to bring up two of the classmates you left behind.” Through her research, which has drawn our attention to the potential for disadvantaged kids to succeed and excel in higher education, Tienda has done just that.

Thinktank Thursday: Fordham's Boy Band Breaks Up! (Inside the Music)

Thu, 17 Jul 2008 11:05:47 -0500

Boy bands have a short shelf life. Not long after "Rage Against the Rothstein" was born, things fell apart. Fordham's new research director, Amber Winkler, came onto the scene and declared the boy band kaput. We already had a glimpse of Christina Hentges in the "Fordham Factor" videos (she also turned heads with her deadpan performance in their spoof commercial, Ed in '08 "This is what $60 Million Gets you.) Then on Flypaper, Amber and Christina were joined by Fordham's new blogette contributor, Stafford Palmieri. Still no one had seen Coby Loup, which led me to suspect he was a robot crafted in Checker's basement.

All of this is a long way of saying that Fordham's stock picture for this website no longer cut the mustard. Hence, I rebranded Fordham as the grown-up version of Disney's hit "High School Musical." (Lucky for me, sometimes people mistake Checker Finn for Zac Efron at the Old Ebbitt Grill.) Because the show has a small core cast, I had to switch things around for High School Musical I and II.

Enjoy. And thanks to everyone at Fordham for having a good enough sense of humor to play along by providing pictures. If we were taking a "Which right of center DC thinktank would you most want to share a beer with?" poll, you ladies and gents win hands down. Stay tuned for next week's "Thinktank Thursday," in which I will give another DC thinktank a free makeover.

(Left to right from 3rd person in: Checker Finn, Stafford Palmieri, Amber Winkler, Coby Loup)

(Left to right from second person in: Christina Hentges, Liam Julian, Vanessa Hudgens (with clothes on), Mike Petrilli)

Nobody Beats the Biz

Thu, 17 Jul 2008 00:36:42 -0500

When you're looking for measured, careful, and thoughtful analysis, there is no better blogger to turn to than Dean Millot at edbizbuzz. He's consistently able to take complex debates and lay out the issues raised in an incisive and even-handed way. Millot follows up on the thinktank/peer review debate here, and promises another post tomorrow. Stay tuned.

Update: Dean Millot posts his second installment.

The Vision Vacuum

Wed, 16 Jul 2008 08:09:06 -0500

"You're too young to be this cynical, " he said, staring across his desk at me with a perplexed half smile.

I was 10, and in the middle of our classroom's simulated presidential campaign in which we followed the election and voted for candidates, my 5th grade teacher had launched into a pep talk about the potential for real change.

The last eight years have done little to temper my built-in skepticism. These are dark times, Diane Ravitch reminds us this morning. If I saw the glass as half-empty when Bush assumed the presidency, I now see it as half full - with poison.

That spin has taken over education policymaking hasn't helped. Accountability, as we used to talk about it back in the 1990s, was a way to evaluate reforms and provide incentives to implement them. It was never intended to be the reform. Now everyone's being tested and rated and graded and held accountable, but no one is supporting schools to improve the day-to-day work of teaching and learning. Policymakers say they want to leave "no child behind," but are willing to deny them health care in their next breath. We've adopted every technocratic solution that newly minted MBAs can come up with, but we have no educational vision.

So it was with cautious optimism that I followed Randi Weingarten's acceptance of the AFT presidency on Monday. As Dan Brown articulates in this post, she's a fighter, and one at the forefront of critiquing our current reform movement's easy slogans, "Too often, testing has replaced instruction; data has replaced professional judgment; compliance has replaced excellence; and so-called leadership has replaced teacher professionalism."

In her acceptance speech, which was bold and unapologetic, she embraced the proposed reforms of the Bolder and Broader coalition, and let us imagine what an alternate educational vision for public schools could look like. Watch the whole speech and let me know what you think - or just take a look at the clip below.

Lessons for Education Policy Research from the Market for Lemons

Mon, 14 Jul 2008 20:32:56 -0500

Does the market for research in education policymaking work pretty well? For once, eduwonk, Dean Millot, and I all agree - it doesn't. The “market for lemons,” which Jay Greene makes reference to in his most recent post, gives us insight into why.

A common rationale given by economists for intervention in selected markets – for example, insurance markets - is the problem of asymmetric information, a gap in information available to buyers and sellers in a market. Using the example of used car markets, Nobel prize winning economist George Akerlof lays out this dilemma in his famous paper, “The Market for Lemons: Quality Uncertainty and the Market Mechanism.”

Imagine you’re selling a used car. You know the problems with your car, but your potential buyers don’t. You may be trying to swindle unsuspecting buyers because you know it has major defects. But your potential buyers aren’t stupid, and they know that they can’t trust you to provide an honest appraisal of your car’s problems.

If buyers don’t decide to avoid this market altogether, they end up betting on averages. They’ll only pay a price that reflects the average frequency of lemons in the used car market. That’s a price that’s too high for a lemon, but too low for a car of good quality. If you’ve got a good car, you know you’re going to get too low a price in the used car market, so you’re likely to not to sell there.

When sellers of good cars refuse to sell, lemons increase in frequency in the market. As a result, the people selling good cars are really in trouble, because they will end up getting an even lower payout for a good car. Now they are even less likely to sell them there, and the frequency of lemons continues to rise.

Left unchecked, the end result is market failure. What this means is that there are people who want to buy good cars and people who have them to sell, but that they are afraid of getting stuck with a lemon keeps that trade from happening.

The situation in education policy is analogous, but a little different. Sellers in the research market know what they are selling, but buyers like policymakers, journalists, and superintendents don’t have the expertise to evaluate what they are buying. They don’t differentiate between a paper in the American Economic Review – the best peer-reviewed journal in economics – and a report issued by a pro-vouchers thinktank. Unlike the used car market, the buyers aren’t always suspicious enough, in some cases because the buyers are constantly changing and don’t have the time to build up knowledge about reputations, which help to regulate markets with asymmetric information. Journalists get moved around from beat to beat, and policymakers come and go.

For some parties, there’s no incentive to be suspicious. Stories need to be written, laws need to be pushed through, and it’s not the editor or reporter or legislator who gets stuck on the side of the road when the car sputters out. It’s the public that gets left holding an empty bag when we rely on potentially flawed research to shape public policy.

Anyone have ideas on how this market could operate better? Or do ideologically driven policymakers, who can find “research” to support just about anything, simply prefer the status quo?

Cerf-ing the Web

Sun, 13 Jul 2008 05:56:28 -0500

Over at eduwonk, the New York City Department of Education is putting its best foot forward by displaying its two strongest (and most becoming!) skills: a remarkable willingness to spin the naked facts and to personally attack anyone who questions their miracle. But Chris Cerf can't manage to slip past Sol Stern's first-rate BS detector, which is on full display in his original post and his drop-kick comment on Cerf's post, which are both must-reads.

Here's what I don't get. If you're a believer in Truth, why spin checkable facts when you're no doubt going to get busted? It's just not good government. But it doesn't strike me as a smart PR strategy either, because it gives us good reason to wonder what else is going on behind that curtain.

My concern is not so much with any individual assertion of Cerf. A much larger problem is the New York City Department of Education's willingness to swap facts in and out as they see convenient.

Here's one example: Cerf insists on taking credit for the gains in the 2002-2003 school year, though Klein's Children First reforms were announced for the first time in January 2003 at the same time that students were taking state tests. Did his words fall upon NYC kids' brains like pixie dust, and so inspire them that they produced huge gains in reading and math? The timing just doesn't add up. Yet Cerf wrote, "You frequently argue that the Mayor and Chancellor should not be given credit for the growth in achievement in their first year. To the contrary, they instituted important changes during that year. Obviously what happened in the past affected the results, just as our work will affect the results of the next chancellor, but that first year was on our watch."

But back in 2003, Joel Klein didn't want to draw attention to or take credit for the large gains that were posted that year. Klein was attempting to overhaul the entire system, and when the ELA and Math results were released in both May and September, his reaction was described as "muted." In fact, he even threw a few sentences in questioning the validity of the math scores because "it is hard to tell the true significance of any one set of results in isolation." Here's a clip from the NY Times article on the reading scores that year:

The city's positive results come at a time when Mayor Michael R. Bloomberg and his schools chancellor, Joel I. Klein, are trying to overhaul the public school system and impose a uniform reading and math curriculum at all but the highest performing schools.

City officials, who might otherwise have been jubilant about yesterday's results, offered a muted reaction, saying that the gains were not broad enough and that the school system as a whole was still failing at least half the city's children.

Klein's reaction in this NY Times article on the 2003 math scores was also less than ecstatic:

But not everyone greeted the news so enthusiastically.

The suggestion that city schools are on the upswing put Chancellor Joel I. Klein, who is overhauling them, in a tricky position. While the chancellor's critics pounced upon the higher scores as evidence that the school system did not need such an overhaul, some of his allies acknowledged that he would now be under even more pressure to show gains next spring.

Mr. Klein's reaction to the good news was muted, as it was to news of higher reading scores in the spring.

''While I am gratified by the test results released today for fourth and eighth graders in New York City, I must emphasize that it is hard to tell the true significance of any one set of results in isolation,'' the chancellor said in a statement. ''We must always look at results in comparison over a number of years. Only through comparison can we truly measure the progress we're making.''

Truth Squad! (Press officer Andy Jacob is eduwonkette's designated truth guru): Of course I've got all this wrong, so do enlighten us wayward philistines.

PS - Have you guys considered tee-shirts? I won't demand any royalties from the image above, which should obviously go on the front (with your last names emblazoned on the back). You can thank me later.

The Influence Spectrum: From Blogging to Academic Research

Fri, 11 Jul 2008 15:24:05 -0500

Let me use the occasion of Jay Greene's response to my earlier post to explain the differences between blogging and research, as I see it. Greene makes no distinction between the two activities, and is, as a result, skeptical about my anonymity. As he explained to me off-blog, "The same basic principles apply. They are both part of the spectrum of how people communicate ideas that may be related to policy decisions."

Blogs provide opinions, commentary, and analysis. Blogs are a place to discuss ideas, consider other points of view, and hear what a community of readers has to say. Blogging is great for testing out ideas, reflecting on the news of the day, and discussing and disseminating existing research. But bloggers don't do academic research. Academic research, in contrast, is subject to norms about method. The central norm in academic research is subjecting your work to the scrutiny of a critical community of scholars.

Undoubtedly, blogs, thinktank research, and academic research are "part of the spectrum of how people communicate ideas that may be related to policy decisions." But different levels of confidence should be assigned to different parts of that spectrum in educational policymaking. Below, from least to most credible:

1) Blogs: Blogging is free-form exchange, and the blogger is judged by the quality of his or her arguments and content by readers who seek out the blogger. Blogs are grassroots online communities where everyone, irrespective of their identity, is entitled to an opinion.

2) Thinktank research: Thinktank research is generally released without external review. The questions that are asked and the policy recommendations that are put forth are usually – but not always – tied to the stated objectives of the organization, which are sometimes ideological in nature. Thinktanks are well-funded and endowed with PR departments that publicize studies to policymakers and the media. As a result, thinktank research on education receives more attention than blogs and academic research in the media.

It is important to note that thinktanks do vary significantly in the extent to which they internally and externally review work before releasing it. They also vary in the extent to which they make their methods transparent enough that their analyses can be evaluated and replicated. Some thinktanks are more judicious than others about describing the implications of their work for policy and in spinning their findings. And some thinktanks don’t appear to sanction researchers when their studies are consistently discovered to be wrong. I imagine that other thinktanks would treat such a violation differently, because at the end of the day, these mistakes reflect poorly on the institution.

3) Academic research: Academic research is intended to contribute to a body of scholarly knowledge, and is subject to thorough peer review and to norms of scholarly inquiry. Though it is often policy relevant, the primary audience for this research is a community of scholars, who judge the research not for its policy contributions but its innovativeness, rigor, and contributions to a body of literature.

But peer reviewers are human, too, and they come with their own set of biases; the idea of a search for truth immune to ideology is a fantasy. Academic research that is imperfect does get published. And people do make mistakes in their papers, both innocent and intentional. That's why one of the norms of scholarly inquiry is to replicate studies and to take caution in declaring that the case is closed on any issue. This can be thorny, because academic research communities are small and dense. Everyone knows everyone else, and the scholars that take on prominent colleagues, even when they are clearly wrong, can pay a handsome price. People also have personal relationships with mentors and colleagues, and sometimes we don’t challenge each other as much as we should.

For all of these reasons, peer review is double-blind. In practice, papers are submitted to conferences before they are submitted to journals. On more than one occasion, I have reviewed papers of scholars who have sat on the same conference panels that I have. But the academics whose work is under review do not know the identity of their reviewers (except when reviewers cry foul that their work wasn’t cited, and suggest references that give away their identity!), and this provides a countervailing force against the social dynamics that sometimes cloud our judgment. And with academic research, no study is taken as a "killer study," and Jeff Henig has advocated for the same in the policymaking arena. Rather, individual studies are put in context of a larger evidence base.

To be sure, I, and some other bloggers, will occasionally present and analyze data in our postings, with the goal of persuading readers of a point of view. When I do so, I provide links to the data, which are generally in the public domain. When these data are not publicly available, I have always extended an offer to my readers to request data from me, which they have often done. When these posts involve more than making figures using publicly available numbers, I also provide detail about what I've done, which is simple descriptive analysis that a competent Excel user can replicate.

But there's no pretense that this is peer-reviewed academic work. And let's be realistic: an anonymous blogger isn't shaping public policy. In equating the two, Greene either overstates the influence of this blog on education policy, or diminishes the contributions of his own work. Of course, if my postings lead readers to think differently about research and policy matters, then those readers may have an influence. I see this as a very different dynamic than with thinktank research, where, because the objective is to influence public policy directly through research, the researchers have a greater obligation to their audience to vet what they've done before taking it public.

Finally, to Greene's point that my anonymity makes it impossible to "consider the source" - is it likely that Education Week would host an anonymous blog by someone working for or funded by "special interests" in education? Or that they would allow me to critique policymakers with whom I have some conflict of interest? The editors at Education Week know who I am, and decided to host this blog with full knowledge of my professional biography. I'm quite proud of - and grateful for - the community that we've built here, which has challenged and refined my own thinking on a wide variety of topics. At the end of the day, potential readers can decide for themselves whether this blog is worth reading, can tell me when they think I'm wrong (and you often do), and can expect me to listen, and even modify my positions in response.

Update: Be sure to check out Dean Millot's exceptional post related to this issue, The Letter From: "In short, I see no problem with research becoming public with little or no review” (I) , as well as Sherman Dorn's from earlier in the week, Can reporters raise their game in writing about education research? eduwonk also weighs in here: Politics of Information.

Cool People You Should Know: Stefanie DeLuca

Fri, 11 Jul 2008 08:19:55 -0500

Stefanie DeLuca is a sociologist who teaches at Johns Hopkins. Self-described as "Ann Coulter's anti-matter, but not as tall," DeLuca has recently been named a W.T. Grant Foundation Scholar - a prestigious five-year award - to study residential mobility in the lives of poor adolescents. Deluca is a rare find in educational research as she is equally skilled in quantitative and qualitative methods, and has used both approaches to study the effects of residential mobility on poor children and their families.

DeLuca's work on the Moving to Opportunity (MTO) program, which attempted to relocate poor families from high-poverty neighborhoods by providing housing vouchers, is a good example of her ability to get her head around tough problems in novel ways. Though everyone expected that moving to a better neighborhood would have a positive impact on students' academic achievement, the MTO project found no effects. DeLuca, who interviewed program participants in Baltimore, has attempted to explain why in this article in Education Next. You can read an interesting bio-profile about her experiences conducting this research here. Her conclusions are worth quoting at length:

The interviews conducted in Baltimore shed light on the explanations for why the MTO experiment didn’t lead to better schools and educational achievement. Many MTO parents told us about frightening conditions in their children’s schools and their concern for their children’s well-being. Yet these fears and realities did not always translate into efforts to remove their children from these environments. Poor mothers and their children juggle myriad extreme conditions, and schooling is not always on the top of the list. Murder, crippling drug addiction, suspicious landlords, diabetes, and depression took center stage in the lives of many, if not most, MTO families we interviewed. While neighborhood change could be a necessary condition to protect children and improve their schooling, it is not sufficient in light of the deep morass of issues that characterize the lives of the urban poor.

Many social policies assume that all low-income parents approach opportunity the same way that most middle-class families do, and that the main problem is a lack of financial resources. Our interviews provide a reminder that poor families are not just wealthy families without a bankbook. Poor parents often have less information about school choice programs and school quality than do middle-class parents. Poor families may approach opportunities, and in particular may secure schooling for their children, in ways that diverge from many research models of educational decision making.

These insights are also relevant to school choice policy in general. Many cities, including those in which MTO families were living, have expanded school choice programs. No Child Left Behind gives parents the option of sending their child to another school if the current one doesn’t make adequate progress. The success of these policies in enhancing education opportunities for the types of families who participated in the MTO experiment will depend on gaining a better understanding of how these families view the school choice process and where it fits into their overall strategies for well-being.

DeLuca's other projects will also be of interest to many eduwonkette readers. She has examined the effects of delaying entry into postsecondary education after high school, finding that delaying college is associated with lower odds of getting a BA. Another strain of her work looks at the effects of non-cognitive skills - for example, student effort, engagement, and motivation - on students' educational attainment (i.e. how long they stay in school and what degrees they earn). Very much looking forward to the papers coming out of that project.

If all of that wasn't enough, here's one more tidbit: It's rumored that DeLuca has impeccable taste in shoes. Can't vouch myself, but it came from a good source. A girl after my own heart indeed.

Gimme Some Truth

Thu, 10 Jul 2008 07:25:47 -0500

If you think "Education Department Employs Squadron in Search for Truth" is a spoof article from The Onion, guess again. You'd think that PR flaks would know better than to name an otherwise mundane 21st century version of letters to the editor the "Truth Squad," and in doing so, make it worth reporting on. Said New York City Deputy Chancellor Chris Cerf, who came up with the Truth Squad concept:

"We try to keep track of what people are saying about us, and we respond periodically. Because we believe in the truth."

Is that truth with a big T, truth with a little t, or truth with an asterisk?

I'll let you be the judge.

Here's what John Lennon would have said:

I'm sick and tired of hearing things
From uptight, short-sighted, narrow-minded hypocritics
All I want is the truth
Just gimme some truth
Ive had enough of reading things
By neurotic, psychotic, pig-headed politicians
All I want is the truth
Just gimme some truth

Update: Check out Alexander Russo and Norm Scott's pithy posts on the same.

New Blogs To Check Out

Wed, 09 Jul 2008 23:36:03 -0500

Here are four blogs - 3 of them are relatively new - that you should definitely check out:

* The Lights of El Milagro: Stories from One of America's Most Innovative Charter Schools: This is the first blog written by a charter school founder and current principal that's come across my screen. Kevin Riley, who chronicles the ins and outs of Mueller Charter School in California, brings a unique set of experiences to the blogosphere, and has a great post up about AYP and baseball box scores. Of the wait to find out if his school cleared AYP, he writes, "we know for certain that a single box score out of context cannot predict whether the Yankees or the Red Sox will win the American League East. Just as we know that the complex drama of teaching and learning and human relationships and keeping children whole will not show up in the box score."

* It's Not All Flowers and Sausages: Written by Mimi, a pseudonymous teacher (and new commenter at this site!), this is a hilarious blog about day-to-day life in schools. I especially like this post, Sippin' on Gin and Juice, about a parent who gave her a bottle of rum as a thank you gift. I won't spoil the ending.

* The London Times School Gate: If you don't pay attention to education policy in England, you're missing out. Since the 1990s, American education policy has borrowed many lessons from UK education policy, just as John Chubb and Terry Moe suggested we should in their 1992 book, A Lesson in School Reform from Great Britain. The New York City Quality Reviews mirror that of the English school inspection system in many ways. (See skoolboy on Ofsted here.) This blog should help us all stay on top of developments across the pond.

* Schoolracetalk.org: In case you missed the announcement embedded in Mica Pollock's guest post, she has just kicked off a new blog. As I have written before, Pollock is one of the most exciting and thoughtful scholarly voices on how educators handle day-to-day issues of race in schools and classrooms.

The Rhetoric of Reform: Does Research Count?

Wed, 09 Jul 2008 07:30:00 -0500

“Better schools. Higher scores. And satisfied parents. That's the record of the D.C. Opportunity Scholarship Program.”

Thus begins Secretary of Education Margaret Spellings’ column in yesterday’s Washington Post. In this piece, she seeks to rally public support to renew the DC Opportunity Scholarship Program (OSP), which provides scholarships up to $7,500 to use towards the costs of a participating private school, including tuition, fees, and transportation. The authorizing legislation stipulated that priority for scholarships was to be given first to students attending schools that were judged in need of improvement (SINI) under NCLB standards.

Last month, the Institute of Education Sciences, the research arm of the U.S. Department of Education which Spellings heads, released the results of the Congressionally-mandated evaluation of the OSP, which reports impacts after two years. As the first federally-funded private school voucher program in the U.S., the OSP is a political football, and this evaluation report and its predecessors have been pored over by policy wonks across the land. The statute that authorized the OSP mandated that it be evaluated in terms of its impact on student test scores and school safety, as well as a more ambiguous criterion of “success,” which was operationalized in the study as parents’ and students’ satisfaction with their schools. The evaluation used a randomized controlled trial (RCT) to assess the impact of the OSP.

The executive summary of the report tells the tale, in unambiguous terms. (a) After two years, there was no effect of the OSP on reading or math test scores either for students who were offered a scholarship or those who actually used a scholarship. (b) If we look at 10 different subgroups of students—girls or boys, students attending SINI or non-SINI schools at the time of application, elementary or high school students, those from application cohort 1 or cohort 2, or students performing relatively higher or lower at the start of the study—there were no statistically significant effects of participating in the OSP on math for any subgroup, and for reading, three subgroups (students attending non-SINI schools at the time of application, relatively high-performing students, and students from cohort 1) might have done better than their nonparticipating peers. But even here, the evaluators caution that the statistical significance of these effects did not hold up when conventional adjustments for multiple comparisons were made. In other words, these subgroup effects might be due to chance, given how many comparisons were being made at the same time. Notably, the subgroup specifically identified in the legislation—students who had attended a SINI public school under NCLB—did not do better either in reading or math.

skoolboy isn’t crazy about using public funds to support private schools, but he’s a big supporter of using public funds to support the education of children in D.C., who historically have been among the lowest performers in the nation. Congress authorized this program, it’s survived legal scrutiny, and it’s deserving of a fair shake. But distorting the results of an evaluation doesn’t serve the public good. If Ms. Spellings wants to argue that the program should be renewed by Congress because parents are more satisfied with their child’s school, or because they are less likely to report serious concerns about school danger, she’s welcome to make that argument. Those are good outcomes, and some might argue that they’re ample justification for renewing the program. (Others might point out that students who received scholarships did not report higher levels of satisfaction with their school, or better school safety.) Or, alternatively, one could argue that the program needs more time to mature in order to be successful. But let’s not kid ourselves, Madame Secretary: the evidence on the academic success of the D.C. Opportunity Scholarship program—measured on your preferred metric, scores on standardized reading and math tests—is far too weak to make a persuasive case. Misrepresenting the evidence does honor neither to education research nor to education policy.

Guest Blogger Mica Pollock on: Everyday Antiracism: Getting Real About Race in School

Tue, 08 Jul 2008 20:02:12 -0500

Mica Pollock is an anthropologist who teaches at Harvard's Graduate School of Education. She has two new books coming out this summer: Everyday Antiracism: Getting Real about Race in School (on which she has written the FAQ below) and Because of Race: How Americans Debate Harm and Opportunity in Our Schools. Her first book, Colormute: Race Talk Dilemmas in an American School, won AERA's 2005 book award. And she has just launched a new blog, schoolracetalk.org. Head on over to her site for what promises to be a provocative discussion.

1) What is "anti-racism?"

By “everyday antiracism,” we mean acts educators can take daily in schools and classrooms to counteract racial inequality of opportunity and outcome, and to counteract racist ideas about “types of people.”

I should note that by “racism,” we don’t mean the willful harming of people of color by white people. (This is how the law has often framed it.) Rather, the authors collectively define racism as any act or situation that, even unwittingly:

- tolerates, accepts, or reinforces racially unequal opportunities for children to learn and thrive;

- allows racial inequalities in opportunity as if they are normal and acceptable;

- or treats people of color as less worthy or less complex than “white” people.

2) Do you think that history, custom, teachers, or students themselves most often propagate racism?

All of the above. Still, this book focuses on acts by educators. They have great power to “deal” with race issues in schools, for good or ill. Students also react to educators’ everyday acts. This is also why educators are so powerful! In my introduction to Everyday Antiracism, I write that:

In schools, people interact across racial lines, distribute opportunities moment to moment, react to “outside” opportunity structures, and shape how future generations think about difference and equality. Interactions in educational settings help build or dismantle racial “achievement gaps.” To a student, one action can change everything. Everyday acts explored in this book include how we talk with our students and discipline them; the activities we set up for them to do; the ways we frame and discuss communities in our curriculum; and the ways we assign students to groups, grade their papers, interact with their parents, and envision their futures.

Everyday Antiracism shows that educators take many acts in educational settings that harm children of color, or privilege and value some children over others in racial terms, without educators meaning to at all. Further, many racist ideas about “types of people” are programmed into our heads as educators, despite our intentions. So, we want educators asking: which everyday acts by me counteract a racially unequal society, and racist ideas about “types of people”?

3) Some authors in your book deny the validity of racial categories, while others claim that to deny the existence of racial inequality is foolish. Explain.

Racial categories are social realities built on biological fictions. As Alan Goodman discusses in his essay in Everyday Antiracism, 20th and 21st century genetics show that there are no biologically meaningful “racial” subdivisions to the human race. How could race categories like “white,” “black,” “Asian,” or “Latino” be genetically valid if someone labeled “white” in Brazil can be labeled “black” or “Latino” here?

Race categories are things people made up. Over six centuries of life in the Americas, people used law, “science,” and everyday activity to distribute opportunities along the lines of physical traits that were simply too small a portion of our genetic makeup to be valid ways of categorizing human beings (skin color, nose shape, and hair texture, for example). Still, we have made these categories socially real in the past nearly six centuries of American life. So, racial categories are false biologically, but real socially.

This is why the “antiracist” educator must negotiate between two antiracist impulses in deciding her everyday behaviors toward students. She must choose between the antiracist impulse to treat all people as human beings rather than racial group members, and the antiracist impulse to recognize people’s real experiences as racial group members in order to counteract racial inequality.

4) Do you think the promotion of anti-racism in schools will lead to the continuation of anti-racism post-graduation and in the workplace?

If our children are educated in settings where children of all “groups” are treated as equally smart and valuable, they will learn to see one another more that way, too. What children learn in school is typically the opposite. One author in the book, Karolyn Tyson, has studied almost-all-black schools in North Carolina where the “gifted” class is completely white. The very existence of that “gifted” classroom teaches students a lie: it teaches them that some “race groups” are more “gifted” than others. Another author in the book, Beth Rubin, discusses how racially patterned tracking “teaches” students the same false lesson: that some “race groups” are smarter than others. How could these false ideas not continue after graduation? Conversely, if students are schooled in environments where educators actively treat students from all “groups” as smart and “gifted,” how could they not learn to see one another more that way, too? And how could that not continue after graduation?

The Trouble with the Education Policy Advocacy Industry: "Building on the Basics"

Tue, 08 Jul 2008 00:51:07 -0500

Today, Marcus Winters, Jay Greene, and Julie Trivitt are releasing a study called, "Building on the Basics: The Impact of High-Stakes Testing on Student Proficiency in Low-Stakes Subjects."

It may be an elegantly executed study, or it may be a terrible study. The trouble is that based on the embargoed version released to the press, on which many a news article will appear today, it's impossible to tell. There is a technical appendix, but that wasn't provided up front to the press with the glossy embargoed study. Though the embargo has been lifted now and the report is publicly available, the technical appendix is not.

By the time the study's main findings already have been widely disseminated, some sucker with expertise in regression discontinuity may find a mistake while combing through that appendix, one that could alter the results of the study. But the news cycle will have moved on by then. Good luck interesting a reporter in that story. And even when researchers working in the policy advocacy industry make sloppy, indefensible errors - for example, when Greene and Winters used data that the Bureau of Labor Statistics warned against using to show that teachers are overpaid - they're not approached with caution by the press when the next report rolls around.

So as much as I like to kvetch about peer review and the pain and suffering it inflicts, it makes educational research better. It catches many problems and errors before studies go prime time, even if it doesn't always work perfectly.

As for the Winters, Greene, and Truitt study, the jury is still out - as it should be until we have more information. I'll get back to you once I've read the technical appendix.

Update:

Let me first apologize to Jay Greene and my readers for shooting off a short post before teasing out all of the complexities around thinktanks, research, and the reporting of research in the popular media. I used Greene's paper as a vehicle for doing so, and that may have made it appear that I was criticizing the quality of that study when I was not in a position to do so. I shouldn’t have raised questions, even hypothetical ones, about the methods in that paper until the technical report was available for review, and you should definitely read Greene's response here.

This issue, however, is much larger than this particular Manhattan Institute report, and I want to use Greene's critique - that I have posted on working papers from the National Bureau of Economic Research (NBER) - to point out some important differences between papers issued by outlets like NBER and thinktank reports:

1) With NBER papers, everything is on the table upfront. They are scholarly papers that include extensive methods sections and robustness checks in every paper. Greene writes that, “If [reporters] requested the technical report, they could get that.” But the press release makes no mention of a technical report at all. The key difference is that there’s an extra step in the process to get to the detailed methods, which reporters writing articles could ostensibly circulate to other scholars for comment before writing an article.

2) There is no PR machine behind NBER papers. It’s one thing for me to write about a study on my blog. It’s entirely another to send press releases to reporters at newspapers and other media outlets, who in turn – and this is their fault, not Greene’s – cover his report like it’s a final product. The more complex the methods are, the more there is a need for peer review because it becomes more difficult to eyeball the problems from the sidelines – and Greene and his colleagues are using sophisticated methods in this report.

3) NBER papers generally aren’t trying to persuade anybody in particular of anything. They are not intended to sway public policy. Contrast this with the press release approach of policy advocacy thinktanks. For example, the press release for this study said, "In this report, Winters, Greene, and Trivitt dispel the myth that high stakes testing in reading and math will harm student proficiency in low-stakes subjects. The data from Florida provides further evidence for policy makers considering the renewal of No Child Left Behind, showing that national testing incentives improve overall educational achievement levels.”

4) NBER has implicit quality controls. It is a community of scholars to which one must be invited, one that has strong norms about how research is conducted and reported. The quality of the average NBER working paper is extremely high. There is much less variation in the quality of NBER papers than there is in thinktank reports. On some level, this is an issue of the trustworthiness of institutions; for example, I trust a report coming out of RAND or Mathematica more than I do one coming out of the Heritage Foundation, because neither RAND nor Mathematica have a stated ideological agenda.

For the best treatment of the thinktank issue I’ve ever seen, see this post by Dean Millot, and his preceding posts here and here.

How to Subscribe to eduwonkette

Mon, 07 Jul 2008 18:24:05 -0500

Many of you have pointed out that it's tough to find the RSS and email subscription buttons, so here are direct links:

1) To subscribe to the RSS feed in a reader, click here.

2) To receive blog posts via email each night, click here.

And thanks to the readers who let me know that the Bloglines feed has been on the fritz. The easiest way to solve this problem is to unsubscribe and resubscribe.

Gender and Stereotype Threat in Math and Science

Mon, 07 Jul 2008 12:30:33 -0500

Can asking women to simply bubble in their gender before a test hurt their performance on math tests? Conversely, does mentioning that a math test is gender-neutral boost women’s achievement? More than a decade of research on “stereotype threat” suggests that the answer to these questions is yes.

When stereotypes – for example, the “math is hard for girls” Barbie myth – are not activated or are actively nullified before math tests, women’s performance improves. Given the ongoing concern about women’s under-representation in the upper echelons of math and science fields, researchers have turned to these social-psychological mechanisms for answers.

A recent paper on stereotype threat by a team of psychologists brings evidence from real college classrooms to bear on this issue. Despite a large body of laboratory evidence on the effects of stereotype threat on women, others have argued that these results do not apply to “real world” settings. In a field experiment at a large public university, psychologists Catherine Good, Joshua Aronson, and Jayne Ann Harder administered an extra credit, pre-final practice exam to students enrolled in the terminal course of the most rigorous and fast-paced calculus sequence offered by the university, a course that satisfied degree requirements for math, science, and engineering degrees. These men and women were, by all accounts, in the pipeline for math and science careers. Students in the “gender nullifying” treatment read just a few extra sentences before taking their tests:

What about gender differences? This mathematics test has not shown any gender differences in performance or mathematics ability. The test has been piloted in many mathematics courses across the nation to determine how reliable and valid the test is for measuring mathematics ability. Analysis of thousands of students' test results has shown that males and females perform equally well on this test. In other words, this mathematics test shows no gender differences.

In the control group, the test was administered under normal conditions, and women and men performed equally. But women who received the “gender nullifying” treatment (reading the statement above) outperformed men. The authors concluded that “even among the most highly qualified and persistent women in college mathematics, stereotype threat suppresses test performance.”

The question for educators is what we should do with this information. Should teachers provide prompts demonstrated to improve girls’ math performance in the classroom? If so, at what level of the education system is this appropriate? Elementary school, graduate school, or all the way through? By the same token, could the widespread idea that men are less verbal, and thus worse at writing and reading, play a role in their performance in those subjects? If you're interested in learning more, this website is a great resource and also provides some ideas for reducing stereotype threat.

You can find more detail on the study, “Problems in the Pipeline: Stereotype Threat and Women’s Achievement in High-Level Math Courses,” published in the Journal of Applied Developmental Psychology, here. And as always - or at least until I get booked for copyright infringement – you can email me if you can’t access the paper but would like a copy.

Image credit: Fatcatenator.