Sherman Dorn

Grading reports that grade states, which have schools that grade

It's now a PR cliche in education wonkery: grade states. Issue grades, and that's a hook for reporters to write stories about the reports, because the reporters at daily metros can say, "[Your state's name here] receives 'F' in think tank report on education." But beyond the PR value of grades, it's facile, which is why I'm surprised Education Sector gave into this particular venal sin in its report on states' higher-ed accountability policies. C'mon folks: can't you figure out a more substantive way of evaluating states? At the very least, this is so 1990s.

So I'm thinking about developing a report over the next year that grades think-tank reports that issue grades for states on some matter of education, where of course schools have teachers who grade students. Among the standards will be the following:

Clear standards for grades: a year before the report is issued, does the entity that issues the report publish grading standards or criteria?

A - Entity publishes grading standards with sufficient criterion specificity that an outside observer would not be surprised at the grade a state receives the next year. (Note: this is a low bar, not requiring agreement with grades.)

B - Entity publishes standards, but standards are too vague to provide benchmarks for policy progress.

C - Entity has previously published reports issuing grades to states, but changed the standards, or described the project and the areas where states would be grade, but no standards for those areas.

D - Entity has previously published the existence of the report project, but there is no previous publication of intent to grade states in this area of policy.

F - Report appears out of the blue with no publication of intent in this area.

Okay, folks: where does today's Education Sector report fit? How about Ed Week's annual Quality Counts phonebook? Fordham's reports that issue grades?

And, yes, if I'm serious about this, that implies I have to develop some more grading criteria. After all, it would be most interesting and ironic if I created a report that contained the mechanism by which the report itself could be torn apart. Hint, hint, ...

Find the typo! and other national-stage blogging

The National Journal unveiled its new education policy blog yesterday. My first response has an embarrassing writing goof; see if you can spot it!

No differences -> politics as usual?

While the DC vouchers debate swallowed more airtime, it's David Figlio's new study of Florida's voucher programs that will reveal the state of voucher politics. Several years ago, opponents of vouchers pointed out the lack of accountability for the programs, and in response supporters inserted a mandated study comparing achievement of students using vouchers to public-school students. Fortunately, they picked one of the best economists of education, who is careful and cautious and has done several studies of Florida's voucher programs in the past decade (including the best article on the topic, published in 2006).

Figlio's conclusion is roughly that given the data he had available, there is no evidence of differences in student achievement between those in the corporate tax-credit voucher programs and similar students in public schools. Further, the usually-cautious Figlio went out on a limb and said if additional data were available, he wouldn't expect the conclusions to change. This is not the only report I expect Figlio to produce on the corporate tax-credit voucher program, since the interesting questions for microeconomists are about how the shape of the market (the presence and size of a voucher program) changes its characteristics (esp. responses of public schools).

But until that report is produced, and probably after it, the no-difference finding here mirrors a bunch of other studies. At this point, it looks like there is no solid evidence that students using vouchers perform better as a result, and in Florida at least, it also looks like students don't perform worse, either. So the voucher debate will not be settled by evidence of effectiveness, and we default back to questions of values embedded in public policy and the way that experiences shape the policy-relevant questions.

Those who support vouchers are spinning the no-difference findings as "vouchers do the job for less money, and choice is a positive value." Those who oppose vouchers are spinning the findings as "vouchers are no panacea, and choice can exist within the public system." And as voucher-receiving schools accumulate in the state, the ordinary politics of constituents make it hard for legislators to oppose eliminating the program. It is the last item that makes Florida (where a number of Democrats have voucher schools in their districts) different from DC (where the governing authority, Congress, has only one voting representative with constituents who use the vouchers). In the end, I think we'll see voucher programs generally stay in the states where they currently exist (primarily from the constituency-experience dynamic) but not expand much (because of the lack of evidence of great effects and because charter-school expansion in cities is an easier political sell).

Prevent backtalk: turn on the television!

I knew it years ago, and in two studies released earlier this month and this week, I think both in peer reviewed journals, we have it confirmed: the best way to prevent teenagers from talking back to you is to turn on the television years earlier so that they don't develop the ability to talk back. So that spring day in 1996 when my wife and I decided to sell our television before moving to Tampa? A big mistake.

And there we were, deciding that we were advancing our children's interests. No, that wasn't it at all: they were 4 and 1 at the time, and we decided that since we didn't like their arguments over the television, we'd see how long we could go without one in the house.Answer: 13 years and counting. And no matter what arguments we have in our household, it's not about the channel the television's tuned to. Instead, it's about who gets the computer...

Serious side: The article released this week is more about the relationship between adult caregiver and child than about television, and it highlights the importance of one-to-one interactions at early ages. I suspect this will be followed by other analyses from the same data set.

The purpose of seminars/discussion

I'm at THATcamp this weekend and having a great deal of fun. (Check the Twitter archive for tweets with the #thatcamp tag...) But there is a lot of serious stuff here, and I was hoping that it would confirm or undermine the way I'm currently thinking about the problems of teaching online. The demography of the group doesn't quite give me enough of that reality check, since I'm in the minority as an experienced teacher; the majority of attendees are graduate students, staff members at one of the digital humanities centers in the country, or library/museum staff, but it still was a first shot at this.

No disconfirmation in the relevant session, but it's honed the way I'm thinking about the purposes of a seminar or discussion. What many great humanities discussions share is the entree into and development of skills in a specific discourse and in "academicizing" more generally (to borrow a term from Stanley Fish). In memorable humanities discussions, teachers model analysis and establish an environment within which students can learn and practice close reading, the identification of key issues in a disciplinary or interdisciplinary context, the articulation of critical perspectives, and engagement in a dicursive community.

Several characteristics of face-to-face classes contribute to that: the ability of a teacher to take any issue and analyze it extemporaneously, the ability to annotate material for everyone present (if verbally), the probing of assertions with either questions or counterarguments, and the capacity to revise arguments on the spot.

There are online tools for some of this, if without the immediacy. Diigo is a great social annotation tool; while it's not the type of immediacy that happens in close readings in class, I have some anecdotal evidence that it can be powerful for students. Teachers could take issues that pop up in discussion boards and expand upon them by modeling analysis and should probably be careful to construct prompts that set the stage for that. And I've been thinking about requiring weekly recorded fishbowl sessions with small numbers of students in my fall online class, as a way to generate some immediacy in the engagement.

In other words, no great insights, but the honing itself is important. And it required a bunch of people who are very comfortable online getting together face-to-face to bat around some ideas. There was an ironic moment in the session related to that fact: One staff member from the Center for History and New Media left the room just before the session to address some technical issues. I started moderating, and we generated a list of functions for seminars and discussions in general. She returned to the room, and as she started to talk a few minutes later, she said, "I'm sorry if this was mentioned before... I wasn't here at the beginning of the session."

How to steer CYA-oriented bureaucracies, or why NCLB supporters need to think about libel law

Someone at USDOE sent me an invitation to listen to the June 14 phone conference where Arne Duncan explained how disappointed he was in Tennessee, Indiana, and other states with charter caps, let alone states such as Maine with no charter law, and how that disappointment might be reflected in the distribution (or lack of distribution) of "Race to the Top" funds (applications available in October, due in December, with the first round of funding out in February 2010). There are a few details that reporters didn't ask about (Duncan's somewhat surprising statement that a good state charter law would set some barriers for entry rather than establish a "Wild West of charter schools," and the way that small charter schools and charter schools with grade configurations outside state testing programs can stay off the radar for accountability purposes), but I was not surprised that two Tennessee reporters were called on for questions.

But apart from the selection of reporters for questions, the phone presser and other DOE moves made me think about the various uses of power in education-policy federalism. In limited ways, explicit mandates can be effective, if there is a sustained willingness within the USDOE (and esp. OCR) to make painful examples of the nastier school systems that try to evade those mandates. Offering technical assistance is another method, and despite the massive conflict-of-interest problems in Reading First, I agree with one of the researchers in the field who thinks that Reading First did improve primary-grade reading instruction, on balance. (Thumbnail version: hourslong scripts, ugh; explicit instruction in phonemic awareness and some other fluency components, obviously necessary.)

But neither heavyhanded mandates nor technical assistance can do everything, and neither works with the greatest motivation for both defensive and hubris-oriented bureaucracies: risk management. If you are a public school teacher or administrator, my guess is that you can identify some fairly silly action by your district that was motivated almost entirely by CYA motives, and if you can marry those CYA activities to pedagogy, you've been lucky or have a black belt in administrative maneuvering. (If you have such victories, please describe them in comments! Otherwise, we'll all wallow in the shared misery of observing defensive administering and the all-too-frequent ensuing train wreck.)

I think the federal government can shape bureaucratic behavior to the good by using that risk management and structuring accountability policies around that. And here's the lesson I take from my high-school journalism class in ninth grade 30 years ago: libel law in the U.S. generally recognizes the truth as a positive defense agaist libel allegations. That seems like a backwards way to frame the legal issue -- after all, isn't it common sense that a publication is libelous only if it's false? -- but the notion of a legal positive defense gives an individual or organization a way to organize behavior in a way that is both professionally appropriate and also make a legal defense aligned with professional expectations. Because the truth is a positive defense against libel claims, even an idiotic general counsel for a newspaper or publisher looks to the professionally-appropriate standard: is there documentation that the published work is true?

Sometimes a positive defense is not explicitly part of jurisprudence but evolves as a practical guidance for clinical legal work and internal advice for school systems. Observing procedural and professional niceties create exactly that type of positive defense in special education law. There is nothing in federal special education law to carve out an explicit positive defense for school system behavior, but many articles written by Mitchell Yell over the past few decades constitute a convincing case that school systems now have a de facto positive defense: professional documentation of decisionmaing and scrupulous adherence to procedural requirements are a positive defense against a broad range of allegations by parents of and advocates for students with disabilities.

Yell has argued (persuasively) that due-process hearing officers and judges use procedural adherence and professional documentation as a filter in special education cases. If a school district can document that it has paid attention to procedural mandates and has met professional standards for documenting decision-making, then hearing officers and judges are extremely reluctant to look at the substantive merits of those decisions. But if a school district has ignored standard procedural expectations that most districts meet, or if a school district has kept no or inadequate documentation of its decision-making rationale, then all bets are off and a hearing officer or judge will be much less likely to defer to the school district on professional judgments.

In essence, Yell implies, school districts can avoid adverse judgments if they pay attention to timelines and other procedural niceties and if they keep teachers and principals on their toes about current "best practices" as well as deadlines, notices, etc. Not all districts are aware of this positive defense, or I suspect that some enterprising special education researchers could make a mint running seminars, "How never to get sued again."

More broadly, I'm beginning to think that the construction of a positive defense against charges of incompetence would be healthy for school systems and state policies. The devil would definitely be in the details, but instead of being frustrated by a consistently observed school system behavior, maybe we should take advantage of that consistency.

The right kind of infection

The Powell et al. article on cultural complexity 90,00 years ago, published in the June 5 issue of Science, has some interesting consequences for education policy, though it's an archaeology article. The argument the authors make is that one needs a certain population density before one can find surviving signs of cultural complexity (archaeological evidence of more sophisticated used of symbolism and technology). Sub-Saharan Africa had both those population densities and archaeological evidence from 90,000 years ago, as did Eurasia 45,000 years ago.

Powell et al. are arguing that the development of the earliest human cultural skills may have depended on nothing other than density. This is an appealing story: get enough humans living in proximity, and whatever culture is developed will be maintained while the various subpopulations (clans, etc.) interact and teach each other, keeping the ideas floating around the population in a way that would not happen in a sparse population with little interaction between subgroups.

I suppose that as someone without an archaeology background, I have no insider knowledge of the contribution this paper makes to studies of human evolution. The authors are portraying the issue as an explanation of how human culture could appear suddenly (on the eon-scale) without resorting to changes in biology (esp. cognitive capacity). We'll see what other researchers of human evolution say about that, but there's something important there for education.

The article suggests that one can categorize various cultural characteristics by the extent of continuity across time. Isolated behaviors and skills may not survive unless they spread beyond the individuals who may exhibit/learn them for a time. With enough contact among people, knowledge, skills, and behaviors can become continuous; that continuity is the subject of the article. But one can look at knowledge, skills, and behaviors that are more than continuously existing. Are they common (maybe the experience of a minority but where everyone knows several people who have that experience)? Are they the normative expectation? Are they ubiqitous (universal or nearly so)? That's a five-category, ordinal variable for the extent of cultural behavior in a population: isolated, continuous, common, normative, ubiquitous.Okay, there's a sixth category, absent.

Many of the debates over education policy are about shoving the national population from a common experience of X to a normative expectation of X, or from a normative expectation to the ubiquity of X. In the space of 70 years, high school graduation moved from a continuous population behavior to a normative experience; that's the story in my first book. But the rhetoric surrounding a national population's experiences often obscures variations. As Claudia Goldin has pointed out, high school graduation became normative in the midwest and northeast by 1940, while it moved much more slowly in the South (for Southerners of all races/ethnic backgrounds). And today, while approximately a quarter of teenagers leave high school without a standard academic diploma, there are many high schools where graduation is common but not the majority experience, and probably a few high schools where graduation exists every year but is not common.While the latter should be alarming to anyone, in reality the majority of high schools in deep crisis fall in the former category, schools where graduation is common but not the majority experience.

There is an argument that the Powell et al. article suggests: if culture "spreads" once there is a sufficient number of "carriers," maybe we should look at education as akin to a disease process that we want to propagate. This is close to the contamination theory Geoffrey Canada has (or had when Paul Tough followed him around while writing his book about the Harlem Children's Zone). There are both ways in which that argument is interesting (esp. in communities where half or more teenagers drop out without a high school diploma) and others in which it is disturbing (assuming that students can be "carriers" of culture in way that adults can manipulate, though they can't shape adolescent experiences directly.. uh, no).

How do you move a behavior from a common-but-minority experience to a normative expectation? That's essentially the question we have in a large number of high schools in the country and with regard to baccalaureate degrees for the entire country. At least in my understanding, there are two requirements, involving both the spread of an idea and set of habits (habitus, in Bourdieau's language of cultural capital) and also institutional infrastructure. Attending high school became the normative experience for teenagers when they could no longer enter the full-time labor market with ease, when people began to think of high school as an experience that could be useful, and when there were enough high schools for majority attendance to be physically possible.

I do not think that there are exact parallels for all circumstances, just a combination of population behavior and institutional behavior. They go together. And, yes, there are cases where the extent of cultural experiences can reverse: working-class attendance at Shakespeare in the late 19th century, if you believe Lawrence Levine, or girls' primary education in Afghanistan from 1995 to 2001.

Nothing fuzzy about fuzzy math

George W. Bush is probably responsible for people calling constructivist math advocacy "fuzzy math". There is a field called fuzzy logic, and while I know very little about it, I'm irritated that the former president's maladroit use of English is messing up technical terminology. Fuzzy logic is a useful tool in engineering, and while Lotfi Zadeh's original term may not be perfectly descriptive, math teachers should be the last to misuse a term that's in their own discipline.

See-no-knowledge in education policy?

I seem to be reading several "we don't know anything so let's plow ahead" arguments in education think-tankery, from Mike Petrilli's argument that because we don't currently have a solid research base about how to turn schools around, we shouldn't try, to Kevin Carey's consistent argument in Education Sector's blog that because there is no research consensus about predictors of good teaching (and considerable research suggesting that there is not a link between effectiveness and countable items like years of experience beyond the first few or graduate degrees), it makes better sense to let people into teaching and then evaluate their effectiveness.

Fortunately, that's not the approach of the Institute of Education Sciences under John Easton, which has just announced a large research initiative on turning around schools. I suspect that both Petrilli and Carey would acknowledge that research in difficult topics is a good thing and argue that IES initiatives are different from policy, because sometimes you have to make decisions based on the state of knowledge you have, not the ... oh, shoot, there's Donald Rumsfeld phrasing again. But you probably know what I mean: Petrilli and Carey's stances are policy stances based on topic-specific agnosticism, not opposition to research.

But there's a serious question buried here: on big questions of policy, where you have to make choices, and the research is nondirective, how do you make decisions? I think the answer has to be incrementally, to allow research to catch up and influence policy later. If you make a huge political and institutional commitment to a policy path that has no research support and no ethical/legal obligation, then you're committing millions of children and hundreds of thousands of educators to a path that is very hard to change later.

For that reason, while I think Arne Duncan's four-choice speech earlier this week is not based on research, and Petrilli is correct that there is no particular reason to believe that charter schools will somehow rescue the education of students otherwise stuck in horrible circumstances, the policy itself is good largely because it doesn't make hard and fast commitments to a particular path. The good thing about a charter is that it can be revoked, and in states such as Florida where there is a single authorizer for a geographic area (here, the county school boards), authorizers can be reasonably aggressive in shutting down shady or incompetent operations. So I share Petrilli's skepticism, but precisely because I am skeptical of any particular approach to schools in crisis, and because Duncan is being wishy-washy, I will applaud the Secretary for being wishy-washy.

Update: I first used the term "know-nothingism" in the title. Ugh. Bad move for an historian. Petrilli and Carey are not members of the 19th century anti-immigrant party. Mea culpa.

One more reason not to use Elsevier

If fake journals created at the whim of pharmaceutical companies weren't bad enough, how about paying people to assign five stars to textbooks in Amazon's review system?

Being paid to sit in a/c

As Notorious Ph.D. wrote at the end of last week, most nine-month faculty do not get a vacation in summer but are just unemployed and often still have to work. Some such folks have plenty of resources to tide them over until the fall, others have a salary that is stretched out over twelve months, and yet others are paid fairly pitifully. I'm in none of those categories, having a salary that's well below the average for rank and discipline but higher than the median U.S. salary and considerably higher than the average wages through human history. I also have a paid teaching gig this summer, one course, and for most of yesterday from about 8 am until 9 pm, I was reading papers or engaged in various class logistics. Last week, any time not in class was spent on union work or a teaching workshop for high-school history teachers on the Spanish Civil War and American involvement in it. The latter is a far cry from a Teaching American History project, but it gives me a taste of what the best of workshops can be like.

This morning is one of those days when I had a substantial incentive to get to campus early: when I woke up, the temperatures were already in the mid-80s (F.). Right now, the weather station at Tampa International is recording 85 F. with 82% relative humidity. I keep telling myself that at least the sauna is free, and the driver of the car parked next to mine this morning added, "In the north, you have to shovel stuff." For the record, 82% humidity in mid-80s temperatures is darned close to shovelworthy, but not yet.

So I'm in my office, and with luck I'll be able to grade some straggler student papers before class. Because of last week's workshop and a whole set of other things, I'm behinder than usual on other matters. And if you think the third-to-last word in the previous sentence is not in fact a word, you may not have been reading a slew of student papers recently, and you might be one of those language mavens who would like to bury the student body, preferably next to Jimmy Hoffa (apologies to Strunk and White).

Still mis-remembering Title IX

Two years ago, I pointed out that our modern discussion of Title IX focuses too much on athletics and ignores the broad sweep of changes brought by Title IX. So what does the White House blog mention when previewing its Title IX celebration? Athletics (in an entry written by United Nations Ambassador Susan Rice).

Conversation often works ... where it's tried

Today, ACTA's Anne Neal thanked the AAUP and AACU for welcoming her outreach efforts.Towards the end of the blog entry, she writes,

ACTA also shares many faculty members' legitimate concern about administrative bloat and about trustees who lack a sensitive understanding of the special protocols and values that underwrite the unique enterprise of higher education. That said, we also believe that it is the professoriate's job to reach out to trustees. Faculty should understand that presidents and trustees are engaged in enormously complex, vital, and often urgent fiduciary endeavors. They should also understand that, going forward, trustees must be included among academia's primary stakeholders, alongside faculty and administrators.

I hope that's possible; that depends both on faculty and on trustees not accepting upper-level administrators as gatekeepers. My experience in Florida is that trustees often accept the role of administrators as gatekeepers of information, so that a president can essentially filter out quite a bit. I know of one UFF chapter at a community college that was able to meet with the chair of the trustees and establish a good working relationship, but that's rare. Far more common is a fairly uncomfortable and unproductive divide between trustees and most faculty, with a handful of administrators controlling the interaction.

I suspect that there's a pretty easy way to prevent greater access from becoming a vehicle for cranks and sophists (who will get their word in, anyway): err... asking faculty to provide the reality-check filter.

For those readers outside Florida, what is your experience with the extent of interaction between governing-board members and faculty?

The world is complicated, part 752

So the Center for Research on Education Outcomes has a report on charter-school performance, the Center on Education Policy has released a report on student achievement trends, NAEP released art-education data, and the spin has begun. Missing from almost all the reporting: Statements about the extent of peer reviewing for any of these reports. I'm not too worried about the professionalism of these reports, since I know that the Department of Education always has an internal review process, CEP usually asks researchers in the area to review draft reports, and I would be surprised if CREDO did not have a pre-publication review process. However, the failure to report on the extent of peer review is a continuing and glaring omission in the reporting of education research.

In terms of the substance of the reports, I'm up to my eyeballs in prior commitments, but it's clear from the brief reading I have been able to do that the findings for all three reports are more complicated than the spin emanating for many of The Usual Suspects.* That's not news, I know, but I am the King of Things That Are Obvious Once He States Them, and I have a job to do.

* a great name for an a cappella group, if you happen to be starting one up.

Iran's university students and faculty under threat

According to a Chronicle of Higher Education report late today, as well as Twitter reports and images/videos from bloggers who have been able to post, Iran's universities are under attack, either physically (with property destroyed and some reports of student deaths) or political pressures. If the clerical authorities gave orders for the Revolutionary Guard or other forces to attack universities, they are willing to throw overboard civic institutions as well as electoral politics to preserve their power. I'm not surprised. I'm very saddened, but I'm not surprised.