DrugMonkey

Dealing with the 12 page NIH grant format

Thu, 11 Mar 2010 16:19:43 -0500

Many of my readers will have already faced the joys of the shorter NIH Grant application. Briefly, the meat of the R01 proposal has now been reduced from 25 to 12 pages and the meat of the R21 from 15 pages to 6. As I observed when the Notice appeared, this is a challenge.

Since I am finally getting serious about trying to write one of these new format grants, I am thinking about how to maximize the information content. One thought that immediately strikes me is....cheat!

By which I mean taking sections that normally I would have put in the page-limited part of the grant and sneaking them in elsewhere. I have come up with the following and am looking for more tips and ideas from you, Dear Reader.

1) Moving the animal methods to the Vertebrate Animals section. I'm usually doing quite a bit of duplication of the Vertebrate Animals stuff in my General Methods subheading at the very end of the old Research Design section. I can move much of that, including possibly some research stuff that fits under point 4 (ensuring discomfort and distress is managed), to the Vertebrate Animals section.

2) Moving physical methods and maybe some Preliminary Data to the Facilities & Other Resources statement. This is supposed to:

Describe how the scientific environment in which the research will be done contributes to the probability of success (e.g., institutional support, physical resources, and intellectual rapport). In describing the scientific environment in which the work will be done, discuss ways in which the proposed studies will benefit from unique features of the scientific environment or subject poopulations or will employ useful collaborative arrangements.

or perhaps the Equipment statement:

List major items of equipment already available for this project and, if appropriate identify location and pertinent capabilities.

Pertinent capabilities? Ways in which the studies will benefit from unique features? Seems to me that this a potential window to not only describe physical equipment but also to slip in some data showing that this equipment is "pertinent" and that the studies proposed will "benefit".

AMIRITE?

Give me some more hot tips in the comments...

Your Grant in Review: The outlier proves I need to appeal!!!!

Wed, 10 Mar 2010 19:14:28 -0500

I've been meaning to pick up on a comment made by a reader over at writedit's epic thread on NIH paylines, scores and whatall. (If you want to swap war stories and score/IC payline grumbling, that is the hot place in town.) The guy was ticked off about a recent review he received and had a question:

I am an establishe investigator. I subnitted a competing renewal ... I got a score of 40 (37 percentile). I was very shocked and dissapointed to find out that my application had a preliminary score of 2.7 (which would have been fundable) but it seems one negative reviewer carried the day, and convinced others to pull down the score. I have not yet seen the comments, but if the comments have factual errors, especially from the negative errors, can I appeal the review and request a re-review?

Recently, as luck would have it, a loyal reader of the blog submitted the following scores, received on the review of her R01 grant proposal. Under the new scoring procedures in place since last June, these are scores which each reviewer suggests for criteria of Significance, Investigator, Innovation, Approach and Environment. I may have slightly re-ordered specific scores for concealment purposes but this is essentially the flavor.

rev#1: 2,1,1,1,1

rev#2: 2,2,3,3,1

rev#3: 3,2,5,4,2

It really is always Reviewer #3, isn't it?

Although we have more detail in the second case, let us credit the first person's description of events, leading to more-or-less equivalent scenarios. The appearance that two of three reviewers loved the proposal a whole lot and the third managed to torpedo it.

My first response to the specific scores would be "Congratulations! You must have written a pretty good proposal!" if you managed to get someone throwing down the 1 scores with a 2 tossed out so they don't look like a total homer- that's good stuff. You have an advocate like that pulling for your app and it is hard to make the case you got a raw deal. The way I'm looking at scores these days, the 2s and 3s of the next-fondest reviewer are pretty schweet too. This is my point about the strong advocate- it could be that this next-fondest person is just looking for a reason to improve the scores. After all, he/she could be sitting on an app that for random reasons he/she felt was better and was trying to spread the scores out. Absent a strong advocate, you are stuck with 2-3 range. If another reviewer is pressing for better, you might just get two pulling toward the 1-2 range. That, in my current understanding, being the entry card for a shot at a fundable score.

But...dum..da..dum..dum what's up with that third reviewer?

4th, 3rd, tomato tomahtoCalm down, calm down. Those scores aren't all that bad really. The allowable range goes up to 9 and there are only 1-2 pt gaps across the ordered reviewers. So the score disparity isn't huge. Functionally, these scores can mean all the difference in the world of course. Unless the odd reviewer finds that s/he just completely misread the app in some particular, no way s/he is going to be talked down below a 3 on those scores.

You will note at this point that I am assuming that the preliminary overall scores are somewhat related to the individual scores. There is not supposed to be any specific relationship, which is a topic of rant-inducing proportion for another day. Suffice it to say that to a first approximation I am comfortable making the leap. So the initial preliminary overall score identified by these reviewers were probably 1 or 2 (2 unlikely), 2 or 3 (about equi-likely) and a 3 or 4. (Here I am making the further assumption that the scores were not substantially edited after the meeting.) What are the possible post-discussion scenarios and voting outcomes?

Well, it could have stayed similar to initial. Perhaps the advocate talked the middle one down so you ended up with 1, 2, 4 or even 1, 1, 4. Maybe the middle or bad one talked the advocate up. So it was 2,3,4 or worse. And then we have to make assumptions about which reviewer was most convincing to the mean of the panel itself. Some 20 more reviewers would be voting, typically within the post-discussion range. Did they lean to the good side, minimizing the contribution of the detested Reviewer #3? Or did they lean towards spiking the app? Were they split?

Getting back to the comment waaaay up at the top of the post, it is generally ridiculous to claim that one reviewer ruined your chances of funding in a way that is unfair or shows that the system is broken. After all, I don't ever (and I mean ever) hear anyone claiming the system is broken or screaming about appeal because of receiving an outlying score in the favorable direction.

Since you've been so patient, the reader was kind enough to relate the app ended up with a 26 priority score (i.e., the vote averaged 2.6). So looks like the panel voted smack dab in the middle of the range identified in the reviewer's original critiques. But so what? If it had ended up toward a 3.5 or so, we would only conclude that the "bad" reviewer was convincing to a whole panel of people. That is no flaw in the system. And if it had trended more toward a 2.1 or so? Everyone would be jumping around high-fiving each other. Except that one outlying reviewer perhaps. But s/he shouldn't be grousing about the system either.

Final thought on reading the tea leaves. There will be additional clues in the "resume" section of the summary statement if your SRO is any good. Best case scenario is that a single issue, maybe two, identifies the core problem. Worst case, there are several things mentioned without a lot of clarity. But this probably means that the panel did not coalesce into a single viewpoint. Sometimes this difference can help you decide what to spend the most time on during your revision process.

When scientist audience is from another field it is still "outreach"

Tue, 09 Mar 2010 18:00:52 -0500

A recent paper from I. Kouper entitled "Science blogs and public engagement with science: practices, challenges, and opportunities" has been receiving a fair bit of bloggy attention. Of the negative sort. Mostly because the paper purports to report on the state of all science blogging but then cherry picks a few blogs to generate data- which is not actually presented for the most part. Furthermore the paper ends up with a subjective review of blog tone, level and commentary that makes one wonder if the author actually reads blogs at all. It is just that detached from the experience of many of us.

Bora was particularly annoyed and held forth at some length. Additional thoughts were advanced at Cosmic Variance, Panda's Thumb andPharyngula.

Since this blog was included in the alleged dataset, narcissistically, I felt I had better point out some more flaws in this paper. Let's get the hilarious one out of the way first.

Kouper includes the following comment example from this very blog (awww...):

Insults, such as "Don't be an idiot.. rtfa" or "Could you possibly sound any more stupid with this comment?" were more common for some blogs than the others. Thus, Wired Science and Panda's Thumb were filled with insulting commentary. Offensive remarks regarding somebody's personality or intellectual abilities most often targeted other commenters and the characters of posts, but sometimes they were directed at blog authors as well, such as the following comment in DrugMonkey blog: "You are correct, I never read a post in which you claim not to be pompous and arrogant".

Now, of course, IMNSHO, that is pretty tepid stuff if it is supposed to be an example of rampant incivility. Which seemed to be the author's intent given the context within which the quote was placed. Nevertheless, the absolute context tone-deafness of the author is staggering. This was a comment from S. Rivlin! The comment, of course, was directed at Comrade PhysioProf who in that very thread gave all the evidence Kouper needed as to why a reader might direct rudeness toward the blog author. Now admittedly it would take a little more digging to figure out that S. Rivlin and Comrade PhysioProf maintain this cartoonish antagonism on the blog for reasons that are clear only to themselves (and possibly their therapists, if any). This exchange is by no means representative of anything. Not representative of comment exchanges on this blog and certainly not representative of the less....er, freely commented blogs.

In fact, looking at the list of blogs and blog aggregations/collectives listed in the Table 1, it is not clear why we were picked out for focus in the first place. This brings me to my main complaint about Kouper's take on "science blogs". One central thrust of the paper is that science blogs are directed at an audience of scientists, are not reaching non-scientists at all and therefore are doin it rongz. As the author concludes:

To become a tool for non-scientist participation, science blogs need to stabilize as a genre or as a set of subgenres where smaller conversations may facilitate more meaningful participation from members of the public

So if the goal was to review science blogs that were designed primarily to engage the nonscientist public, what in blazes is she doing selecting DrugMonkey, a blog which focuses in very large part on inside-science-baseball topics? When we're talking about how to succeed at the NIH grant game around here, we are most certainly not talking to nonscientists!

Now admittedly, I do post something on a more general interest topic now and again. Of course, if Kouper had focused her reviews on those posts, it would have been obvious that I reach greater number of nonscientists when talking about drugs. Let us back up and review the author's source of information on the demographics of blog readers:

Readers of science blogs also had some relationship with science, i.e., they were not exactly non-scientists
or lay persons. One author posted a message titled "Who are you?" and asked his readers for information about themselves and their background. The answers to this post as well as the overall analysis of readers' comments demonstrate that those readers who engage in commenting are almost always associated with science one way or another.

I'm not positive but I suspect, given the data were collected in "Summer of 2008" it was this post of mine, given my title. Unlikely that it was the related one at NERS that the author should have reviewed given that I linked it as my motivation! Or the similar threads from ScienceWoman, Coturnix and drdrA that I linked. Also see Janet Stemwedel's post. If the author had read with a close eye she would have seen that while scientists dominate the readership, they are by no means the only demographic. There are detectable numbers of non-scientist commenters. Kouper makes the further mistake of not understanding that comments are not equal to readers. In fact, many bloggers bandy around numbers that suggest reader comment numbers are only about 5% or less of the number of unique visits to a post. Perhaps the scientist/nonscientist ratio is similar between lurker and commenters but perhaps it is not. We simply do not know.

What we do know is that nonscientists are being reached, even if they are not the majority demographic. So the concluding tone in the article is a bit overdrawn.

There is one final, and critical, point that seems to have escaped the author's notice entirely. Scientists ARE the uninformed public. In my case, for example, I know very little about paleontology. So when I read Laelaps or Tetrapod Zoology, I learn a tremendous amount. Similarly, I go to the earth science folks to give me a little context for the latest earthquakes causing disasters around the world.

Now whether I inform them or not, it is indubitably the case that I have a lot of scientist readers that know very little about drug abuse. Beyond the typical user-level of knowledge that is. They are not working in substance abuse or related fields. Maybe, at best, we have an appreciable number of neuroscientists. And a lot of folks who work on biological systems in some way. We also have nonbiologists in the audience. When it comes to the topic domain of substance abuse, they are all more or less equivalently non-expert.

So just as I am in the "outreach" target demographic for the bone jockeys, my readers are most emphatically in the target demographic for any scientific education and outreach that might motivate my blogging.

Kouper seems to have missed this.

Exercise Phys Dudes Join Up

Mon, 08 Mar 2010 20:45:27 -0500

Go entertain yourself with the Obesity Panacea blog (previously here).

If you don't laugh at the Ten Most Annoying Gym Personalities you need to, well, hit the gym.

Welcome Peter and Travis!

Okay, disgruntledocs, NIGMS is listening- go nuts!

Thu, 04 Mar 2010 14:56:04 -0500

From the NIGMS Strategic Plan site:

NIGMS has a long-standing commitment to research training and biomedical workforce development. As science, the conduct of research, and workforce needs evolve, we want to be sure that our training and career development activities most effectively meet current needs and anticipate emerging opportunities, and that they contribute to building a highly capable, diverse biomedical research workforce. To this end, we are engaged in a strategic planning process to examine our existing activities and articulate strategies to help us build and sustain the workforce that the nation needs for improving health and global competitiveness.
We are seeking broad input for this planning effort from university and college faculty members and administrators, current and former predoctoral and postdoctoral trainees, industry representatives, representatives of professional and scientific organizations, and other interested parties.

From March 2 to April 21, 2010, you may give us your input on our Web site in response to the series of questions below. These submissions will be completely anonymous.

1. What constitutes "success" in biomedical research training from the perspectives of an individual trainee, an institution, and society?

2. What can NIGMS do to encourage an optimal balance of breadth and depth in research training?

3. What can NIGMS do to encourage an appropriate balance between research productivity and successful outcomes for the mentor's trainees?

4. What can NIGMS do through its training programs to promote and encourage greater diversity in the biomedical research workforce?

5. Recognizing that students have different career goals and interests, should NIGMS encourage greater flexibility in training, and if so, how?

6. What should NIGMS do to ensure that institutions monitor, measure, and continuously improve the quality of their training efforts?

7. Do you have other comments or recommendations regarding NIGMS-sponsored training?

Go comment.

Your Grant in Review: Where did that fourth critique come from?

Thu, 04 Mar 2010 12:33:18 -0500

A comment over at writedit's unending thread on NIH grant paylines caught my attention.

Hello All,

I just received my summary statement for an R15 I submitted in October and found out that I had four reviewers score my proposal. I thought that three reviewers was routine. Would anyone know why a fourth was included?

Thanks for your help.

I had the following response, slightly expanded for this context.

Any member of the panel can review any proposal (save conflicts) assigned to that panel and write a critique if they so choose. In theory you could end up with 20 or more of them!

Of course this never happens. Although I will say that it seems to me that the chances of picking up extra reviews is enhanced when your application is reviewed in a tiny Special Emphasis Panel convened to cover just a handful of applications. (I think I've had a 5-reviewer summary statement as my high water mark.)

What I have seen occur at a normal study section meeting is that someone not assigned to the application will get really involved in some point of discussion (pro or con, people, pro OR con). The system recognizes that the summary statement is supposed to reflect the discussion. Revision of the critique to more accurately depict the discussion is highly encouraged.

So sometimes the SRO or Chair or a panel member will say "Hey, will you 1) write that up as a critique; 2) write up a brief blurb for the SRO to include when prepping the resume or 3) write up a point for one of the other reviewers to include when they are editing their review?"

Or, occasionally, a reviewer will just be so ticked about where the assigned reviewers and panel votes went that s/he will just write up a dissenting view on his/her own hook. I'd suggest that if you see a fourth, highly disparate review, perhaps focused on a single issue, that you recognize that this probably had very little effect on your score... Nice to get the warm fuzzies if the outlier loved your proposal, I suppose.

One final scenario may also be at work. The way I understand it* the official rules require that at least two people assigned to the application have to be in the room if it is to be discussed. So if for some reason 2 of the 3 are phone reviewers, the SRO may tap a 4th "discussant" reviewer to fulfill this requirement.

*it is possible that this is only a strongly encouraged practice and/or a practice limited to selected study sections. I'm not easily finding instructions-to-SROs on the CSR website.

Ask DrugMonkey: Will you comment for attribution?

Tue, 02 Mar 2010 16:37:11 -0500

Every now and again I get a request to provide information to an individual who writes for a mainstream media print outlet. Sometimes ones that are focused on the scientific enterprise, sometimes ones that have a regional focus and sometimes the larger, more general national news organizations. The latest one was from Sharon Begley of Newsweek .

The typical request includes some stroking "I'm a big fan of your blog" (yeah, you and the spam commenters, bucko) and a request for a voice interview on some topic of interest that is obviously related to my blogging.

So far, so good.

Even if I do not really understand the fetish with the voice interview over the email exchange. (Oh, ok, yes I do*.)

Then, with various degrees of clumsiness, they work around to the question of my real identity and how they would like to cite that real identity in their article.

Of course, I refuse.

Now, here is what fascinates me. Actually, first I need a little disclaimer. It is possible that I have lingering irritation with the current journalist because of an article she wrote about the NIH funding system. It is the usual wah-wah about how the NIH refused to fund the lone genius who turned out later to be RightAllAlong, therefore the SystemIsBroken. Aiiiieeeee. I was going to blog it but I see I never did. Ah well.

Anyway, back to the task at hand. See if you can follow the MSM journalist's logic.

They allegedly read the blogging of DrugMonkey, find it informative and interesting to the point where they wish his input on a story they are writing. Input that they would like to use as whatever they think of* as a direct attributed quote.

In the absence of direct quoting or attributing they are happy to use the input of DrugMonkey in creating the eventual story. On background.

The alleged reason for not naming a pseudonymous source in the article, is that they "owe it to the readers" to tell them whose opinion is coloring the story.

I find this fascinating. First, because there is by now a rather large body of opinion written by DrugMonkey on various topics. These may be easily located from just about anywhere one can find an uncensored Internet connection. These writings are far more informative about "who" I am than my real world name could possibly be. My real world self does not have any specific role opinionating for MSM type audiences. At best, I would hope that some narrow subset of my research articles reach an audience beyond the narrow scientific field at which they are directed. Those are rather dry scientific communications, however, and convey very little about "who" I am when it comes to opinionating on topics of interest to the consumers of MSM. So why would citing my real self be somehow better than my pseudonym? It actually conveys less information to the reader. In a sense.

The second implication is a bit nastier for the journalists. The claim is that the journalist "owes" transparency of sources to the reader. I can tentatively sympathize with this goal. However, the journalist is full willing to interview a source who does not want to be identified in print and to incorporate that information either overtly or unconsciously into the resulting article. In such a case, the source of the slant or opinion expressed as the journalist's own take on the issue is entirely opaque! The journalist does not reveal how many sources she or he consulted, which primary literature or random websites were reviewed or anything like that in the article. That is all 'background'. And yet this background, and the resulting slant, can have a much greater effect on the reader than do direct quotes*.

In the substance abuse fields, probably the largest category of potential conflict of interest with respect to opining in news media is the category of therapy for drug addiction. Whether we are talking proprietary rehabilitation schemas, computer assisted partial replacements for old fashioned talk therapy or drugs to be approved by the national regulatory bodies, this is where we start to touch on traditional areas of conflict of interest in which it is very obviously in the interest of the reader to know "who" is talking. Is the person's loyalty more to the accurate depiction of the scientific support? Or is the source looking to enhance the stock value of a company for whom the source consults?

In my case, I like to use the general disclaimer of "assume that I am conflicted to the hilt" because I do not fear any decrement in the impact of what I have to say. I have no need to establish my bona fides in that way. I deduce that MSM journalists would not prefer to use this standard and in fact like to pretend they are giving balance by including competing biases in the story quotes. Or, at the least, interviewing people with opposed opinions. And overall to compartmentalize and minimize any implication that the source even has a bias.

The fact is, however, this balance is not readily apparent to the readers. The reader has to trust that the journalist is not expressing a biased viewpoint that has been heavily influenced by one of his or her "not for attribution" sources. The reader has to trust that the journalist has vetted the source for conflict. Because the resulting article provides no clues whatsoever as to the identity, viewpoint or potential biases of the source.

When considered in this light, citing an pseudonymous blogger with an extensive track record seems to serve the journalist's readers a heck of a lot better than off-the-record background interviewing.

__
*The practice of journalism is, from what I can discern, interested solely in cherry picking and quote mining to support the pre-existing story instead of conveying an accurate representation of a source's position. Once you understand this, many of their bizarre practices become more intelligible.[Related: Coturnix on The Ethics of the Quote]

Things White People Love: Comparing Black People to Monkeys

Fri, 26 Feb 2010 12:57:04 -0500

No, I don't mean the self-imagined political wag, nor those of a similarly fantastical oppressed ethnic subculture of the US. I mean the kind of (over)educated middle to upper-middle class, progressive liberal occasionally self-avowed skeptic, contrarian and/or scientific white folk.

I've seen the odd social justice action now and again in the US over the past decades. Whether at the local municipal level, the University level or on national TV. African-Americans are typically very well represented even when the issue at hand is not a "black issue" per se. When it comes to the incredible underrepresentative University campus population, it is particularly striking because you will find the black faces that you'd never known were on campus appearing in support of social justice causes.

There is one notable exception and that is the animal-rights campaign. The practitioners of animal rights theology would have you believe they are engaged in a social justice battle akin to many familiar ones. They never seem to look around and ask why their tiny band of followers are so unusually devoid of the black folks.

Perhaps it is because one of their favorite memes is viscerally offensive to African-Americans?

Not to mention entirely inaccurate from a scientific and historical perspective.

I know, I know. I know what you are thinking. It is a post in woo- and sensation-friendly Huffington Post, noted enemy of science and lover of all that is wacky. Why get all bloggy about a dog-bites-man story about yet another scientifically disingenuous entry at HuffPo?

I suppose my motivation is that you will see this sentiment elsewhere; so we can't just blame it on HuffPo. Besides, the blogger in question, Eric Michael Johnson, identifies himself in his tagline as "Scholar, scientist and journalist". So he might be expected to bring some good scientific support for the meme that is offensive to black folks. If he does not, as a self-avowed "scientist" he should tolerate, even welcome, a critical assessment of his theorizing.

In his tepid chastisement of the violent ARA extreme, EMJ focuses mostly on tactical outcome:

I would urge you to think about your ultimate goal in this struggle and consider whether your tactics are helping to bring this goal to fruition or whether they are emboldening your adversaries and further entrenching the divide.

Notably missing is any sense of moral or ethical outrage over the violent excesses directed against scientists- most likely because EMJ has a lot of "sympathy" for their "frustrations". You will recognize this as the typical double-talk coming from prominent mouth pieces of the extremist groups which are nominally in the accepted public sphere. I'm sure others will be addressing that aspect of the post at great length but I wanted to return to the "Monkeys are just like black slaves" argument. As formulated by EMJ in his post at HuffPo:

There are many who would object to the comparison between enslaved human beings and enslaved nonhuman primates. However, like you, and as a primatologist, I view this to be a difference of degree rather than kind. Research on primates in the wild has shown that they have rich emotional capacities including affectionate family bonds; long-term social relationships; the conscious awareness of self as separate from others; altruism; communication through gesture, body posture, facial expression and sound; learning by observation; making and using tools; using medicinal plants to treat illness; understanding and using abstract symbols for communication; and manipulating social situations for their own purposes. They are our next of kin in an evolutionary sense and I believe that rejection of our common kinship today is similar to the rejection that whites felt towards blacks just a few centuries ago.

Yes, I object. First because of a sort of historical mischaracterization of the existing strength of the evidence that enslaved black humans were similar to non-enslaved white humans on a list of critical characteristics. Second, because this brief list of observations supposedly derived from "research" in "nonhuman primates" substantially mis-characterizes the strength of the evidence. There is evidence, yes, that could be viewed as supporting human-like qualities of (some of) these nonhuman primates. But if we have some familiarity with the strength of the evidence, the comparison with enslaved human beings becomes inescapably offensive.

Let us start with the fact that the comparison is historically laughable. Just because grand pronouncements about enslaved black people were being made by people who claimed to be learned and scientific, this does not mean their evidence at the time had any merit. In fact it did not take any effort at all to readily observe black individuals, enslaved or not, who demonstrated consciousness of self, altruism, language (not the "abstract symbols for communication" misdirection, language), etc, etc, up to and including the sort of complex skills required to oversee the banquet kitchen of the President of the United States! At the very time, all society had to do was basically to step out of the way of black folks and to stop artificially preventing them from expressing their full behavioral repertoire to falsify any claim that black people differed in any way on these traits that EMJ has listed off. As history has progressed to modern day, we have all the evidence you could wish for that a person with black skin and more proximal African heritage is possessed of any fundamentally human-like traits you might assign to white people.

Again, this evidence emerges trivially. It does not require unusual training or intervention. All it requires is giving the white and black person an approximately similar set of environmental circumstances. Any layperson of unprejudiced mind can see it. The data support it. Any highly educated person that wishes to argue the contrary has a very tough job indeed parsing rarified statistical arguments about highly non-essential traits of humanity (like IQ three standard deviations above the mean) to establish any relevant differences between those of differing skin phenotype. On a group basis or even (given normal development) on an individual basis.

In very sharp contrast, we have the evidence that nonhuman animals, of the primate Order or otherwise, express traits or capabilities that are similar to those of human animals.

There have been many highly effortful attempts to let chimpanzees, gorillas and even African Gray parrots express their capacity for language. From the Premacks to the Georgia State University Language Resource Center to Irene Pepperberg, investigators have spent unbelievable amounts of time with painstaking instrumental conditioning of behaviors that might look like language. The demonstrations can be made to appear highly impressive to the naive viewer. However, just like Clever Hans and just like Epstein's insightful pigeon, appearances can be deceiving. One can take the end result of one best individual exemplar of a species, among several other exemplars who have failed to exhibit the same degree of 'success', and make it look very impressive.

Then the uncritical mind fills in the blanks. The uncritical mind assumes that any other individual can do the same thing, which is demonstrably not true. These are the best exemplars and the investigators have other considerably less-capable exemplars that never seem to garner the same attention or be integrated into the Gestalt interpretation. The uncritical mind glosses over or ignores the incredibly wide gulf between what Alex and Washoe and company can do...and what a normally developing human toddler, or even an adult second language-learner, can do. The uncritical mind ignores the incremental shaping that was required to bring about the best exemplar*. The uncritical mind ignores the fact that you have to go so some unbelievably unusual lengths to prevent a normal human from acquiring what is undubitably genuine language.

Do I need to remind you that black folks, even the enslaved ones, acquired true language, often that of their white owners, trivially?

Altruism, numerosity, tool use...the list goes on and on. There are data. There have been often highly effortful investigations. To produce very slim evidence that maybe, perhaps, nonhuman animals have traits and behaviors that are something like human traits and behaviors. And really, to assert otherwise is simply magical thinking that good skeptics and atheists and Darwin botherers should dismiss out of hand. Of COURSE humans are animals and share behaviors with the descendants of their common ancestor. It would be ludicrous to pretend otherwise. So why are well educated animal rights activist tilting at this ridiculous straw argument?

Since EMJ offered up tactical advice about making the AR case, I should reply in kind. I dunno, maybe the black folks I fail to see in animal rights activities are demonstrating elsewhere that day. Maybe they exist in droves. If you know any, send them over here to comment. If I'm right however, EMJ and other fans of the slavery / Tuskeegee analogy might want to think on recruiting allies to their marginalized cause. They may want to think on what it says to black people when they make this insulting comparison of black people's capabilities with those of monkeys. It doesn't come across well.

__
*[For those of you who are not behavioral scientists. Think about it. Think about your own assays and experiments. How often would you take a highly effortful assay that takes years of daily labor to prepare, observe a result in one of many preps, a result which is highly debatable and may require your mother's eye to distinguish from background..and conclude that that one is the truth? Never. Ask these comparative behavior scientists some tough questions.]

Repost: Insightful Animal Behavior: A "Sufficiently Advanced Technology"

Fri, 26 Feb 2010 10:13:42 -0500

We've been talking about the use of animals in research lately. One thing that always comes up is how animals share some critical capacity with humans. I try to point out in all of this that some of these questions are amenable to investigation. Some of the claims can be supported, nullified or qualified on the basis of existing data. The process of describing or interpreting the data is never simple. And it seems that many people who parrot what seem to be simple claims actually have very little understanding of the evidence on which they rest. I have at least one observation in the archive that points out where not thinking hard about the strength of the evidence can lead to unsupported conclusions being widely disseminated. This post was originally published Feb 25, 2008.

In the midst of World War I, Wolfgang Köhler conducted a famous series of experiments to investigate problem solving ability in chimpanzees. The lasting impression of these experiments, reinforced by just about every introductory Psychology text, was Köhler's assertion that the chimps demonstrated "insightful" learning.

Did they now?

From this site:

Köhler's most well-known work on chimp cognition was in the use of tools to gain access to food. A chimp would have to stack boxes to reach a banana that was suspended out of reach, or insert a narrow stick into a thicker one to produce a tool long enough to reach food. While Köhler's star chimp, Sultan, did not immediately put two shorter sticks together to make one long one, he worked on the sticks for over an hour. When they had fitted together, Sultan immediately used the new tool to retrieve the bananas. This solution demonstrates insight - recognizing the "problem space" - rather than foresight.

Another set of descriptions from Köhler gushing over how the chimpanzee "spontaneously" jumps to the correct solution ("Eureka!") without evidence of incremental training. Amazing!

Or, is it...

OMG! Pigeons have insight! We must reorganize our whole understanding of the supremacy of the apes. Right?

The scholarly description of this little experiment, while less intuitive than the video, resulted in a peer-reviewed publication. From some OldeSkool behaviorists making the point that when the behavioral and environmental antecedents are unknown, we might be led astray by our assumptions.

Epstein R, Kirshnit CE, Lanza RP, Rubin LC. 'Insight' in the pigeon: antecedents and determinants of an intelligent performance. Nature. 1984 Mar 1-7;308(5954):61-2.

Somehow I am reminded of Arthur C. Clarke's Third Law: "Any sufficiently advanced technology is indistinguishable from magic".

Now it may be the case that all brain/behavior relationships in animals more complicated than a cockroach approach Clarke's "sufficiently advanced technology" as far as we are concerned at present. Nevertheless, at least the behaviorist tradition results in demonstrations that seem understandable to the average college freshman. You have stimuli, you shape incremental responses with the provision of consequences for particular responses and you observe learning or acquisition curves. It makes sense.

However sometimes we visit phenomena for which we do not have a great understanding of, or appreciation for, the "technology". We understand much less about what generates the higher cognitive skills out of the (admittedly complicated) anatomy and signaling (chemical and electrical) properties of the brain. We may not have much appreciation for the behavioral history of animals in the wild or semi-feral captivity. And so. And so. When faced with phenotypes which seem beyond our ability to explain with our comprehensible level of technology, we grasp for magical explanations. "Insight"? "Consciousness"? "Self-awareness"? We struggle with understanding and classifying these properties of the brain.

I don't want to belabor the analogy too much. I am not suggesting that believing in "insightful learning" or "consciousness" is the same thing as believing in "magic". My point is that we should be cautious in assigning inferences of qualitatively different processes just because it is not immediately apparent to us, e.g. how a particular skill was acquired. Cautious in assuming that just because a specific species exemplar sitting before us can or cannot demonstrate a particular behavior this means that all members of the species can / cannot perform similarly.

And most importantly, we should be cautious in assigning order-of-magnitude or quality-based rankings. For example if we have the "understandable" technologies and the "sufficiently advanced" technologies lined up, the lowliest of the "magic" technologies may be, in objective terms, much closer to our higher "understandable" technologies than they are to the highest of the "magic/incomprehensible" technologies. In other words, the "insight" expressed by Köhler's chimpanzee subjects may be in objective terms much closer to the "insight" expressed by Epstein's pigeon than it is to some version of human "insight" which leads, for example to an advance in understanding brain function in a neuroscience laboratory.

Returning to a couple of additional "technologies" of the nonhuman ape brain, it turns out that chimpanzee's highly touted ability to fish for termites with a bit of stick is a meticulously learned behavior taught by the mother. But..but.. tool use was supposed to be some qualitative watershed. A hugely significant difference in the animal kingdom between the "haves" and "have nots". And believe you me it is a very large part of the comparative cognition theme to insist on examining "spontaneous" behaviors that are not trained and shaped though incremental learning. (It will not surprise you that the above Epstein video and similar behaviorist demonstration are precisely the reason for this distinction.) Now we have come to appreciate that this supposedly species-typical behavior, termite fishing, is incrementally learned. Now what would happen if some members of a species received incremental training and some did not?

It has been shown that other tool use in chimpanzees is culturally limited, meaning that one group of the species may be found to use rocks to crack hard-shelled coula nuts whereas another group many miles away may never do so, despite plenty of nuts and rocks lying around. So at one point, a theoretical pair of field primatologists working with distant groups of subjects might reach opposite conclusions about the "innate capability" of a single species for use of a particular tool. Should we conclude from this that chimpanzee group A is more "insightful" than chimpanzee group B in a constitutive, unalterable way? That they exhibit qualitatively and significantly different cognitive "ability"? Of course not...once we put all the data together. Then we realize we need to go looking for additional environmental contributions to the observed behavioral phenotype.

And tone down our rhetoric on nice neat species differences in "innate capability".

A nice side effect, is that by considering the different environmental contributions to brain/behavior phenotypes which may on first blush seem fantastical or nearly miraculous, we can gain a more realistic perspective on the "technological difference" scale from simpler organisms all the way up to what is the most complicated brain technology of our acquaintance, the human.

An Open Letter to assorted Pharynguloids, Insolents, Mathphiliacs and the like.

Thu, 25 Feb 2010 14:32:51 -0500

Thank you.

In response to the blog posts of PZ, orac, MarkCC and many others, you did what you do best. You commented. At the home blogs and at the sites of the violent Animal Rights extremists who feel it is acceptable to threaten the children of a man who no longer does the work to which they object.

Some of you were in support of animal research and some were not. But even many of the latter stood up for the civil society of discourse, unfettered by the intimidation of personal threats.

You know this, of course, because you are already engaged in the blogosphere. What you may not see is the gratitude of scientists who work with animals. In the past day I have seen several expressions of gratitude from my scientific friends, peers and colleagues in other venues. They are happy, and in cases overjoyed, to see such public support. Even if many of you are pseudonymous or anonymous. Their joy is a surprised one because it is so rare in their careers to see much in the way of individual support for their research and, more importantly, personal antagonism directed toward the extremist violent fringe of the AR movement. Even as you delight in acting the skeptic, calling out asshats, FWDAOTI and other assorted blogospheric games, know that you have real beneficial impact on the scientists who are just trying to do their jobs, create knowledge and, in many cases, improve human health in ways both small and large. It helps for these individuals to see the personal and individual expressions of outrage and support that you have posted around in various places. It makes them see in a very tangible way that they are not facing the onslaught alone.

In the name of my colleagues I thank you.

On redefining "sentience"

Wed, 24 Feb 2010 18:58:33 -0500

You may have noticed a rash of posts around the ScienceBlogs decrying the ARA terrorist extremists who have vowed, again, to target the children of a UCLA neuroscientist. Dario Ringach famously gave up his nonhuman primate research in 2006 because of threats against his family. His participation in last week's dialog held at the UCLA campus apparently induced the extremist attention seekers, angry at having the momentum and PR shift to their slightly more rational co-travelers, to renew their threats. This is utterly despicable. Utterly.

This would be a great time for people who purport to be non-extremist animal rights advocates or sympathizers to do some deep soul searching. Soul searching that does not just easily write off the terrorists as a crazy fringe but asks penetrating questions about the nature of their own beliefs.

I cannot help you with this difficult work but I noticed something a little odd and new to me popping up in comment threads following the posts linked above. It has to do with the concept of sentience.

Wandering over to the Wikipedia entry I find a rather interesting set of observations.

Sentience is the ability to feel or perceive subjectively. The term is used in philosophy (particularly in the philosophy of animal ethics and in eastern philosophy) as well as in science fiction and (occasionally) in the study of artificial intelligence. In each of these fields the term is used slightly differently.
In eastern philosophy, sentience is a metaphysical quality of all things that requires our respect and care. In science fiction, sentience is "personhood": the essential quality that separates humankind from machines or animals. Sentience is used in the study of consciousness to describe the ability to have sensations or experiences, known to some Western academic philosophers as "qualia".

Some advocates of animal rights argue that many animals are sentient in that they can feel pleasure and pain, and that this entails being entitled to some moral or legal rights.

Well this certainly explains my confusion. To me, "sentience" has always been the science fiction concept. I suspect quite strongly that for most people, this is the connotation of the term.

Interesting, is it not, that animal rights people would co-opt this term to mean "can feel pleasure or pain"? Why create this new use for the term, particularly when it has such strong associations with the full-human capacity, different from animals and machines science-fiction type of definition?

Just another dishonest ploy to sway people to their way of thinking on something other than the merits. Of course they know what they are doing. Of course they know that they are creating this blurring of definitions in the minds of the undecided public. And of course they are hoping to lure everyone into using their terminology so that when people who are in favor of animal research say, well of course animals can feel pain, the ARA nut can claim that such people are admitting to sentience.

When of course they are doing no such thing.

Challenge anyone who uses this "sentience" gambit, eh? Get them to specify exactly what they mean. And ask what they are trying to pull with this redefinition nonsense.

As many dependent on cannabis as have tried heroin?

Wed, 24 Feb 2010 11:53:56 -0500

A recent review paper covers animal models of adolescent drug taking, which is in and of itself an interesting read. Human adolescence tends to be a time when people first encounter psychoactive substances taken for recreational purposes. Unsurprisingly, problematic drug taking which emerges later in life often has antecedent roots in adolescent drug taking. The epidemiology goes further in identifying age-graded risk such that the earlier one starts using some drugs, the greater the chances of problematic drug use later in life.

Given the inescapable limitations human epidemiology (lack of random assignment means you cannot eliminate risk-associated variables from the genetic to the environmental) animal models are required to determine if there are neurobiological sensitivities in the adolescent brain which confer increased risk of developing dependence on a given recreational drug. This paper reviews much of the animal studies to date.

Are adolescents more vulnerable to drug addiction than adults? Evidence from animal models. Schramm-Sapyta NL, Walker QD, Caster JM, Levin ED, Kuhn CM. Psychopharmacology (Berl). 2009 Sep;206(1):1-21.

In providing the background context, Schramm-Sapyta et alia did something dear to my heart, as my readers will quickly appreciate. They created an interesting graphical depiction of the conditional probability of dependence on several drugs from the US National Survey on Drug Use and Health (NSDUH, 2007).

Fig. 1 Percentages of the US population over the age of 12 years who have ever tried the indicated drug (top number, light gray circle); who used the indicated drug in the past month (middle number, darker gray circle); who meet criteria for dependence on the indicated drug (bottom number, black circle). Numbers in the center of each diagram represent the percentage of people who have ever tried the indicated substance who are currently dependent. Data obtained from the NSDUH 2007, lifetime use, past month use, DSM-IV dependence criteria (for all drugs except tobacco), and daily cigarette use (for tobacco)

Yes, these are cross-sectional data much like the Anthony et al 1994 paper to which I frequently refer. But these are also large representative sample data so trite dismissals on this basis are hard to support.

What draws my eye? Two things. First, of course, is the number in the center of each set of concentric circles. This represents the percentage of the sample of individuals who have sampled a given drug in their lifetime who are currently dependent. I emphasize "currently" because sometimes you will see data on "ever met dependence criteria"; this latter would result in a higher estimate of dependence risk, obviously. The Anthony et al 1994 data use lifetime rates, not current rates for example. Nevertheless this central number is another estimate of the conditional probability of becoming dependent and more importantly gives us relative relationships for several drugs of interest. [ As always in science, the more replication the better. Slightly different sampling / survey methods will give us different numbers of course. So let us not make the common mistake of viewing every scientific paper in isolation. These numbers need to be integrated with other estimates of conditional probability of dependence that we can find...in a thoughtful way. ]

The second thing that draws my attention can be found in the three numbers listed to the left of the circles. The bottom one represents the rate of dependence in the general population. To my view this is a critical bit of information for those who want to have an informed discussion of public health related policy. The number of citizens affected by a given condition is as assuredly relevant as is the severity of that condition.

These data make it emphatically clear that the number of people in the US who are currently dependent on cannabis slightly exceeds the number who have ever so much as tried heroin. The number dependent on cannabis is three times as large as the number dependent on cocaine and 16 times as large as the number of people dependent on heroin.

_
h/t: dr_leigh for passing this along

Mendel's Pink Sheets

Mon, 22 Feb 2010 21:27:21 -0500

One of our readers managed to unearth an unbelievable bit of scientific history and we are able to provide it to you thanks to his generosity. With no further delay we present....

click image

__
All credit, and my gratitude for the LOLz goes to Donn Young who describes the source of his re-creation as follows:

Long ago and far away [like the 1970's], I read an article in the old Journal of Irreproducible Results [JIR] that had a copy of Gregor Mendel's pink sheets. I have no idea who the author(s) was and I can no longer find the issue [or much else], but have this rewrite I put together using the 90's review criteria. I'd hand it out [on pink paper] to junior faculty as encouragement in the face of an unscored application - but hey, Mendel didn't have to worry about tenure.

Donn Young
Research Scientist and Director of Biostatistics [retired]
Ohio State University Comprehensive Cancer Center

A curious editorial from Nature, defending against "myths"

Mon, 22 Feb 2010 19:11:06 -0500

Did you read this fascinating bit of....whatever...from Nature? I picked this up from writedit's thread. The title and subtitles are, I kid you not, this.

Nature's choices: Exploding the myths surrounding how and why we select our research papers.

You might almost think they had read my recent post on the business of spin. Apparently they think they are taking some unfair knocks about their process for publishing papers and want to spin the story back around to their liking. Fair enough.

I don't buy their argument so I'll take my hand at spinning it back my way.

Now, what are these evil myths you might ask? They have a list.

One myth that never seems to die is that Nature's editors seek to inflate the journal's impact factor by sifting through submitted papers (some 16,000 last year) in search of those that promise a high citation rate. We don't.

Right. Sure you don't. You don't have to* "sift through submitted papers" to satisfy your shared interest with other GlamourPubs. Every bloody thing you do is directed at this goal! From defining the acceptable article as that which is the latest and greatest, to prioritizing first over best, to schmoozing PIs, fields and techniques which have a recent track record of astronomical citations, to engaging in co-publication shenanigans.....yeah. GlamourMags, of which Nature is unmistakably one, all seek to inflate their impact factor. Of course nobody can lay a glove on them with an accusation of a specific practice like sitting around the editorial table writing down an expected citation value. It doesn't work like that. This doesn't mean that the substance of the critique isn't valid though. Doesn't mean that editors aren't steeped through and through in policies and decision making that are designed to select papers for publication which will be cited frequently. So no, sorry, your protestations aren't going to do much to dispel the 'myth' that your journal is motivated to publish papers which you expect will result in high citation numbers.

but wait, they have data which will blow this idea clean out of the water...

Indeed, the papers we publish with citations in the tens greatly outnumber those in the 100s, although it is the latter that dominate our impact factor. We are proud of our full spectrum.

HAHAHA! So because you can't predict perfectly this is evidence that you aren't making the predictions in the first place? Or that since you have a broad mission across fields with different citations practices / rates / timelines and because "tens" might be the very top of some fields we're supposed to believe you don't know this? And oh, just btw, how hard do you work to make public the entire distribution of citations to show us just how proud you are of your "full spectrum"?

On to the next howler.

Another long-standing myth is that we allow one negative referee to determine the rejection of a paper. On the contrary, there were several occasions last year when all the referees were underwhelmed by a paper, yet we published it on the basis of our own estimation of its worth. ... But we make the final call on the basis of criteria such as the paper's depth of mechanistic insight, or its value as a data resource or in enabling applications of an innovative technique.

I think this is a straw argument and a straw defense. Of course any disgruntled rejected author is going to view one negative and two seemingly positive reviews as evidence the editors should have accepted their paper! This happens all up and down the taxonomy of scientific journals. The only knock would be whether Nature does this at a different (higher) rate than other journals. This defense does not address the real issue, instead rebutting the easy scenario of "we did overturn a negative review that once so there". Pfagh. [As always, I'll leave it as an exercise for the reader to determine if arguing for definitive proof based on single examples, instead of plural data, is discordant with what Nature seems to consider good science *cough*representative gel*cough*error bars*cough*cough*]. How often do you do it? How often do real journals do it?

Considering this issue we come to the next obvious question which is whether some negative reviews are more equal than others. This may be getting closer to the truth of this particular myth. After all, the Nature

editors spend several weeks a year in scientific meetings and labs

so when they assert

our decisions are not influenced by the identity or location of any author... we commonly reject papers whose authors happen to include distinguished or 'hot' scientists...another myth is that we rely on a small number of privileged referees in any given discipline. In fact, we used nearly 5,400 referees last year,

it does not really tell us anything about what happens with close calls or highly contested reviews. Is the cadre of hot scientists listened to more regularly on those cases? What happens when a paper is initially rejected (or gets a devastating request for unending additional data)? Do they take the phone calls of the distinguished scientists more frequently? Are you more likely to accept their revisions or resubmissions? And in any case, what a tired defense in this continued mode of throwing up irrelevant chaff. Who cares that you used 5,400 referees. Who cares that you occasionally reject the papers of a Nobel Laureate or your top Ten cited PIs ever? The question is whether on the whole, statistically, considering your entire publication approach...is there an edge? 10%? 20%? 50% improvement in acceptance rate? Personally I doubt they even bother to look at their data and they sure haven't presented even a taste of a real analysis here...

Myths about journals will continue to proliferate.

Yes they will. Including the myths about the "best" science perpetuated in your own interest. As with all myths, there is often a lot of truth behind them. It can take more than a trite denial of the "nuh-uh" variety to deflate the ones that are untrue and actively harmful. I appreciate they feel a low cost, low effort martyred denial helps with the spin game. It sure doesn't get down into the truth of the matter though.

Since I'm all about productive suggestions, let me end by reacting to the final statement:

We can only attempt to ensure that the processes characterized above remain as robust and objective as possible, in our perpetual quest to deliver to our readers the best science that we can muster.

Objective? Okay, I can help out there.

A, Numero Uno- stop with the interpersonal interactions about manuscripts. No informal email or phone inquiries about editorial interest from the authors and no soliciting papers on the part of editors. No more conference schmoozing or spending time in labs. No arguing on the phone or in email for reconsideration after rejection. Everything done by the book of the submission process which is open to all.

Second- look at the reviewer data. How many reviews are coming from the same lab group or reviewer? In mixed-review conflict situations, whose opinion coincides with the editorial decision? How often is it the junior, unknown, third world institution scientist out of these fabled 5,400 referees and how many times is it the "distinguished or hot scientist" that sways the editor?

Third- get real about subtle misconduct timelines and stop pretending you don't know anything about it. You *know* when you've sent something to Group A to review, they sit on it, ask for more experiments and surprise, surprise, have a hurried draft to you juuuuuuust in time for you to decide co-publish. Or even if this stuff goes on between GlamourMagz...would it really kill you to pick up the phone and call your counterpart once a raging author has leveled all sorts of accusations? Wouldn't you all want to tamp down that whole rush-to-first competitive stuff? Since, you know, you are after the best possible science and not just trying to beat your GlamourMag competitors to the get.

Fourth- you know, maybe it IS time to get real about blinded review. I tend to be skeptical because I think the radical restructuring of the typical scientific paper that would be required for blinded review, i.e., totally de-citing it, would break something important. That link to other scientific work. But I thought to my self, "Self, you know the manuscript submitted to a GlamourMag already doesn't look very close to the published article (in startling contrast to real journals), does it? So heck, just go ahead and de-cite that puppy for the review and we can put that back in at some later stage, after the heavy lifting of review is complete".

Let me end on one final counter-spin.

The supposed "myths" of the process of getting published in a GlamourMag journal could be just sour grapes from those authors who have been rejected or from those subdisciplines that will never publish in such journals. Could be. I think that is what this Nature editorial would like you to believe and to infer that these "myths" are totally unfounded. But what the spin meisters at GlamourMagz fail to remember is that science can be a very small and very well connected business at times. Laboratories that publish frequently in their pages tend to be very large, meaning a whole lot of people are right there at the primary point of interaction. People who have friends, neighbors, relatives and the like who are also in the sciences. Lots of trainees who go on to other labs. Or even jobs as Glamour Mag editors. Not all of these latter drink the KoolAde. Some of them tell tales out of school.

Most of these people don't have any particular reason to lie or even embellish all that much to their friends. OTOH, whomever is writing an official Editorial offering from one of the GlamourMagz does have a clear interest in.....well, let us just leave it at spin. So I'm going to need something a little better than your tone of self-righteous woundedness to get on your side, 'k?
__
* Because you use negotiation with ISI over what counts as citable matter and what doesn't instead!

Doctoral Student Training: The Hazing Process Qualification Exam

Sat, 20 Feb 2010 10:50:58 -0500

One contentious part of graduate student training in the US system is the mechanism by which we evaluate student potential for continuing on to the dissertation about halfway along.

Oral or written "qualification" exams. Closed book, open book or take home. Research or grant proposals. Review articles. On specific subfield topics or designed to cover the breath of the fields defined by the Department itself.

It is almost always a constant argument within the faculty, within the student body and between students and faculty about the "best" way to construct the process. I recall that over the course of my doctoral stint, my training department had three distinct qualification processes in place!

As a student I was a big fan of the breadth exam, either written as closed-book or as oral exam. My rationale was basically what I saw as the continued "value" of me getting a doctorate from the department in question. It was a matter of the reputation gained by other grads from the program conversing with scientists across the field. I wanted them to come across as informed as possible, to as many discussants as possible.

Maturing through the career arc, I care less for this. Mostly because I've come to realize nobody that is judging me now gives a rat's patootie what University or Department of -ology appears on my doctorate. They care about the papers I have published. Period. Full freaking stop.

So if I were dictating a graduate program, I'd be looking to enhance the ability of the students to publish papers. This would pretty much rule out the examination approach.

My younger self would be absolutely appalled.