Unfortunate arbitrariness at NSF costs our graduate students

Last week, a student in our lab had an NSF DDIG (Doctoral Dissertation Improvement Grant) returned without review. The reason is the proposal did not fit the goals of the panel, according to the program officer. Unfortunately, the PO thought the proposal fit more squarely in a different panel, which does not accept DDIG proposals. To me, this points to a very unfortunate difficulty at NSF, which is that in the IOS (Integrative Organismal Systems) division, only one panel - Animal Behavior - accepts DDIG proposals. This puts the Animal Behavior Program Officers in a difficult position: They are forced to define discrete boundaries to a scientific field. While all fields are a continuum, defining discrete fields within the division of "Integrative" biology seems particularly arbitrary because the very definition of the division is to use multidisciplinary perspectives.

This week, I learned through social media that the students of multiple colleagues experienced the same rejection without review. I know how hard students work on these proposals, and to have them returned without review is a disservice to the students who are the future of our disciplines. On one hand, it reinforces a mantra that NSF promotes - "always contact your program officer". Yes, I certainly should have checked that we were within the scope of the panel. That said, my excuse is that I really didn't imagine, as I will describe briefly below, that our proposal on the evolutionary and genetic basis of behavior would not be considered animal behavior. But this points to the challenge - the current structure forces an arbitrarily discrete definition of a field. According to the call "In the Division of Integrative Organismal Systems (IOS) only proposals within the scientific area of animal behavior supported by the Behavioral Systems Cluster are eligible."

What is (Animal) Behavior?

Since I am a newcomer to behavior (through training in evolution and phylogenetics), I've actually been wondering a lot about the question of what is behavior.  What about bacteria that follow chemical cues? Is that behavior? Do plants or slime molds behave? Viruses? I think they do. The NSF panel has the word "animal" in it, which would rule out these questions (even that seems arbitrary to me). But still there are ambiguities. Do light-following, swimming sponge larvae that lack nervous systems behave? What about annelid worm larvae or box jelly larvae that have ciliated cells that respond to light? In these animals, a single cell acts as a sensor (light) and a motor (cilium) to cause swimming. Is that behavior, or is that physiology? To ask interesting questions about these systems, we don't need to classify it into a field. But to get a DDIG proposal funded, a student is restricted in the questions she can ask.

My student's proposal walks the line between behavior and physiology, a little bit. But the student and I both feel very strongly that we are studying behavior (even animal behavior, since these are cephalopods). Here are the first few lines from the project description:

"Cephalopods dazzle prey, woo mates and seamlessly blend into the background using skin that changes in both color and texture (Hanlon and Messenger, 1998). These diverse behaviors in cephalopods and countless others found throughout animals inspire the question of how new behaviors evolve to produce the riotous variety we see today (Tinbergen, 1963). Do novel behaviors stem from evolved mechanisms? Or do new behaviors primarily arise through evolutionary “tinkering”, which may co-opt, retool and recombine existing mechanisms and modules? Answering these questions is fundamental to understanding how behaviors evolve."

Our goal really is to understand how behaviors evolve, especially at the genetic level. Yet the program officers write:

"... As written, this proposal does not address questions or theory in the field of Animal Behavior. Thus it is not appropriate for this competition and is being returned without review."

and yet, here is how the NSF Behavioral Systems Cluster describes its goals (my emphasis):

The Behavioral Systems Cluster supports research on the development, function, mechanisms, and evolutionary history of behavior, with emphasis on a vertically integrated understanding of the behavioral phenotype in nature. To foster this integrative goal, the Cluster specifically encourages projects that seek to understand how combinations of neural, hormonal, physiological, and developmental mechanisms act synergistically as a system from which behavior emerges.  Laboratory work or the study of animals in captivity is encouraged, to the extent that it contributes to the understanding of behavior in natural systems.

I still feel our proposal addresses questions in the field of Animal Behavior. Yet, I recognize the root of the problem is that only one area in IOS accepts DDIG proposals. This forces a line to be drawn somewhere, and even if our proposal is animal behavior, there must be discrete decisions about where animal behavior ends that will affect someone.

A broader view

National Academy Member, David Hillis of the University of Texas, astutely pointed out that the current difficulty points to a fundamental decision about funding many small grants or funding fewer larger grants:

"The problem, as I understand it, is that only DEB and part of IOS make the case that DDIGs are worthwhile. To me, they are perhaps the best use of NSF funds, and result in the highest return for the buck of any federal funding. But the science communities (outside of DEB) need to make this case to NSF, if they agree. There is much more of a tradition of independent research by graduate students in the fields represented by DEB, compared to many other areas. I've heard that some other programs resist DDIGs because they disperse a small amount of money per proposal, so they have a relatively high administrative cost per dollar of research funds dispersed, and graduate students are much less likely to be doing truly independent research outside of DEB. But I think any increase in admin costs is more than made up by the high return in each dollar used, which I think is far more efficient for DDIGs than most other programs. The trend at NSF is actually in the opposite direction: funding of very large, multi-institutional grants and centers. I think these probably result in the least efficient use of research dollars (certainly far less efficient than DDIGs, in my experience). But that balance may differ among fields, which is why DEB makes the argument for DDIGs, but most other programs don't. "

I wholeheartedly agree with those comments. Supporting DDIG proposals is a very wise investment, not only to fund up-and-coming researchers, but to provide a platform to learn how to write grants, to learn about the NSF funding system, and to promote early independence in scientists. As such, I very much advocate expanding the DDIG call to all of IOS.

Action

So, what can I/we do besides complain about the situation on an obscure blog? I propose to organize anonymous peer reviews for those students whose DDIG proposals were returned without review. For the students in this situation to whom I've spoken, this is their biggest regret: They worked hard and cannot even get feedback. I propose to facilitate obtaining that feedback. Hey, maybe we could get really creative and start a Kickstarter (or similar) program to *actually fund* some or all of the top reviewed proposal. Here is what I propose:

1. Any student whose DDIG proposal was rejected without review can send their proposal to me by email on or before December 2. I imagine they can just download a pdf from fastlane and send it along to me. I will treat the proposal as confidential, just as I would if I reviewed it.
2. I will act as a sort of 'program officer' and solicit 2 or 3 anonymous reviews per proposal. I will ask reviewers to follow precisely NSF reviewer guidelines and make sure they keep the grants confidential, just as if NSF asked them to review. The reviewers will be anonymous to the student.
3. I will summarize the reviews and send them back to the student. The goal is for them to get feedback about their research ideas from anonymous reviewers.

In addition to providing the students with feedback, and an idea of how their proposal might have reviewed, another goal is to raise awareness of what I see to be unfortunate arbitrariness that is costing our graduate students.

If you are willing to help, here is what you can do:

1. Volunteer to serve as an anonymous reviewer of proposals. Just send me an email: oakley@lifesci.ucsb.edu and tell me you are willing
2. Spread the word, so that affected students can hear about this and get their proposals reviewed, if they like.
3. Contact NSF and tell them you support expanding DDIG to all of IOS.
4. Send any additional ideas or concerns to me by email or as comments on this blog post

Thank you for listening!

Todd Oakley, Professor, UCSB