The End of Two Strikes You’re Out: Good or Bad News for Grant Applicants?

Yesterday, NIH and AHRQ announced that they had changed its resubmission policy. While a given application is still allowed only one resubmission, if you are unsuccessful on the A1, you can then submit the application again as an A0 without having to substantially redesign the content and scope of the project. This “new submission” will be reviewed without any association to the previous attempts. There will be no Introduction to the Revised Application, no explanation of how you addressed the reviewers’ concerns. Even if reviewers have seen the application in prior review cycles, they will be instructed to review it as new. (However, reviewers are human, and one wonders if this is realistic. It’s like asking a juror to ignore evidence they just heard, and we know from social psychologists that that doesn’t actually work.) Of course, the idea is that an applicant will use previous reviewer comments to strengthen the application, thereby improving their odds of funding.

Click here to read the full notice

In 2009, as part of their Enhancing Peer Review project, NIH eliminated the A2 in what has been dubbed by unhappy researchers as the “Two Strikes You’re Out” policy. NIH eliminated the A2 because meritorious research was most likely funded on the A2, which meant quite a delay to funding. The resubmission policy did indeed result in an increase in the number of awards made on A0 applications (although most funded applications are successful on the A1). About the “Two Strikes You’re Out” policy, Deputy Director Dr. Sally Rockey states: “…we heard increasing concerns from the community about the impact of the policy on new investigators because finding new research directions can be quite difficult during this phase of their career. Also, established investigators voiced concern about the need to redirect the research focus of productive labs in order to submit future NIH applications.” For one of many discussions on Dr. Rockey’s blog about the decision to sunset the A2 submission, click here.

In theory, it would appear that one could submit the same idea endlessly until one finds the best way to sell it to reviewers. The policy will no doubt appeal to many researchers, because it is now up to a researcher to decide when it is time to abandon a given project. If one waits long enough, one could submit to the same study section after most of the members have turned over, thereby having a new set of reviewers to weigh in on the project. One risk I see is that sometimes a PI is so blindly enamored with their idea that they have difficulty hearing that the reviewers are trying to tell them that the idea is simply not fundable in any form; i.e., no amount of tinkering with the writing or the details will fix it. A PI could waste a lot of time if they cannot see when it is time to abandon an unfundable project.

Judging from the comments on some of the NIH blogs (for examples, click here), many reviewers are thrilled to have more submission attempts, while others are dismayed that this marks a return to a huge number of submissions and long delays to funding. What do you think?

Will NIH Eliminate The Unpopular “Two Strikes You’re Out” Grant Review Policy?

In January 2009, NIH instituted a policy to sunset A2 applications, i.e. to eliminate a second resubmission. Since, investigators submit a grant application (A0) and if the application is unfunded they may submit one revised application (A1), and no more (previously, one could resubmit twice). This policy has been detested among research scientists since its inception and NIH has heard an earful of complaints about it. NIH spoke out recently on this topic. Will they consider eliminating the wildly unpopular policy? The short answer: Not bloody likely.

In a recent blog post, Sally Rockey gives some data NIH has compiled on this policy. One of the goals of the policy was to reduce the time to award, and in fact the time from A0 submission to award has been reduced from 93 weeks to 56 weeks by this policy. It is conventional wisdom that NIH grant applications, if funded, tend to be successful on their final submission, with very few succeeding as an A0. Yet surprisingly, since the policy went into effect, the proportion of funded A0 applications as compared to A1 apps has increased.

One major complaint of the policy is that it favors established investigators over new investigators. The data do not appear to support this idea, because the time-to-award for new applicants is not appreciably longer than for the entire population.

In response to the suggestion that NIH allow A2 applications only from those investigators whose apps fall just outside the payline, NIH modeled the potential impact of such a policy using FY11 data. From their model, NIH concluded that the result would be to shift awards from A0 and A1 apps toward A2 apps, which was what the policy was designed to avoid to begin with (see original blog post for details of their method.)

Rockey concludes: “Overall, these data indicate that the policy to sunset A2 applications continues to achieve the stated goals of enabling NIH to fund as much meritorious science as possible in as short a time period as possible. Any revision to the policy to allow additional resubmissions of all or a subset of A2 applications will displace equally meritorious A0 and A1 applications, and increase the time to award for many applications. For these reasons, we have decided to continue the policy in its current form.”

Sorry folks, it’s here to stay.