The Center for Scientific Review Board Makes Grant Review Suggestions to NIH

The NIH Center for Scientific Review (CSR) publishes Peer Review Notes to inform reviewers, NIH staff, and others interested in news related to grant application review policies, procedures, and plans. The latest issue of Peer Review Notes was sent out last night (they publish three times per year). Here are items that caught my attention:

  • Become more scientific in assessing approaches to improve the efficiency and particularly the quality of NIH peer review.
  • Work hard to understand and address possible disparities in NIH awards.
  • Collaborate with the NIH and scientific communities to identify critical problems, such as the definition of a “new” application, and to develop solutions.
  • Help the public understand the role of NIH peer review in advancing science and health in the United States.

I certainly wanted more clarity on those first three bullets, some of which I found in another article in this issue of Notes, which I have copied below (my comments appear after each numbered suggestion):
CSR’s Council Suggests Five Ways NIH Can Help Applicants CSR’s Advisory Council recently asked NIH to consider five ideas for helping applicants with promising research ideas to stay in the game despite historically low funding rates. Because these ideas deal with trans-NIH policies beyond CSR, Council members asked CSR’s Director to share them with the appropriate NIH officials.

CSR Council Ideas

1. Treat all applications as new and let investigators instead of NIH decide when resubmission is futile. Council members suggested that the resulting reviews would be more independent and simplified since earlier reviews would not be considered. Reviewers might also be more focused on merit because they wouldn’t get sidetracked by considering how investigators responded to previous reviews. Our Council suggested doing a pilot where investigators who opt-in could resubmit any R01 application as many times as they wanted, but they could submit no more than two research project grant applications in any 12 month period. Reviewers would be encouraged to send strong messages about applications that need substantial revision.

          Meg’s comment: I suspect PIs would love to submit an R01 application as many times as they like, though some folks would balk at being limited to two RPGs per 12-month period (I assume they mean any RPG at any IC). Many PIs I know would appreciate a clear message from reviewers about whether they should resubmit. The grant score alone does not always help them decide, as I have seen applications go unscored because the reviewers wanted an entire aim added or taken away, but they were very favorable about the rest of the application (in this instance, clearly the team should resubmit even though the A0 is unscored.)
_________________________
2. Encourage more NIH Institutes and Centers (ICs) to allow investigators to respond to their reviews prior to Council consideration so very promising applications that might slip through the system could be identified. Principal investigators (PIs) with “gray zone” applications would be asked to provide a response to their reviews. IC Program staff would submit these comments and applications to their Councils, which provide the second level of peer review.
          Meg’s comment: Again, I can imagine that most PIs would welcome the opportunity to speak persuasively about their project if they score near the funding line, though this strategy adds to the workload for both the PI and PO, which is something NIH has been trying to avoid.
________________________
3. Enhance communications with PIs: Study sections and NIH program staff should do better at communicating to PIs about applications that are unlikely to be successful or, alternatively, are of potential interest.  [See our last PRN newsletter: Make the Best Use of the “Additional Comments to Applicant” Box]
          Meg’s comment: My clients and I spend plenty of time trying to read between the lines of pink sheets to figure out if the reviewers would welcome a resubmission. It can be exceedingly difficult for a PI to read his/her pink sheets, let alone accurately assess the subtleties of the comments. On the one hand, a clear message from the reviewers about resubmission would be welcome. On the other hand, if the PI feels he can address the problems, or feels the review was less-than-fair and s/he would like to wait it out and resubmit to the same study section after there has been some turnover, they should have the opportunity to do so regardless of what the study section said in the pink sheets. Note that this CSR Council recommendation seems to contradict their first recommendation above: “…let investigators instead of NIH decide when resubmission is futile.”
______________________
4. Encourage NIH ICs to take full advantage of the R56 funding mechanism to provide bridge funding to promising investigators. These “High Priority, Short-Term Project Awards” provide 1 year funding for high-priority new or competing renewal R01 applications that score just outside an ICs funding limits.
          Meg’s comment: The little-known R56 funding mechanism is not one for which a PI can apply directly, it is awarded to a PI with a promising application to another grant mechanism. The award is made at the discretion of the Program Officer, which is one of the very many reasons I am a huge advocate of the PI cultivating a relationship with the PO. Note also that I had a client receive an R56 even though his original application was nowhere near the funding line. He had a great relationship with his PO, and the PO believed in him and his work.
___________________________
5. Provide longer-term funding for some PIs: For investigators with large and successful programs, NIH should consider offering funding for a longer duration but at a lower overall amount. The savings would be used to fund more applications. Restrictions on participating PIs would be necessary to ensure that the result would be revenue-positive.
          Meg’s comment: This is such a mixed bag I don’t even know where to begin.
____________________________
What do you think of the CSR Advisory Board recommendations to NIH?

Reviewers of NIH Grant Submissions May Pay A Heavy Price When Their Own Submissions Are Reviewed


Grant scores are being posted right now. I was at a policy meeting last week with one of my grant clients and we waited with baited breath to see her scores. Although she has had a great deal of success on past NIH submissions, she was worried about this one, and with good reason. She recently agreed to serve as a reviewer on an NIH study section, but quickly realized that it meant that the resubmission of her own R01 competing renewal could no longer be reviewed in the most appropriate study section. Instead, it was assigned to a Special Emphasis Panel, or SEP. My client studied the list of reviewers on the SEP and learned that there was no one on the panel with the expertise needed to review her submission. She informed her SRO of this problem, but was told to sit tight and wait and see how the review went.

My client is considered an eminence in her field. She chairs a nationally-ranked academic department in her area, has landed numerous R01s, a U01, ARRA funding, and more. On her R01 competing renewal she scored in the 20th percentile, but the problems identified by the reviewers were certainly fixable and she resubmitted with hope, if not confidence. Then she accepted the position on the study section and the resubmission went to the (underqualified) SEP. The results of the resubmission were posted: Unscored.

It is hardly the first time I have heard such a story, it is just the most recent. It is considered an honor to be invited to serve on a study section. Many, like my client, choose to do so despite the time and effort involved because they feel they should give back for the many years of funding they have received from NIH. But the inadvertent result may be that the reviewer’s own submissions must be reviewed elsewhere, often on a panel that lacks the necessary expertise. As a result, my client and others have decided to terminate their service on study sections, which has the result of denying other NIH grantees reviews from those deemed the most qualified to provide them.

The current policy creates a situation where reviewers of a study section may be unable to get a fair review of their own grant applications at the most appropriate study section, thereby essentially penalizing them for service to NIH.

When discussing this problem last week in Washington with an NIH program officer, I was told to contact my local congressman. The PO felt that if we wanted to effect change to this NIH policy, the only approach was to inform a congressman that his constituents were not receiving millions in federal funding due to this policy. I plan to do so, and urge you all to do the same.

Learn About The NIH Application Review Process


Most grant applications and contract proposals to NIH are reviewed in the Center for Scientific Review. The CSR puts out a newsletter periodically with updates and tidbits for NIH applicants. It is well worth subscribing to this newsletter, as well as spending some time perusing past issues. Here is the link to the latest edition:

Latest Issue of the Peer Review Notes

http://internet.csr.nih.gov/WebAnalytics/QRRedirect.aspx?param=PRNSep12Listserv

*   How Well Is NIH Identifying and Advancing Innovative Research?

*   Former Study Section Chairs Share Advice for New Reviewers

*   Make the Best Use of the “Additional Comments to Applicant” Box

*   Who Are the Other People in Your Review Meetings?

*   CSR Reviews for the NIH-FDA Collaboration on Tobacco Control Regulatory Research