The Center for Scientific Review Board Makes Grant Review Suggestions to NIH

The NIH Center for Scientific Review (CSR) publishes Peer Review Notes to inform reviewers, NIH staff, and others interested in news related to grant application review policies, procedures, and plans. The latest issue of Peer Review Notes was sent out last night (they publish three times per year). Here are items that caught my attention:

  • Become more scientific in assessing approaches to improve the efficiency and particularly the quality of NIH peer review.
  • Work hard to understand and address possible disparities in NIH awards.
  • Collaborate with the NIH and scientific communities to identify critical problems, such as the definition of a “new” application, and to develop solutions.
  • Help the public understand the role of NIH peer review in advancing science and health in the United States.

I certainly wanted more clarity on those first three bullets, some of which I found in another article in this issue of Notes, which I have copied below (my comments appear after each numbered suggestion):
CSR’s Council Suggests Five Ways NIH Can Help Applicants CSR’s Advisory Council recently asked NIH to consider five ideas for helping applicants with promising research ideas to stay in the game despite historically low funding rates. Because these ideas deal with trans-NIH policies beyond CSR, Council members asked CSR’s Director to share them with the appropriate NIH officials.

CSR Council Ideas

1. Treat all applications as new and let investigators instead of NIH decide when resubmission is futile. Council members suggested that the resulting reviews would be more independent and simplified since earlier reviews would not be considered. Reviewers might also be more focused on merit because they wouldn’t get sidetracked by considering how investigators responded to previous reviews. Our Council suggested doing a pilot where investigators who opt-in could resubmit any R01 application as many times as they wanted, but they could submit no more than two research project grant applications in any 12 month period. Reviewers would be encouraged to send strong messages about applications that need substantial revision.

          Meg’s comment: I suspect PIs would love to submit an R01 application as many times as they like, though some folks would balk at being limited to two RPGs per 12-month period (I assume they mean any RPG at any IC). Many PIs I know would appreciate a clear message from reviewers about whether they should resubmit. The grant score alone does not always help them decide, as I have seen applications go unscored because the reviewers wanted an entire aim added or taken away, but they were very favorable about the rest of the application (in this instance, clearly the team should resubmit even though the A0 is unscored.)
_________________________
2. Encourage more NIH Institutes and Centers (ICs) to allow investigators to respond to their reviews prior to Council consideration so very promising applications that might slip through the system could be identified. Principal investigators (PIs) with “gray zone” applications would be asked to provide a response to their reviews. IC Program staff would submit these comments and applications to their Councils, which provide the second level of peer review.
          Meg’s comment: Again, I can imagine that most PIs would welcome the opportunity to speak persuasively about their project if they score near the funding line, though this strategy adds to the workload for both the PI and PO, which is something NIH has been trying to avoid.
________________________
3. Enhance communications with PIs: Study sections and NIH program staff should do better at communicating to PIs about applications that are unlikely to be successful or, alternatively, are of potential interest.  [See our last PRN newsletter: Make the Best Use of the “Additional Comments to Applicant” Box]
          Meg’s comment: My clients and I spend plenty of time trying to read between the lines of pink sheets to figure out if the reviewers would welcome a resubmission. It can be exceedingly difficult for a PI to read his/her pink sheets, let alone accurately assess the subtleties of the comments. On the one hand, a clear message from the reviewers about resubmission would be welcome. On the other hand, if the PI feels he can address the problems, or feels the review was less-than-fair and s/he would like to wait it out and resubmit to the same study section after there has been some turnover, they should have the opportunity to do so regardless of what the study section said in the pink sheets. Note that this CSR Council recommendation seems to contradict their first recommendation above: “…let investigators instead of NIH decide when resubmission is futile.”
______________________
4. Encourage NIH ICs to take full advantage of the R56 funding mechanism to provide bridge funding to promising investigators. These “High Priority, Short-Term Project Awards” provide 1 year funding for high-priority new or competing renewal R01 applications that score just outside an ICs funding limits.
          Meg’s comment: The little-known R56 funding mechanism is not one for which a PI can apply directly, it is awarded to a PI with a promising application to another grant mechanism. The award is made at the discretion of the Program Officer, which is one of the very many reasons I am a huge advocate of the PI cultivating a relationship with the PO. Note also that I had a client receive an R56 even though his original application was nowhere near the funding line. He had a great relationship with his PO, and the PO believed in him and his work.
___________________________
5. Provide longer-term funding for some PIs: For investigators with large and successful programs, NIH should consider offering funding for a longer duration but at a lower overall amount. The savings would be used to fund more applications. Restrictions on participating PIs would be necessary to ensure that the result would be revenue-positive.
          Meg’s comment: This is such a mixed bag I don’t even know where to begin.
____________________________
What do you think of the CSR Advisory Board recommendations to NIH?

The State of Despair Among Many NIH Grantees

I am on a flight returning from a trip to Washington DC. I was visiting friends and former colleagues from my days at NIH. Several of my friends are now running research labs at medical centers, one of whom has served on several study sections. Another former colleague has gone on to become a Program Officer at NIH. One person does public health education at NIH, and another is running a successful freelance business. Each of them was interested in discussing the state of NIH grantwriting, especially given that the preliminary summary statements from the previous grant cycle have just become available on eCommons. We are all disturbed by the pervasive feeling of despair that I hear among NIH grantees.

One concern I hear repeatedly from both grantees and NIH program staff is concern about the quality of the review process on study sections. There are those who feel that regardless of the quality of the proposal, the best scores tend to go to the big-name labs who already are flush with funding. (I have heard it suggested by more than one person that reviews would work better if proposals were submitted anonymously.) Some feel that while New and Early Stage Investigators are given better opportunities to obtain funding, mid-career researchers are left in the cold because of the tendency to dole out money for the big-name labs. Another concern I often hear is that there are “cliques” within a given field, and the power to award great scores rests in the hands of the “in group” in a study section, while those outside the clique remain unscored and unfunded. I sometimes hear grantees and even program staff at NIH complain that the Summary Statements are illogical or contradictory– or worse, unintelligent. (When I see Summary Statements that are illogical or contradictory, often it is because the grant was confusing. Poor writing is not always the cause of such reviewer responses. But you can decrease your odds of a confused or ill-informed reviewer by writing more clearly and concisely.)

Almost certainly, there is some element of truth to each of these concerns. But I hate to see such talk discourage promising researchers from entering or remaining in the field. One could speculate endlessly about how to game the system when it comes to NIH grantsmanship. I think a great deal of such speculation is wasted energy. I hear a lot of stories from people in the field about what has gone wrong with their career, their proposals, the myriad ways in which they have been screwed. Being a proposal writer is a bit like being a bartender at times. And I see grantees making a lot of poor choices that are directly within their control to change. Here are some suggestions based on the mistakes I see:

When it comes to interacting with your colleagues, do your level best not to make enemies. Areas of biomedical research expertise have become so narrow and esoteric that you cannot afford to antagonize anyone in the handful of researchers in your field. That said, given the level of desperation over the current funding climate, you probably also should play your cards close to the vest. Be careful with whom you discuss your ideas. Your draft Aims may be best discussed at departmental chalk talks, where you can elicit great feedback while also divulging your ideas to a larger group who may serve as witnesses later on that the ideas were indeed yours. (Yes, I hear lots of talk of researchers stealing each other’s ideas.) Be assertive. It pays to ask for everything and anything you need, as the worst you will hear is no. I have a client who requested funds for proposal writing support from everyone—her Chair, the Dean, anyone who would listen. She got a little money from each source that, together with a little money from her start-up, helped pay for help on a K01 and a Robert Wood Johnson proposal (she landed both.) Her colleagues have whined about the help she has gotten, and why haven’t they been offered such help? (The answer: They never asked.) More examples: If you have done the work, insist on being first or last author on the manuscript. Conversely, if you are not the PI on a grant, do not do all the work. You will get no recognition. Above all, behave with integrity– even when your colleagues do not.

I have a great deal of respect for researchers who remain in the trenches of biomedical research, continuing to apply for grants even in the current funding climate. Such work is much more difficult than what I do. Increasingly, medical research facilities are shifting toward the elimination of tenure while demanding that their faculty rely 100% on soft money. It is not for the faint of heart.

But if you choose to remain, you must work to develop an extraordinarily thick skin. Proposal writing is an iterative process. With each submission, you use the Summary Statements to hone your grantsmanship. You work to find a great Program Officer in an institute that is a good fit for your work, and then you work with the PO to figure out how to tailor your research to fit the funding priorities and interests of the institute. If you are suspect of the quality of your study section, shift your focus and request a different one. There is little use in dwelling on your fears (real or not) about the inequities and injustices in the review process, at least not while you are putting together a proposal submission. Your energy is best spent on improving your proposal and your grantsmanship on that submission, to the best of your ability.