Top Ten Things NIH Reviewers Should NOT Say In A Review

Credit: Ambro at FreeDigitalPhotos.net

The Center for Scientific Review publishes their Peer Review Notes three times a year, and the most recent issue came out yesterday. The news items are always interesting and it is worth subscribing, if you don’t already. This issue contained an item about things NIH reviewers should not say. I repeat the list in its entirety here—I thought it might be fun for my grantees to see reviewers critiqued for a change.

What do you think of this list? Have you seen one or two of these on your Summary Statements? Me personally? I have seen variations on # 2, 4, and 10 in Summary Statements, and have strongly suspected reviewers of #1 and 5. I almost fell out of my chair laughing when I read # 7, sometimes I think CSR is a little out of touch with what actually happens on Study Sections:

  1. “I didn’t read the application, but I scanned it and saw the applicant said XXX. He doesn’t know what he’s doing.” Damning statements like this can skew a review discussion over something that might be insignificant in the context of the overall application. It’s better for you to ask other reviewers who have read the application carefully what they think about XXX.
  2. “This New Investigator does not appear to be fully independent since he continues to co-publish with his fellowship mentor/department chair, or does not have designated lab space, or has not been promoted in the past several years.”  Academic research organizations have widely diverse policies for faculty advancements and lab space, and many PIs maintain productive and healthy collaborations with mentors for many years after establishing themselves as bona fide investigators. You should focus more on the investigator accomplishments, such as being the first or senior author on a significant publication or giving presentations at major scientific meetings.
  3. “This application is not in my area of expertise . . . “  If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.
  4. “I don’t see this basic science research affecting my clinical practice any time soon.” An application does not necessarily have to show the potential for clinical or timely impact—if the applicant doesn’t make such claims. Basic research often takes time to pay off, and you’re charged to assess the “likelihood for the project to exert a sustained, powerful influence on the research field(s) involved.” Absence of an effect on public health does not necessarily constitute a weakness in basic science.
  5. “I like this project but I’m giving it a poorer score because the applicant has too much money.” Other funding is not a scoreable matter. You should focus on the application’s scientific and technical merit. However, you can note an excessive budget request in the budget section for NIH to consider.
  6. “This application has 2 great aims and 1 bad one. I would recommend deleting Aim 3, and I can give it a 1 or 2.” You cannot trade aims with scores. The application needs to be evaluated as a whole.
  7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.” R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.
  8. “The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.” IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.
  9. “This application was scored a 25 and 14th percentile last time it was reviewed . . . .” You should not mention the previous score an application got, because this could skew the review discussion. Focus on the strengths and weaknesses of the current application as well as the responses to previous critiques.
  10. “This is a fishing expedition.” It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.

Rapid Advances in Ebola Research

The current Ebola outbreak is by far the largest since this hemorrhagic fever was identified in 1976. Previous outbreaks involved dozens or hundreds of infected people (click here for CDC chronology). Estimates of the current outbreak are 2,473 infections and 1350 deaths thus far. Outbreaks begin by transmission through close contact with infected animals, then rapidly spread through human communities via direct contact with bodily fluids of infected people, or through contact with items contaminated with such fluids. Once infected, case fatality is as high as 90% (click here for WHO fact sheet). There are currently no vaccines, treatments, or cures. Traditionally, outbreaks have been controlled largely by infection control measures (masks, gloves, etc.) and quarantine, and supportive care such as hydration of the infected patient.

 

Experimental Treatments: A promising drug called ZMAPP was given at Emory University to two missionaries who were infected with Ebola. Both have gotten better. The drug was also given to a Spanish priest who died soon thereafter, though the timing of drug delivery may have played a part in the drug’s efficacy in this case. As of this week, it appears to be helping three Liberian health care workers. The drug is manufactured by Mapp Biopharmaceutical Inc. It is not FDA approved at present, nor can this monoclonal antibody be produced quickly in large quantities. Other drugs are in development but have yet to show as much promise as ZMAPP. Ebola is a rare disease and affects poor countries almost exclusively, so limited funding is provided mostly by government agencies (see $28 million consortium led by Scripps and funded by NIH, and the recent $10.8 million initiative announced by Wellcome Trust and the United Kingdom’s Department of International Development.) I generally distrust .com coverage of anything related to medicine (and so should you), but this recent CNN piece on ZMAPP seems reasonable, if you would like more information.

 

Cause of the Current Outbreak: NIH announced this morning that researchers funded by NIH have used advanced genomic analysis to determine the single point of infection from an animal that led to the current outbreak, and that since that initial infection the spread has been solely human to human. Importantly, through their genetic analysis, the researchers can see how the virus has mutated since December to outsmart human immune systems. As we know, viruses are little more than tiny pieces of DNA that can mutate with diabolical speed to outsmart the comparatively slow human immune response. By understanding how infection occurs, how disease is spread, and how viruses are mutating to defy immune attack, these researchers have taken a giant step toward improved treatments and a cure. The team was led by Pardis Sabeti, MD, PhD (who not surprisingly won a highly prestigious NIH Director’s New Innovator award in 2009.)

 

Experimental Vaccines: Next week, NIAID will begin the first of several phase I clinical trials of an Ebola vaccine produced in collaboration with GlaxoSmithKline (for details, click here). They will also test an Ebola vaccine developed by the Public Health Agency of Canada and licensed to NewLink Genetics Corp. NIH will partner with a British-based international consortium to test volunteers in the UK, and in the West African countries of Gambia (with approval of local authorities) and Mali. The CDC is in discussion with Nigerian officials about testing vaccines there.

NIGMS To Dramatically Limit Funding for Scientific Meetings

In a December blog post, NIGMS states it will “accept very few R13 or U13 applications in the future. We strongly encourage potential applicants to contact us before requesting approval to submit an application.” Associate Director for Extramural Activities at NIGMS, Ann Hagan PhD, explains further:

“We understand the importance of these meetings, but we receive a large number of requests to support them and have concluded that it is not cost-effective to consider most of these requests. The numerous applications for small conference grants are costly to process and review, and the funds used for them compete directly with research project grants, including R01s. Our priority is to use our resources in the ways that most directly promote research and training, which already include mechanisms to allow students and fellows to attend scientific meetings.”

For the full blog post, click here. Note the comments below the post from the President of Cold Spring Harbor Lab and the Chair of an upcoming Gordon Research Conference, as well as Dr. Hagan’s reply about the use of the R25 mechanism to fund courses and training workshops.