Everyone knows that only the highest scoring grant proposals get awarded. For this, you have to comprehend its scoring process. There are multiple scoring systems worldwide, but the most frequently used system is of the National Institutes of Health (NIH). It has a certain priority scoring system to judge the applications. Each criterion has its marks. This article will especially focus on the NIH grants scoring rules. You can further use this piece of information while scripting and reviewing your grant proposal.
Why is the Scoring Necessary?
A few decades ago, either experienced investigators or those with high references were able to get the grants. It was a difficult time for the new impactful investigators to find their place in the field of research. Hence, many questions started to arise regarding this issue. The authority members then proposed that having a specific scoring process will highlight the applications having more impact on science. The target was also to introduce a meritorious sorting system applicable to all the research applications. Hence, the concept of a scoring system hatched that created possibilities for inexperienced but innovative scientists.
Preliminary Scoring Systems
All the applications are assembled and scrutinized for all the necessary documentation. These applications are then assigned to their specific reviewers. These reviewers will badge your application with two types of independent scoring systems. These are:
Pre-session Criterion Scoring
Usually, scores are given based on five criteria before the peer review session. Each criterion is given a separate numerical score in the range of 1 to 9, with 1 being exceptional and 9 being poor. Every reviewer has his unique perception of scoring an application using the set criteria (Elizabeth et al., 2017). These criteria are different for independent research (R) grants and the training (F & K) grants. These are discussed in detail below.
Independent Researcher Grant Criteria Scoring
Following are the criteria for the scoring of an independent researcher’s grant application:
- Significance: The reviewers read the significance section of every application and rate it according to their given scale from 1 to 9. This criterion shows the importance of your proposed researched. The greater the impact of your research on the improvement of science and healthcare system, the better the score you will get. The focus of your research on your respective funding opportunity announcement is also observed here while scoring. Hence, the significance section of your application holds the key to impress your reviewer in the first criterion.
- Experimental Approach: Here, your application is judged for the quality of the experimental approach proposed. Your reviewer will search whether your research study is feasible enough to achieve the aims that you are claiming. You will be scored for the durability and authenticity of your study methods. The level of organization in your study design is deeply scrutinized. Answers like how will your study techniques contribute to achieving your project’s objectives. Often, this criterion is given special importance by the reviewer. It is discussed in detail in the peer review meeting session, as well.
- Research Innovation: This criterion is made to assess the degree of newness in your proposed research. Here, a reviewer tries to find the answers to several questions. How is your research different from past researches? How can your research help in coining a new way of scientific thought process? What are the features that make your research a feasible one? How much authentic is your novel proposal? How is your claimed innovative research justified by you? You will be awarded an excellent score if all these questions are greatly handled by you in your application.
- Project Researcher: For this criterion, the reviewer searches through the bio-sketches of you and your research team. He seeks whether you have any prior experiences in the field of research or not. Your capability as an independent researcher is questioned at this point. Hence, you have to justify your qualifications and how are you a suitable candidate to conduct this proposed research. The same goes for all the research teams. They are also expected to provide solid reasons for their roles in your team. You will be given an excellent score here only if your reviewer is convinced enough by your bio-sketches.
- Research Environment: For this criterion, your application is judged for the resources and facilities that are available to you in your research environment. The tools, equipment, and other infrastructure provision are examined here. Your research environment is inspected for the capability to carry out the claimed research experiments. For this, you have to provide a letter of support attributing your ability in conducting independent research and offering you research support by providing all the necessary facilities to aid in your research.
Training Grant Criteria Scoring
Training grants include the F and K-type grant applications. Following are the criteria for the scoring of training grant applications:
- Applicant: Unlike the researcher criterion above, this criterion emphasizes more on the potential for advancement in the applicant rather than his previous experiences. Here, your recommendation letters are highly preferred for scoring. After the consultation from your provided bio-sketches, the reviewer moves onto the recommendation letters where your capabilities are defined by the experienced professionals. Each word is analyzed, and an image regarding your learning potential is made in the reviewer’s mind. Your scoring is then prescribed according to this derived image.
- Environment and Mentors: This scoring criterion especially focuses on the sponsors and the environment they are providing for the training. All the information regarding your mentors is analyzed. For this, your mentor’s previous records, achievements, and experiences with the trainees should be provided in detail. Mentors with the previous record of funding from NIH impress the reviewers (Karina, Thomas, Arleen, Judy, Joann & Ira, 2007). A judgment is then made whether the facilities in the environment provided will be able to produce a positive result to become a brilliant investigator. Only an optimally productive environment will be able to score best in this criterion.
- Research Proposal: This criterion focuses on the experimental approach with a slight emphasis on the novelty and importance of your research. The reason behind this change in focus is that the training grant investigator is not an established researcher. Here, the priority is to observe the researcher from the angle of his research methodological approach. The ability to choose a study design and to follow it until the outcomes are the main idea of the training grant. Hence, your central focus should be on this section if you want to score high.
- Training Capability: This criterion is based on the evaluation of your training regimen as a whole. An overall assessment is made on the procedures for better training from your sponsored mentors. The feasibility of your training activities regarding your development as an independent researcher is questioned again. Another important role is played by the mentors in the provision of formal exercises set for you to improve your research skills. These exercises can be in the form of lectures, seminars, and laboratory demonstrations. All such activities should be provided in your application to be scored for this criterion.
Pre-session Impact Scoring
The second system of scoring is the impact scoring system. In this, a general evaluation of the grant application is done. This system depicts the overall impact of the proposed research applications in the innovation and advancement of science. The impact score is independent of the criterion score given to it. The biohazards of your research are also observed here. The scale for scoring is from 1 to 9, with 1 being exceptional and 9 being poor. These scores vary according to the application characteristics (Matthew, Robin, Deepshikha, Katherine & Katrina, 2016). Hence, the reviewers are specially instructed to utilize a full scale for impact scoring so that greater judgment should be made between the applicants.
Peer Review Meeting
A peer review meeting is held after the pre-scoring of applications by their assigned reviewers. Many applicants may develop mistrust concerning the scoring process (Hendy et al., 2012). This meeting is held to ensure transparency. Here, every reviewer puts forward his entrusted applications and explains how he gave the scoring. The secondary reviewers then come forward and give remarks about the scoring of the applications. Hence, you can also call it a debate session. A reviewer may alter his given score if the reviewer has a change in opinion about any application’s criterion score after the peer review session. Therefore, the final criterion and impact scores are pinned after the meeting ends.
Priority Scoring, the Final Judgment
The mean of overall marks of the application is multiplied by 100 (Michael et al., 2019). This is the mean score derived from the numbers given to your application individually by all the members of the peer review meeting. Its scale ranges from 100 to 500, with 100 being best and 500 being the worst for an application. This scoring goes for both research and training grants. The priority scoring is the final score of your application. After this, each application is prioritized accordingly for funding according to the merit score.
The term un-scored is used when your application has a priority score equal to or greater than 250. These applications are also called streamlined applications. They are set aside to save time in the meeting. The reviewers then correspond to their applicants with the necessary comments and criticisms. They are given a chance to revise their applications according to their shortcomings.
Conflict of Interest
Several applications will have a conflict with the reviewers, which may lead to bias. These conflicts can be in the form of any direct or indirect relationship between the applicant and the reviewer. To avoid this bias, the reviewers with conflicts are not chosen for the peer review meeting panel.
It is of utter importance to know the scoring process before writing any grant application. In this way, you will be able to focus on the points that are relevant and necessary for the reviewers to score your application. The panel set for the peer review meeting is chosen by the NIH itself. These are experienced individuals in their respected fields. They use a set guideline for scoring and prioritizing the submitted grant proposals. Following these rules is mandatory for all the peer reviewers as the future of many great researchers lies on their shoulders. Although NIH has an excellent scoring procedure but making sure that the reviewers adhere to these rules is also the responsibility of the NIH that should be evaluated from time to time.
- Brauer, M., Carnes, M., Ford, C.E., Kaatz, A., Nathan, M.J., Pier, E.L. & Raclaw, J. (2017). ‘Your Comments are Meaner than Your Score’: Score Calibration Talk Influences Intra- and Inter-panel Variability during Scientific Grant Peer Review. Research Evaluation, 26(1), 1–14. https://doi.org/10.1093/reseval/rvw025
- Berg, K.M., Brown, A.F., Elmore, J.G., Gill, T.M., Wilson, I.B. & Zerzan, J. (2007, November). Demystifying the NIH Grant Application Process. Journal of General Internal Medicine, 22(11), 1587–1595. https://doi.org/10.1007/s11606-007-0301-6
- Eblen, M.K., Patel, K.C., Pearson, K., RoyChowdhury, D. & Wagner, R.M. (2016, June). How Criterion Scores Predict the Overall Impact Score and Funding Outcomes for National Institutes of Health Peer-Reviewed Applications. Plos One, 11(6), https://doi.org/10.1371/journal.pone.0155060
- Abdoul, H., Alberti, C., Amiel, P., Durand-Zaleski, I., Gottot, S., Perrey, C. & Tubach, F. (2012, September). Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices. Plos One, 7(9), https://doi.org/10.1371/journal.pone.0046054
- Burkhardt, J., Carlson, J. N., Gottlieb, M., King, A. M., Lee, S., Wong, A. H., & Santen, S. A. (2019, January). Show Me the Money: Successfully Obtaining Grant Funding in Medical Education. The Western Journal of Mmergency Medicine, 20(1), 71–77. https://doi.org/10.5811/westjem.2018.10.41269