About NIH

Frequently asked questions on NIH-commissioned study on race, ethnicity and NIH research awards

Ginther, DK, Schaffer, WT, Schnell, J, Masimore, B, Liu, F, Haak, LL, Kington, R. Race, Ethnicity, and NIH Research Awards. Science. (333)1015-1019.

What is this paper about?
As part of a larger effort to examine and improve the diversity of its funded biomedical research workforce, NIH commissioned a study in 2008 to determine if researchers of different races and ethnicities with similar research records and affiliations had similar likelihoods of being awarded a new (Type 1) R01 grant, which is one type of NIH research project grant. This study focused solely on race and ethnicity, since there are few studies on the racial and ethnic composition of federal research funding applicants. The study found that black applicants from 2000-2006 were 10 percentage points less likely than white applicants to be awarded research project grants from the National Institutes of Health after controlling for factors that influence the likelihood of a grant award. Although Asian applicants also were less likely to receive an award than white applicants, those differences disappeared when the sample was limited to U.S. citizens. The award probability for Hispanic applicants did not differ significantly from that of whites. The study controlled for education, citizenship, country of origin, training, employer characteristics, prior research awards, and publication record.

What methods were used in this paper?
The authors used multiple regression techniques to estimate the correlation between particular variables, organized into different models, and the probability that a grant was scored or an award was received. Regression studies can demonstrate a relationship between a variable and an outcome within the sample, but do not demonstrate causality. The study examined the probability of these outcomes for Type 1 R01 research project grant applications received during the period between fiscal years 2000 and 2006. Applicants’ race and ethnicity were voluntarily disclosed by the applicants themselves as part of the application process.

Which data were used in this paper?
The authors used data from the NIH IMPAC II grant database combined with data from the National Science Foundation, the Association of American Medical Colleges, Thomson Reuters Web of Science, and other sources. The de-identified dataset from the analysis is available at: http://report.nih.gov/investigators_and_trainees/index.aspx

What accounts for the difference in award rates?
Further research is needed to identify the causes. A few possible causes that are being considered for further examination are:

Quality of educational and mentoring experiences may differ for applicants of different races and ethnicities.
There is a possibility of conscious or unconscious bias in the grants process. Although information on race and ethnicity of the applicant is not available to reviewers, an applicant’s name or institutional affiliation included in the application biography could be suggestive of their race or ethnicity.

Does citizenship make a difference?
When the analysis was restricted to U.S. citizens, the difference in award probability between Asian-Americans and whites was no longer statistically significant. The difference in award probability between black and white applicants did not change.

Does prior NIH training support make a difference?
Prior NIH training improved R01 award probability within each racial and ethnic group when the sample was limited to U.S. citizens and permanent residents. However, NIH training programs did not reduce the differences in award probability between black, Hispanic, Asian, and white applicants. According to the authors, this suggests that current NIH graduate and postdoctoral training programs do not narrow race and ethnic group differences in award probability.

Does affiliation with a research-intensive organization make a difference?
Yes. Affiliation with a research-intensive organization improves award probability, and this did not vary within racial and ethnic groups. The differences in award probability between racial and ethnic groups, however, were not decreased.

Are there differences in persistence across race and ethnic groups in revising and resubmitting unfunded applications?
The authors found that blacks, Hispanics, and Asians were less likely than whites to resubmit unfunded applications.

Can an applicant continue to resubmit an application multiple times if he or she has not received an award?
During the study period (fiscal years 2000 to 2006), NIH allowed applicants to revise and resubmit an application twice. In the 2009 fiscal year, to accelerate the eventual funding of meritorious science, the NIH adopted a single revision and resubmission policy.

Is the probability of receiving a numerical priority score related to the race or ethnicity of the applicant?
Even though the application does not contain information on the race or ethnicity of the applicant, there is a relationship between race, scoring and funding. The proportion of applications receiving a score is lower for those submitted by self-identified black or African-American and Asian investigators than from white investigators. Because the study does not identify causality, NIH will conduct further research to identify potential causes, including the possibility of bias.

Note: The NIH uses a streamlining procedure to focus discussion at review meetings on the more meritorious applications that are likely to be considered for funding. By consensus of the entire review group, an application that is considered by the review panel to be less meritorious than others undergoing review may be streamlined and not discussed further, rather than receiving full discussion and a numerical score.

Which other factors improve the probability of an application not being streamlined?
According to the study, having one or more publications for which the investigator is the last (senior) author, NIH review committee service, prior NIH research awards, and highly cited publications were all associated with an increased probability of an application being discussed and not being streamlined.

Do NIH reviewers have access to race/ethnicity/gender information on grant applicants?
No. Information on race, gender, and ethnicity is collected from applicants on a voluntary basis and maintained in a confidential electronic data system called the NIH Commons, which is not associated with grant applications in any way. It is possible, however, that reviewers can infer this kind of demographic information from the applicant’s name or institutional affiliation included in the applicant’s biography.

Do these findings have implications for the NIH review process?
The approaches used in this paper do not permit conclusions about causal factors; however, in the absence of explanatory variables, inferences about the quality of applications, the effect of the research field on the success of the application, and the objectivity of the review process need to be explored. NIH is implementing actions to identify and remedy factors that may contribute to the gap.

Will the NIH further examine the grants process?
Yes. The NIH intends to examine carefully each step of the process from application preparation to review and funding decisions to see whether conscious or unconscious factors influence the outcome. The NIH relies on the integrity of the NIH peer review process to identify the most meritorious extramural research projects.

What does NIH intend to do to close the gap in R01 grant success rates?
NIH has developed and is implementing a framework for action to:

Increase the number of early career reviewers including those from underrepresented populations. Consistent with the study’s finding that service on NIH’s grant review study sections correlates with success in grant applications, NIH has initiated the new Early Career Reviewer program to encourage promising junior faculty to participate in peer review panels, and learn how these groups discuss, evaluate, and score grant applications. The goal is to develop their skills as reviewers and contribute to the overall effectiveness of peer review. NIH is making a special effort to inform investigators from underrepresented groups about this program. It recently requested nominations of faculty or researchers, specifically including those at less research-intensive institutions who are actively involved and established in biomedical research and have published in high-quality scientific journals, but who may not yet have received major peer-reviewed research support. The aim is to have 50 early career reviewers assigned to each of NIH’s three rounds of grant review in the 2012 fiscal year.
Examine the grant review process for bias and develop interventions. NIH will conduct innovative experiments to shed light on possible sources of bias in the grant review process, and to develop appropriate interventions. Possibilities include removal of all identifying information from applications, testing reviewer ability to determine applicant race, assessing different types and timing of training against bias, and assessing whether the proportion of minority reviewers on a peer review panel affects outcome for minority applicants.
Improve support for grant applicants. NIH will assess the value of providing additional technical assistance to applicants in grant preparation, and supporting innovative approaches to encourage more extensive and effective local mentoring of junior faculty.
Gather expert advice on additional action steps. NIH will gather expert advice through two high-level advisory groups that have been formed by the NIH Director: the NIH Diversity Task Force (part of the NIH Director’s Steering Committee) and an external panel of distinguished experts on a newly formed Diversity in Biomedical Research Working Group reporting to Dr. Collins through the Advisory Committee to the Director. These two groups will address how NIH can better articulate its interest in diversity, better identify actions NIH can take to achieve its stated goals of recruiting the best and brightest minds to the biomedical research workforce, and sustain an environment that nurtures and encourages diversity.

Is other NIH-funded research taking place to develop and assess the effectiveness of interventions to promote diversity in the biomedical research workforce?
Yes, the National Institute of General Medical Sciences has issued a series of RFAs that solicit studies on interventions related to diversity. The most recent version of this announcement is RFA-GM-12-002 and is called “Research to Understand and Inform Interventions that Promote the Research Careers of Students in Biomedical and Behavioral Sciences (R01).” The following studies have been funded under this program:

Sylvia Hurtado, Ph.D. University of California, Los Angeles. Promoting Diversity: Access and Engagement in Biomedical and Behavioral Research
Kenneth Maton, Ph.D., University of Maryland, Baltimore County. Analyzing the Impact of the Meyerhoff Scholars Program: Pathways to the STEM Ph.D.
Shine Chang, Ph.D., University of Texas MD Anderson Cancer Center. Improving Retention of Minority Trainees: Mentoring in Scientific Communications
Mary (Molly) Games, M.D., of the University of Wisconsin-Madison will develop an interactive tool that will help faculty recognize and self-correct implicit, stereotype-based bias that affects the participation and advancement of groups underrepresented in science, technology, engineering, mathematics and medicine.
Bradley S. Duerstock, Ph.D., of Purdue University, West Lafayette, Ind., will create an accessible wet laboratory for practical training as well as a Web-based interactive community for individuals with disabilities pursuing biomedical research careers.
Vivian Lewis, M.D., of the University of Rochester, N.Y., will test the hypothesis that mentoring interventions will promote the resilience of biomedical researchers from underrepresented groups, resulting in greater career satisfaction, confidence and academic success.
Richard McGee, Ph.D., of Northwestern University Feinberg School of Medicine, Chicago, will test a model employing coaching to complement what scientific mentors typically provide as an approach for improving the professional advancement of students from underrepresented groups toward academic research careers.
Joan Reede, M.D., M.P.H., of Harvard Medical School, Boston, will conduct a study of the institutional and environmental factors that impede and/or support the careers of clinical and research faculty from diverse groups, as well as how these factors impact an individual's career-related networks.
Hannah A, Valantine, M.D., of Stanford University, Calif., will test the hypothesis that mitigating the experience of stereotype threat will improve the advancement and retention of female faculty members.