NatureJobs reports on the contentious topic of possible gender bias in peer review (Nature 459, 602; 2009). Peer review assesses what is of value in science, yet it has been criticized for biases. One such perceived bias is gender, although evidence for such a bias has been contradictory. A 2007 meta-analysis (L. Bornmann et al. J. Informet. 1, 226–238; 2007; reported at the time in NatureJobs Nature 445, 566; 2007) concluded that women are at a disadvantage in peer review of funding applications. As this study incorporated all known research on this issue, it was suggested to be definitive.
Nevertheless, H. W. Marsh et al. (Am. Psychol. 63, 160–168; 2008) presented conflicting results the following year. This study was comprehensive, based on data from the Australian Research Council (10,023 reviews by 6,233 external assessors of 2,331 proposals from all disciplines), and concluded that the gender of the applicant had no effect on the outcomes of peer review, irrespective of the discipline, gender and nationality of the reviewers, and whether reviewers were selected by a funding panel or chosen by the applicants.
Why should these two studies have conflicting results? To investigate, Marsh and Bornmann now report in NatureJobs how both research teams worked together to reanalyse the data and extend the original meta-analysis. They describe how they applied new, stronger statistical approaches to 66 sets of results representing 353,725 proposals from 8 countries. In this extended study, which will be published in Review of Educational Research, they found no effect of the applicant’s gender on the peer review of their grant proposals. This lack of effect held across country, year of publication of the studies included in the meta-analysis, and disciplines ranging from physical sciences to the humanities.
At least for grant applications, all of the co-authors from each of the research teams agree that the weight of evidence suggests that the applicant’s gender has no effect on the outcome of peer review, and that these findings are robust and broadly generalizable.
Herbert Marsh is a professor of education at the University of Oxford, UK
Lutz Bornmann is a PhD student at the ETH University in Zurich, Switzerland.
This Peer-to-Peer post is an edited version of their NatureJobs article (which is free to access online).
There is some discussion of this NatureJobs article and the timing of its publication at Nautilus blog.
Note: readers interested in the Review of Educational Research paper in advance of its publication can obtain a copy directly from Dr Bornmann or from the NatureJobs editors.