Assessment of gender bias in grant peer review throughout the application and the evaluation phases
We examine the application and review materials of a large, multi-disciplinary, pan-European research funding scheme: EUROpean COllaborative RESearch (EUROCORES) Scheme (2001-2010). EUROCORES promoted investigator-driven, multinational collaborative research in multiple scientific areas and brought together international research teams. <br />We expect that GIGA will provide empirical support to understand the gender biais and to funding organizations and policymakers to design policies aimed at reducing gender bias in the allocation of funds.
The GIGA project exploits micro-level data originating from the 2001-2010 EUROCORES grant scheme covering 10533 grantees across all scientific domains, 1642 outline proposals, 886 full proposals along with 2246 assigned external reviewers and 829 panel members under 47 programmes. The quality of the original version of the data is not sufficient to perform any robust statistical analysis. In this reporting period, the GIGA project team has engaged in an in-depth data cleaning and harmonization process that is composed of manual cleaning of tables (e.g. missing demographic and proposal information, misspelled entries, etc.) and adding publicly available publication info via external databases (Web of Science). The following steps has been taken in order to ensure integrity, validity and accuracy of the data:
(i) Data cleaning for Principal Investigators (PI) and Project Leaders (PL): personal information (academic status, age, gender, affiliation) is manually re-imputed for about 50% of the original sample, corresponding to about 5000 individuals. Information is retrieved from the web.
(ii) Data cleaning for Reviewers and Panel Members: same as for point (i), in this case we count about 2200 individuals.
(iii) From printed form to electronic version: a large proportion of the information about reviews (i.e., the evaluation scores and texts of EUROCORES programme from 2001 to 2005) are stored in .pdf format, corresponding to about 1000 single documents. These documents need to be converted in electronic format (e.g. excel) and harmonized with the existing dataset.
(iv) Publication data for PIs and PLs: the scientific outcomes of about 10500 individuals are retrieved from dedicated archives (Web of Science).
(v) Publication data for Reviewers and Panel Members: same as for point (iv), in this case we count about 3000 individuals.
(vi) All data are combined and re-formatted for econometric and neuro-linguistic analysis.
(vii) A thorough literature review is conducted.
The main results at this stage are publish in Bianchini, Stefano;Llerena, Patrick; Ocalan-Ozel, Sila ; Ozel, Emre (2022) Gender diversity of research consortium contributes to funding decisions in a multi-stage grant peer-review process, in Nature - Humanities and Social Sciences Communications, 9/1, 1-10. www.nature.com/articles/s41599-022-01204-6
«We found systematically unfavourable evaluations for consortia with a higher proportion of female PIs. This gender effect was evident in the evaluation outcomes of both panel members and reviewers: applications from consortia with a higher share of female scientists were less successful in panel selection and received lower scores from external reviewers. Interestingly, we found a systematic discrepancy between the evaluative language of written review reports and the scores assigned by reviewers that works against consortia with a higher share of female participants. Reviewers did not perceive female scientists as being less competent in their comments, but they were negatively sensitive to a high female ratio within a consortium when scoring the proposed research project.«
Several works are in progress:
- a paper on a conceptual framework for the analysis of gender bias
- an analysis of gender biases when setting up/composing teams responding to calls for projects.
In addition, a workshop is planned for March 2023 to finalize the project.
1. Bianchini, Stefano;Llerena, Patrick; Ocalan-Ozel, Sila ; Ozel, Emre (2022) Gender diversity of research consortium contributes to funding decisions in a multi-stage grant peer-review process, in Nature - Humanities and Social Sciences Communications, 9/1, 1-10. www.nature.com/articles/s41599-022-01204-6
2. Ocalan-Ozel, Sila; Llerena Patrick (2021) Industry Collaborations of Research Teams: Are They Penalized or Rewarded in the Grant Evaluation Process? in Frontiers in Research Metrics and Analytics, 6. www.frontiersin.org/articles/10.3389/frma.2021.707278/full
3. Corsini, A., Pezzoni, M., Visentin, F. (2022) What makes a productive Ph.D. student? Research Policy 51, 104561. https ://doi.org/10.1016/j.respol.2022.104561
4. Ayoubi, C., Pezzoni, M., Visentin, F. (2021) Does It Pay to Do Novel Science? The Selectivity Patterns in Science Funding. Science and Public Policy scab031. doi.org/10.1093/scipol/scab031
Gender equity is one of the determinants of economic growth, social development and well-being. It was the case for the economic development during the era of industrial development; it is probably, even more, the case in the knowledge economy, where scientific advance is a key element of human progress.
Despite the general agreement on gender equity benefits, women have been found to have lower salaries than men, to encounter greater difficulties in obtaining career promotions and in accessing resources. While women are increasingly participating in science, they still have difficulties in succeeding and the scientific community is debating about the reasons behind the phenomenon.
The overarching objective of GIGA is to assess gender bias in research funding. Research grants are critical resources that can affect researchers’ scientific productivity and career progress. Nowadays, the ability to raise funds is becoming a base in the evaluation of scientists’ performances along with publication record. In this context, access to grants is crucial for any researcher.
In recent years, funding agencies and policymakers have attempted to develop guidelines to promote female scientists’ participation in competitions for research funding. The rationale of these efforts is the concern that women have a lower propensity to apply for grants and, eventually, to access financial resources. On the one hand, a series of contextual factors could explain differences in the likelihood to apply, such as family burden, the presence of female role-models, and diverse networks. On the other hand, the causes of female under-representation among the applicants for grants may be traced back to the grant allocation process, that is, the selection process carried out by evaluators.
GIGA aims to explore the funding process throughout all its phases: the application phase as well as the awarding phase. In the application phase researchers self-select themselves to participate in the competition. In the awarding phase, a committee of experts select the applications that deserve funding support.
This study aims to acquire novel insights on the impact of gender bias on research funding mechanisms at both applicant and evaluator (referee and panel) level, and to explore the role played by confounding factors (i.e. career, scientific productivity, network and novelty) in the allocation of grants. To this end, GIGA puts forward an analysis through both dimensions: the application process and the selection process of grant applications.
We exploit a unique large database from the European Science Foundation (ESF) covering information on a large funding scheme over the period 2003-2014, namely the EUROpean COllaborative RESearch Scheme (EUROCORES). The scheme encouraged researchers to network and collaborate across Europe and a number of other countries to promote interdisciplinary basic research activities and to disseminate the results, thereby opening new horizons in science. As a ‘bottom-up’ support to collaborative research projects, EUROCORES scheme represents a crucial opportunity to reveal ‘intrinsic’ biases in the allocation of grants in many aspects, including gender biases. The database includes about 1600 projects, more then 10.000 applicants, about 2400 external reviewers and more then 300 panel members.
We will exploit the information both quantitatively (by using relevant econometrical methods) and qualitatively (via interviews and survey) . We expect that GIGA will influence the current research on gender by proposing a holistic view that goes beyond the simple identification of gender bias in the allocation of grants.
This project will provide empirical support to funding organizations and policymakers to design policies aimed at reducing gender bias in the allocation of funds and increasing the number of women participating in the competition for grant attribution.
Monsieur Patrick Llerena (Bureau d'économie théorique et appliquée (UMR 7522))
The author of this summary is the project coordinator, who is responsible for the content of this summary. The ANR declines any responsibility as for its contents.
BETA _ UNISTRA Bureau d'économie théorique et appliquée (UMR 7522)
UNS - GREDEG Université NIce Sophia Antipolis - Groupe de Recherche en Droit, Economie et Gestion
Help of the ANR 313,200 euros
Beginning and duration of the scientific project: September 2019 - 36 Months