Are Italian research assessment exercises size-biased?
- PDF / 1,012,296 Bytes
- 17 Pages / 439.37 x 666.142 pts Page_size
- 17 Downloads / 178 Views
Are Italian research assessment exercises size‑biased? Camil Demetrescu1 · Andrea Ribichini1 · Marco Schaerf1 Received: 10 February 2020 / Published online: 5 August 2020 © The Author(s) 2020
Abstract Research assessment exercises have enjoyed ever-increasing popularity in many countries in recent years, both as a method to guide public funds allocation and as a validation tool for adopted research support policies. Italy’s most recently completed evaluation effort (VQR 2011–14) required each university to submit to the Ministry for Education, University, and Research (MIUR) 2 research products per author (3 in the case of other research institutions), chosen in such a way that the same product is not assigned to two authors belonging to the same institution. This constraint suggests that larger institutions, where collaborations among colleagues may be more frequent, could suffer a size-related bias in their evaluation scores. To validate our claim, we investigate the outcome of artificially splitting Sapienza University of Rome, one of the largest universities in Europe, in a number of separate partitions, according to several criteria, noting significant score increases for several partitioning scenarios. Keywords Research assessment · Bibliometrics · National evaluations · Graph partitioning
Introduction Research assessment exercises have been adopted by an increasing number of countries in recent years. Their objectives include guiding public funding of research institutions, stimulating improvement through competition and assessing the effectiveness of adopted support policies (Abramo and D’Angelo 2015). Research assessments are conducted through a variety of methodologies, and techniques used in a given country may even vary from one iteration to the next, based on experience, theoretical advancements, availability of resources, and policy aims (Abramo et al. 2011).
* Marco Schaerf [email protected] Camil Demetrescu [email protected] Andrea Ribichini [email protected] 1
Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Via Ariosto 25, 00185 Rome, Italy
13
Vol.:(0123456789)
534
Scientometrics (2020) 125:533–549
Italy’s most recently completed exercise, namely VQR (Research Quality eValuation) 2011–2014, was based on a hybrid approach (i.e., bibliometric indicators for hard sciences and peer review for social sciences and humanities) and examined a relatively small selection of research products (2 per researcher for universities, and 3 per researcher for other institutions, chosen in such a way that no two researchers belonging to the same institution could be assigned the same research product). We refer the reader to “The Italian research evaluation exercises” section for more details. The constraint that researchers from the same institution are not allowed to select the same research products for evaluation might penalize larger institutions, where collaborations among colleagues may be more frequent. While mos
Data Loading...
