Uncertainty and the ranking of economics journals
- PDF / 1,082,892 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 2 Downloads / 206 Views
Uncertainty and the ranking of economics journals Johan Lyhagen1 · Per Ahlgren1 Received: 14 May 2020 © The Author(s) 2020
Abstract Journal rankings often show significant changes compared to previous rankings. This gives rise to the question of how well estimated the rank of a journal is. In this contribution, we consider uncertainty in a ranking of economics journals. We use the invariant method of Pinski and Narin to rank the journals. We propose an uncertainty measure, which is based on a bootstrap approach. The measure is the average absolute change in rank, which we see as a reasonable uncertainty measure regarding rankings. We further calculate, based on the bootstrap method, 95% confidence interval for the observed values of the invariant method. We show that ranks of the highest, as well as the lowest, ranked journals are well estimated, while there is a high degree of uncertainty regarding the rank of many midranked journals. The distribution of the underlying measure is useful for identifying groups of journals that are more or less of the same quality (from the point of view of the invariant measure). The journal with the highest observed value of the invariant measure, Journal of Political Economy, has the best performance and constitutes a singleton, whereas Quarterly Journal of Economics and Econometrica form the next group (there is a slight overlap between the two with respect to confidence intervals). The journals ranked between about 190–230 form another group in which there are no major quality differences between the journals, as the confidence intervals are overlapping. Keywords Bootstrapping · Economic journals · Invariant method · Ranking · Uncertainty
Introduction Nowadays, there is no lack of rankings of entities related to research. Examples of such entities are scientific journals and universities. For the latter, there are many ways to rank them. Academic Ranking of World Universities (more known as the Shanghai ranking), QS World University Ranking and THE World University Rankings are the most well-known university rankings. These rankings differ in indicators used and the weighting of the indicators, and their results have been shown to be only moderately correlated (Olcay and Bulu * Johan Lyhagen [email protected] Per Ahlgren [email protected] 1
Department of Statistics, Uppsala University, Uppsala, Sweden
13
Vol.:(0123456789)
Scientometrics
2017; Mussard and James 2018). CWTS Leiden Ranking, which uses an advanced bibliometric methodology and which do not rely on reputation measurement, avoids a composite indicator of university performance, and this constitutes a fundamental difference between this ranking and the three more well-known ones. University rankings have been exposed for various criticism, though. Regarding Academic Ranking of World Universities, Billaut et al. (2010) concluded that all criteria used by the designers of the ranking are only loosely connected with what they intended to capture. High volatility from year to year has been
Data Loading...
