ERIC Number: EJ1251540
Record Type: Journal
Publication Date: 2020-Apr
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0958-2029
EISSN: N/A
Like-for-Like Bibliometric Substitutes for Peer Review: Advantages and Limits of Indicators Calculated from the e[subscript p] Index
RodrÃguez-Navarro, Alonso; Brito, Ricardo
Research Evaluation, v29 n2 p215-230 Apr 2020
The use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In this study, we use the REF results to investigate whether the e[subscript p] index and a top percentile of most cited papers could substitute for peer review. The probability that a random university's paper reaches a certain top percentile in the global distribution of papers is a power of the e[subscript p] index, which can be calculated from the citation-based distribution of university's papers in global top percentiles. Making use of the e[subscript p] index in each university and research area, we calculated the ratios between the percentage of 4-star-rated outputs in REF and the percentages of papers in global top percentiles. Then, we fixed the assessment percentile so that the mean ratio between these two indicators across universities is 1.0. This method was applied to four units of assessment in REF: Chemistry, Economics and Econometrics joined to Business and Management Studies, and Physics. Some relevant deviations from the 1.0 ratio could be explained by the evaluation procedure in REF or by the characteristics of the research field; other deviations need specific studies by experts in the research area. These results indicate that in many research areas the substitution of a top percentile indicator for peer review is possible. However, this substitution cannot be made straightforwardly; more research is needed to establish the conditions of the bibliometric assessment.
Descriptors: Bibliometrics, Peer Evaluation, Foreign Countries, Universities, Citation Analysis, Research Reports
Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: jnls.cust.serv@oxfordjournals.org; Web site: http://applij.oxfordjournals.org/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom
Grant or Contract Numbers: N/A