Presentation at the IREG 2019 Conference. Bologna, Italy, May 10, 2019.

1. Ranking universities responsibly
Ludo Waltman
Centre for Science and Technology Studies (CWTS), Leiden University
IREG 2019
Bologna, Italy
May 10, 2019

2. CWTS Leiden Ranking and U-Multirank
1

3. Principles for
responsible university
ranking
2

4. Social responsibility of ranking and ranked
organizations
3
Source: www.researchresearch.com/news/article/?articleId=1368350
Source: https://vimeo.com/279712695

5. Ten rules for ranking universities
1. One size doesn’t fit all
2. Separate the relative from
the absolute
3. Be explicit about the
definition of a university
4. Be transparent
5. Compare and contrast
6. Acknowledge uncertainty
7. Look at the underlying data
8. Not everything that counts
can be counted
9. Know your level
10. Handle with care, but don’t
discard
4

6. One size doesn’t fit all
5

7. Separate the relative from the absolute
What is the world’s wealthiest country?
6
GDP per capita GDP

8. Be transparent
7Weak correlation between citation scores
in Leiden Ranking and Times Higher Education ranking

9. Be transparent
8

10. CWTS Leiden Ranking
9

11. 10

12. Open access and gender
11

13. 2019 edition of the Leiden Ranking
• Scientific impact (citations)
• Collaboration (co-authored publications)
• NEW: Open access (gold, hybrid, bronze, and green OA)
• NEW: Gender (male and female authors)
To be released next Wednesday (May 15, 13h CEST)
at www.leidenranking.com
12

14. Percentage of open access publications
of University of Bologna
13

15. Percentage of open access publications per university
140% 20% 40% 60% 80% 100%
PP(OA) (2006-2009)
0%
20%
40%
60%
80%
100%
PP(OA)(2014-2017)

16. Percentage of female authorships in publications of
University of Bologna
15

17. Percentage of female authorships per university
160% 10% 20% 30% 40% 50% 60%
PA(F|MF) (2006-2009)
0%
10%
20%
30%
40%
50%
60%
PA(F|MF)(2014-2017)

18. To conclude
• Ranking and ranked organizations need to take their social responsibility
more seriously
• Universities can do more to encourage rankers to behave in a responsible
way
• CWTS Leiden Ranking, although limited in scope, aims to show how ranking
can be done responsibly
• New indicators in CWTS Leiden Ranking 2019 demonstrate the broader
value of bibliometrics, beyond just measuring ‘research excellence’
17

19. Thank you for your attention!
18