Presentation at the 41st EAIR Forum. Leiden, the Netherlands, August 26, 2019.

1. New developments in the CWTS Leiden Ranking
Ludo Waltman
Centre for Science and Technology Studies, Leiden University
41st EAIR Forum
Leiden, the Netherlands
August 26, 2019

2. 1

3. Overview
• CWTS Leiden Ranking
• Indicators of gender balance
• Indicators of open access publishing
2

4. CWTS Leiden Ranking
3

5. Differences with other university rankings
• Focused on research, not on teaching
• Based purely on bibliometric indicators; no survey data or data provided by
universities
• Advanced bibliometric methodology
• No composite indicators
• Multiple views, not just a simple ranked list
4

6. Indicators
• Size-dependent and size-independent indicators
• Output:
– P
5
• Scientific impact:
– P(top 1%) and PP(top 1%)
– P(top 5%) and PP(top 5%)
– P(top 10%) and PP(top 10%)
– P(top 50%) and PP(top 50%)
• Collaboration:
– P(collab) and PP(collab)
– P(int collab) and PP(int collab)
– P(industry) and PP(industry)
– P(<100 km) and PP(<100 km)
– P(>5000 km) and PP(>5000 km)

7. 6

8. Visitors of the Leiden Ranking
7

9. Ranking universities responsibly
8
Source: www.researchresearch.com/news/article/?articleId=1368350
Source: https://vimeo.com/279712695

10. Ten rules for ranking universities
1. One size doesn’t fit all
2. Separate the relative from
the absolute
3. Be explicit about the
definition of a university
4. Be transparent
5. Compare and contrast
6. Acknowledge uncertainty
7. Look at the underlying data
8. Not everything that counts
can be counted
9. Know your level
10. Handle with care, but don’t
discard
9

11. One size doesn’t fit all
10

12. Indicators of gender
balance
Joint work with Hanjo Boekhout and Nees Jan van Eck
11

13. Author disambiguation
• Determine which publications
belong to the same author
• Author names often are not unique
and are used inconsistently
• ORCID identifiers have not yet been
widely adopted
• CWTS has developed its own author
disambiguation algorithm for Web of
Science
12

14. Indicators of gender balance
• Based on disambiguated authors
• Each author’s country of origin is determined
• Gender of an author is determined based on first name and country of origin,
using Gender API, Gender Guesser, and genderize.io
• Gender is assigned only if confidence level is above 90%
• For a significant share of all authors, in particular authors with Asian names,
no gender can be determined
13

15. Size-dependent gender indicators
• A Number of authorships
• A(unknown) Number of authorships with unknown gender
• A(M) Number of male authorships
• A(F) Number of female authorships
• A(MF) Number of male and female authorships
14

16. Size-independent gender indicators
• PA(M) Male authorships as a proportion of all authorships
• PA(F) Female authorships as a proportion of all authorships
• PA(M|MF) Male authorships as a proportion of all male and female
authorships
• PA(F|MF) Female authorships as a proportion of all male and female
authorships
15

17. PA(F|MF) for universities in the Leiden Ranking: Time
trends in different geographical regions
16

18. An indicator to assess gender bias in universities?
17

19. UK universities
18

20. Eastern European universities
19

21. Indicators of open
access publishing
Joint work with Rodrigo Costas and Thed van Leeuwen
20

22. Different types of OA
• Gold OA: Publications in an OA journal
• Hybrid OA: Publications openly available in a subscription journal, with a
license
• Bronze OA: Publications openly available in a subscription journal, without a
license
• Green OA: Publications that have appeared in a journal and of which the final
peer-reviewed version has been (self-)archived in an OA repository
• An article can be both green OA and gold, hybrid, or bronze OA
21

23. Determining OA status of publications based on
Unpaywall data
22

24. Proportion of OA publications per university
23

25. Bilkent University
24

26. London School of Hygiene & Tropical Medicine
25

27. Conclusion
26

28. Conclusion
• Bibliometric indicators are limited in the type of information they can provide
• However, in addition to traditional output, scientific impact, and collaboration
indicators, the Leiden Ranking now also provides gender and open access
indicators
What would you like to see next in the Leiden Ranking?
27

29. Thank you for your attention!
28