The document summarizes Ludo Waltman's presentation on responsible university ranking at the 1st Moscow International Symposium in Scientometrics, Higher Education and Law Research. It discusses CWTS, Leiden University's work in developing quantitative analyses of science, including the Leiden Ranking of universities and the Leiden Manifesto providing principles for responsible use of metrics. The presentation emphasizes the need for transparent, multi-dimensional rankings that consider different university missions and account for uncertainty. It also introduces the new Quantitative Science Studies journal.
1. Ranking universities responsibly
Ludo Waltman
Centre for Science and Technology Studies (CWTS), Leiden University
1st Moscow International Symposium in Scientometrics, Higher Education and Law Research
RUDN University, Moscow
November 20, 2019
2. Outline
• Centre for Science and Technology Studies (CWTS), Leiden University
• Ranking universities responsibly
• Quantitative Science Studies (QSS) journal
1
4. Centre for Science and Technology Studies (CWTS)
• Research center in science and technology studies
• Located at Leiden University in the Netherlands
• Our mission: Making the science system better!
3
8. Leiden Manifesto
• Leiden Manifesto provides 10
principles for responsible use of
scientometrics in research
assessments
• Used by research institutions to
develop internal policies for use
of scientometrics
7
9. More diverse journal indicators
• Call for more diverse journal
indicators covering the different
functions of journals
8
13. Landscape of science
12
Social sciences
and humanities
Biomedical and
health sciences
Physical
sciences and
engineering
Mathematics and
computer science
Life and earth
sciences
14. Research on research in the landscape of science
13
Social sciences
and humanities
Biomedical and
health sciences
Mathematics and
computer sciencePhysical
sciences
Life and earth
sciences
scientometrics
science and
technology
studies
innovation
studies
science of
science
higher
education
data sharing
metaresearch
(research quality)
metaresearch
(team science and
research funding)
health
policy
metaresearch
(reproducibility)
metaresearch
(reproducibility)
18. Ten rules for ranking universities
17
Source: www.researchresearch.com/news/article/?articleId=1368350
Source: https://vimeo.com/279712695
19. Ten rules for ranking universities
1. One size doesn’t fit all
2. Separate the relative from
the absolute
3. Be explicit about the
definition of a university
4. Be transparent
5. Compare and contrast
6. Acknowledge uncertainty
7. Look at the underlying data
8. Not everything that counts
can be counted
9. Know your level
10. Handle with care, but don’t
discard
18
26. Quantitative Science Studies (QSS)
• Official journal of the International Society for
Scientometrics and Informetrics
• Published by MIT Press
• Launched in early 2019
• Managed by the former editorial board of Journal of Informetrics (JOI)
• Openness and transparency:
– Open access (APC of USD 600/800; covered by a subsidy from TIB in Germany)
– Open data (whenever possible)
– Open metadata (available in Crossref)
– Reporting of competing interests
– APC transparency
– Peer review transparency (pilot project in 2020) 25