This document discusses contextualized scientometrics and outlines an agenda for the scientometric community. It begins by showing bibliometric indicators for top Chinese universities and explores what factors influence these numbers through techniques like identifying research areas and topics in citation networks. It argues that scientometrics should focus on contextualization over representation by making diverse statistics and information available rather than basing decisions solely on indicators. Open data sources like the Initiative for Open Citations and Dimensions platform can help accomplish this by providing open metadata. The agenda calls on scientometricians to promote responsible use of indicators, support open data initiatives, use open sources in research, and support open metadata as authors and reviewers.

1. Contextualized scientometrics:
What’s behind the numbers?
Ludo Waltman
Centre for Science and Technology Studies (CWTS), Leiden University
International Forum on Quantitative Studies of Science and Technology
Zhejiang University, Hangzhou, China
April 14, 2019

2. Outline
• Bibliometric indicators for Chinese universities
• What’s behind the numbers?
• From representation to contextualization
• Data sources for contextualized scientometrics
• An agenda for the scientometric community
1

3. Bibliometric indicators
for Chinese
universities
2

4. CWTS Leiden Ranking
• Bibliometric ranking of major
universities worldwide
• Advanced methodology based
on most recent insights from
bibliometric research
• www.leidenranking.com
3
Data source: Clarivate Analytics Web of Science

5. Chinese universities
4
Fuzhou Univ.
Hong Kong Univ.
Science and Tech.
City Univ.
Hong Kong
Zhejiang Univ.
Shanghai Jiao
Tong Univ.
Tsinghua Univ.
Peking Univ.
Fudan Univ.

6. Chinese universities vs. US universities
5
Rockefeller Univ.
Harvard Univ.
Johns Hopkins
Univ.
Univ. Michigan
Stanford Univ.MIT

7. Chinese universities vs. Dutch universities
6
Leiden Univ.

8. What’s behind the
numbers?
7

9. Leiden algorithm
8

10. Identifying research areas in citation networks
9

11. VOSviewer
10

12. Micro-level research areas
11
Social sciences
Biomed.
& health sciences
Math. & comp.
science
Physical
sciences & eng.
Life & earth sciences

13. Zhejiang University’s research strengths
12
Share of an area’s
highly cited
publications
authored by Zhejiang
University

14. Zhejiang University’s research strengths
(social sciences)
13
Share of an area’s
highly cited
publications
authored by
Zhejiang University
bibliometrics
China demography
housing
market

15. Area 1127 (China): Topics
14

16. Area 1127 (China): Focus of Zhejiang University
15

17. Area 145 (bibliometrics): Topics
16

18. Area 145 (bibliometrics): Focus of Zhejiang University
17

19. From representation to
contextualization
18

20. Representation vs. contextualization
Representation
1. Concepts of interest (e.g., production and impact) are specified
2. Scientometric indicators (e.g., publication and citation counts) are defined
to represent, or operationalize, the concepts
3. Decision making is based on, or supported by, the indicators
Contextualization
1. Scientometric information is made available in a variety of ways:
Statistics, distributions, network diagrams, publication lists, etc.
2. Interpretation is done by decision makers, not by scientometricians
3. Scientometric information provides context for decision making 19

21. Unlike representation, contextualization is difficult to
accomplish using closed data sources
20

22. Data sources for
contextualized
scientometrics
21

23. Initiative for Open Citations (I4OC)
22
https://i4oc.org/

24. Initiative for Open Citations (I4OC)
23
https://i4oc.org/

25. Support for I4OC
24

26. Large publishers not supporting I4OC
25

27. Scientometricians supporting open citations
26
www.issi-society.org/open-citations-letter/

28. Scientometricians supporting open citations
27
www.issi-society.org/open-citations-letter/
April 26, 2018:
• 324 signatories
• 46 countries

29. Scientometricians supporting open citations
28
www.issi-society.org/blog/posts/2018/april/open-citations-to-open-science/

30. Journal of Informetrics ‘flip’
29

31. Dimensions: Open data for research purposes
30

32. An agenda for the
scientometric
community
31

33. An agenda for the scientometric community
Scientometric practice
1. Promote responsible use of scientometrics
– Focus on contextualization instead of representation
2. Promote open metadata of scientific outputs
– Support open citations and open metadata initiatives such as I4OC
Scientometric research
3. Whenever possible, use data sources that facilitate reproducibility
– Use data sources to which other researchers also have access (e.g., Dimensions)
4. Support open metadata as author, reviewer, or editor
– Support the new Quantitative Science Studies journal!
32

34. Quantitative Science Studies
33

35. Thank you for your attention!
34