Can we use altmetrics at the institutional level? A case study analyzing the coverage by research areas of four Spanish universities. The study analyzed altmetric data from Altmetric.com for papers published in 2014 from four Spanish universities. Altmetric.com covered 36% of papers, with significant differences between universities and research fields. While altmetrics can provide institutional benchmarking, coverage is low overall and concentrated in multidisciplinary journals and fields like science and nature. Factors like DOI use, language, and type of research limit altmetrics' ability to represent all areas equally at the institutional level.

1. Can we use
altmetrics
at the institutional level?
A case study analysing the coverage
by research areas of four Spanish universities

2. Can we use altmetrics at
the institutional level? A
case study analysing the
coverage by research
areas of four Spanish
universities
Daniel Torres-Salinas*, Nicolás Robinson-García**
and Evaristo Jiménez-Contreras***
•
* Univ. Navarra and Univ. Granada, EC3metrics & Medialab UGR - @torressalinas
** INGENIO (CSIC-UPV), Universitat Politècnica de València - @nrobinsongarcia
*** Universidad de Granada, Grupo EC3 and EC3metrics
To be published in:
Proceedings of the 21st International Conference
on Science and Technology Indicators
València (Spain) | September 14-16, 2016
Full proceeding paper avaible at arxiv:
https://arxiv.org/abs/1606.00232

3. Can we use altmetrics at institucional level?
Introduction

4. Can we use altmetrics at institucional level?
Introduction: altmetric studies
Social media based indicators or
altmetrics have been under
scrutiny for the last seven years.
1.- The relation between citations and altmetric indicators
2.- Coverage and diversity of social media sources
3.- Their meaning as impact indicators
Three lines of work can be observed

5. Can we use altmetrics at institucional level?
Introduction: a gap in altmetric studies
1. Most studies are focus on the understanding of the
nature and relation of altmetric indicators with citation data
No study has been found
profiling larger units of
research such as institutions,
especially universities
2.- Most papers are analysing altmetric indicators at paper
(paper level metrics) or author level (author level metrics)

6. .
Can we use altmetrics at institucional level?
Introduction: information sources
On the other hand, many data
providers offering social media
metrics have been developed
Plum analytics
Impact Story
Altmetric.com

7. Can we use altmetrics at institucional level?
Introduction: altmetric.com
Altmetric.com has positioned
itself as the most used source
for altmetric studies
1.- Variety of social media platforms with regard to mentions, readership,
etc. of scientific papers
2.- Offers an aggregated indicator or ‘altmetric’ score which is a weighted
sum of values based of a given article in different social media
3.- Maintaining the ‘history’ of each altmetric indicator to an article
Advantages of altmetric.com

8. Objetive 1: Analyse the coverage of Altmetric.com at the
institutional and field level and verify its validity as evaluation tool
Objetive 2: Analyse differences when obtaining subject bibliometric
profiles from Altmetric.co and explore the possibility to make
benchmarking studies at university level
Can we use altmetrics at institucional level?
Objetives: altmetrics and universities
Our objectives are:

9. Can we use altmetrics at institucional level?
Material & methods

10. Can we use altmetrics at institucional level?
Material & methods: 4 universities
y
Size
Staff
Type
and main field
Foundation
Year
Region
& City
University
of Granada
2399 Multidisciplinary 1531 Granada
University
Pompeu Fabra
288
Specialized
Medicine/Biology
1990
Barcelona
Polytechnic
University Valencia
1847
Specialized
Engineering
1971 Valencia
University
Carlos III
539
Specialized
Social Sciences
1989 Madrid
1.- We will analyze 4 different Spanish universities
2.- Present different sizes and research publication specialization
3.- Different Foundation year and located in different regions
Selection of spanish universities

11. Can we use altmetrics at institucional level?
Material & methods: Web of Science
1.- Citable papers for the 2014 year indexed in the Web of Science for
the 4 universities were retrieved (5922 records)
2.- Records from the Web of Science provide among other information
the DOI number of each publication
3.- The DOI number is very important, as it will allow us to query the
Altmetric.com API for each document
4.- A total of 93% of the total share from our data set included DOI
Search for Web of Science papers and check DOI coverage
5.- Publications have been assigned to research areas

12. Can we use altmetrics at institucional level?
Material & methods: Web of Science
1.- We retrieved altmetric data available for our set of papers with
DOI.
2.- Then, the share of records with a score assigned by Altmetric.com
has been calculated
3.- Altmetric.com covered 2149 records from the Web of Science,
representing 36% of our data set
Search for altmetric.com score using DOI

13. Can we use altmetrics at institutional level?
Results

14. Can we use altmetrics at institucional level?
Results: altmetric.com coverage
Documents retrieved
4 universities
5922
93% with DOI
5922 web of Science
Documents
NOT Indexed in altmetric.com
3773 - 74%
Indexed in altmetric.com
2149 -36%
Indexed = at least with a value of 1 in altmetric score

15. Can we use altmetrics at institucional level?
Results: altmetric.com coverage
NOT Indexed in altmetric.com
3773 - 74%
There are significance difference in the % of Web of Science
documents indexed in altmetric.com

16. Can we use altmetrics at institucional level?
Results: altmetric.com coverage
Altmetric.com score are very concentrate just in
Twitter and Mendeley Readers

17. Can we use altmetrics at institucional level?
Results: altmetric.com coverage
We can compare the “social media” patron for the four universities

18. Can we use altmetrics at institucional level?
Results: altmetric.com coverage
Important differences by university

19. Can we use altmetrics at institucional level?
Results: Research field profile
GranadaPompeuFabra
P.ValenciaCarlosIII
We have obtained the coverage
profile (Web of science papers) of
altmetric.com per research fields:
• Significant differences in
research field coverage
(ej: Pompeu Fabra Vs Carlos III)
• Best coverage in all fields in
universities with S&T profile
(ej: P. Valencia & Pompeu Fabra)
• Dramatic coverage of
Humanities (ej: Granada) and
quite well in Social Science

20. Can we use altmetrics at institucional level?
Results: Research field profile
Totalaltmetric.comScore
Nr of Web of Science Papers indexed in Altmetric.com
Map of Web of Science Categories
We can
benchmark
universities
and compare
altmetric
performance
at category
level
Multidisciplinary Sciences
Pompeu Fabra
Biochemistry
Pompeu FabraBiochemistry
Granada
Multidisciplinary Sciences
Granada

21. Can we use altmetrics at institucional level?
Results: Research field profile – top10
Multidisciplinary
Science (Science,
Nature, Plos one…)
have higher
coverage and
concentration of
altmetric.com score.
In the case of
Pompeu Fabra 60%
of altmetric.com
score are related to
papers published in
multidisciplinary
journals.
We can see that social sciences and humanites
are not well represented in top categories

22. Can we use altmetrics at institucional level?
Results: Research field profile – top10
Top papers according
Almetric.com Scores and
journals where have
been published
We can observed the
“Nature” and
“Science” effect that
determine altmetrics
in these case studies
Final results remarks

23. Can we use altmetrics at institucional level?
Concluding remarks

24. Can we use altmetrics at institutional level?
Concluding remarks
Can we use altmetrics at institutional level?
YES
√ We can make a local bibliometric database combining bibliometric data
from Web of Science and altmetric data from altmetric.com
√ We can calculate altmetric indicators at university level and offer an
altmetric research field profile in different levels (fields and categories)
√ We can compare and make universities benchmarking studies

25. Can we use altmetrics at institutional level?
Concluding remarks
We observed a low coverage of altmetric indicators with only 36% of
all documents have an ‘altmetric’ score
To take into account:
Low coverage of web of science papers:
The DOI´s problem: One of the main factors to such low coverage could
be the use of DOI in altmetric.com
not all papers indexed in the Web of Science include DOI
not all people that mention a paper in social media use the DOI
So using DOI we have a loss of information

26. Can we use altmetrics at institutional level?
Concluding remarks
To take into account:
Multidisciplinary Journals concentrate altmetrics
The Nature and Science effect
We observed that “multidisciplinary sciences” accumulate a big piece of the
total altmetric score at university level. So universities publishing in nature,
science, pnas, etc.… also have high levels of mentions in social media.

27. Can we use altmetrics at institutional level?
Concluding remarks
To take into account:
Social science & humanities: low coverage and impact
The national factor and language bias. It is necessary to verify if there
is a national or linguistic factor influencing the coverage of altmetric
indicators in the areas of Socials Sciences and Humanities & Arts.
Another explanation could be that these areas are represented in
social media by small communities, limiting their capacity to
disseminate their research papers in social media.

28. Can we use
altmetrics
at the institutional level?
Full proceeding paper avaible at arxiv:
https://arxiv.org/abs/1606.00232