Presented at the 2016 ALISE conference in Boston, MA on January 6, 2016

1. Awareness and use of altmetrics
among LIS scholars and faculty
Sarah W. Sutton, Ph.D.
Rachel Miles, MLS
January 6, 2015
ALISE Annual Conference

2. Abstract
Altmetrics measure the impact of scholarship
via mentions in social media and other non-
traditional venues. For LIS faculty, altmetrics are
also a new area for research. The focus of this
presentation is the results of a survey of LIS
scholars’ awareness and use of altmetrics.

3. Outline
• A little bit about altmetrics
– Recent literature
– LIS faculty awareness
• The study
– Who, what, how, when?
– Limitations
• Results

4. What is the level of LIS scholars’ and
faculty awareness of altmetrics?
12.34%
24.68% 24.03%
32.47%
6.49%
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
1 - never heard of them 2 3 4 5 - I'm an expert

6. The literature from 2015: Efficacy and
use of metrics
• In academic libraries
– Suiter and Moulaison (2015)
– Booth and Hendrix (2015)
• Information policy
– De Groote, Shultz, and Smalheiser (2015)
• Influential altmetrics
– Bornmann (2015)
– Zahedi, Costas, and Wouters (2015)
• Spurious metrics
– Gutierrez , Beall , and Forero 2015
– Davis, 2015

7. The literature from 2015: Explanations
and definitions
• Criticism
– Gaming
– Correlation with citations
– Non-academic social media mentions
• Benefits
– Non-article research output
– Replacing JIF
– Measuring attention from the general public
(Roemer & Borchardt, 2015)

8. The literature from 2015
• Related LIS faculty surveys
– Peekhaus & Proferes (2015)
– Syn and Oh (2015)
• Developing metrics: Relative Citation Ratio
– Hutchins, Yuan, Anderson, and Santangelo (2015)

9. The Survey
• Based on Sutton and Miles. (2015)
• Surveygizmo.com
• 2,312 invitations sent, 159 responses received
• 3 weeks, 1 reminder
• 25 – 30 questions
• ~ 25 minutes to complete

10. Limitations & Criticism
• Response rate = 6.88%
• No “Goodness of Fit”
• Anonymity
• Part-time faculty
• No research

11. Variables
• Appointment (full time, part time)
– Full time teaching appointment: teaching area
– Full time teaching appointment: research area
• Tenure
• Years of teaching
• Academic status (assistant, associate, full,
emeritus)

12. Familiarity with altmetrics by
appointment type
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
1 - never
heard of
them
2 3 4 5 - I'm an
expert
Full time (n=111)
Part time (n=43)

13. Familiarity with altmetrics by tenure
track status
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
1 - never
heard of
them
2 3 4 5 - I'm an
expert
Tenure track (n=97)
Non-tenure track (n=11)

14. Familiarity with altmetrics by years
teaching
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
< 1 year
(n=10)
1 - 5 years
(n=51)
6 - 10
years
(n=34)
11 - 20
years
(n=30)
> 20 years
(n=28)
1 - never heard of them
2
3
4
5 - I'm an expert

15. Familiarity with altmetrics by years of
teaching
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
1 - never
heard of
them
2 3 4 5 - I'm an
expert
>=5 years (n=61)
<= 6 years (n=92)

16. Distributions of faculty by years
teaching
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
ALL (n=155) Part-time (n=43) Full-time (n=112)
Less than one year
1 - 5 years
6 - 10 years
11 - 20 years
More than 20 years

17. Familiarity with multiple metrics
1 - never
heard of
them
2 3 4
5 - I'm an
expert
Altmetrics 12.34% 24.68% 24.03% 32.47% 6.49%
Citation counts 1.91% 1.91% 22.93% 56.69% 16.56%
Usage 0.00% 9.74% 24.68% 52.60% 12.99%
Author h-index 0.00% 24.82% 27.74% 37.23% 10.22%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%

18. Other metrics mentioned
• None, 10
• Google Scholar Metrics, 8
• Qualitative measures of impact (e.g. who is saying
what) , 6
• Aggregations (e.g. Altmetric.com, Impact Story,
Academia.edu), 4
• SciMago, 3
• Research Gate, 2
• G-index, 2
• Eigenfactor, 2
• Books sold/library holdings, 2

19. Other metrics mentioned once
• Scientometrics
• Epistemetrics
• Entitymetrics
• M-quotient
• SNIP (Source Normalized Impact per Paper)

20. Questions? Comments?
Please contact me:
ssutton3@emporia.edu
620-341-5816

21. References & Bibliography
Booth, H., A. & Hendrix, D. (2015). Libraries and institutional data analytics: Challenges and opportunities. The Journal
of Academic Librarianship, 41(5), 695–699. http://doi.org/10.1016/j.acalib.2015.08.001
Bornmann, L. (2015a, March 10). How much does the expected number of citations for a publication change if it
contains the address of a specific scientific institution? A new approach for the analysis of citation data on the
institutional level based on regression models. Retrieved from
http://figshare.com/articles/How_much_does_the_expected_number_of_citations_for_a_publication_change_if_it_co
ntains_the_address_of_a_specific_scientific_institution_A_new_approach_for_the_analysis_of_citation_data_on_the_
institutional_level_based_on_regression_models/1330139
Bornmann, L. (2015b, March 13). Overlay maps based on Mendeley data: The use of altmetrics for readership
networks. Retrieved March 13, 2015, from
http://figshare.com/articles/Overlay_map_for_Science_Nature_PNAS/1334179
Bornmann, L. (2015c, March 20). Usefulness of altmetrics for measuring the broader impact of research: A case study
using data from PLOS and F1000Prime. Retrieved from
http://figshare.com/articles/Usefulness_of_altmetrics_for_measuring_the_broader_impact_of_research_A_case_stud
y_using_data_from_PLOS_and_F1000Prime/1344583
Davis, P. (n.d.). Knockoffs erode trust in metrics market. Retrieved from
http://scholarlykitchen.sspnet.org/2015/03/10/knockoffs-erode-trust-in-metrics-market/
De Groote, S. L., Shultz, M., & Smalheiser, N. R. (2015). Examining the impact of the National Institutes of Health public
access policy on the citation rates of journal articles. PLoS ONE, 10(10), e0139951.
http://doi.org/10.1371/journal.pone.0139951

22. References & Bibliography
Dhlman, A. K. (2015). Bibliometrics to altmetrics: Changing trends in assessing research impact. DESIDOC Journal of
Library & Information Technology, 35(4), 310–315.
Ding, Y., Song, M., Han, J., Yu, Q., Yan, E., Lin, L., & Chambers, T. (2013). Entitymetrics: Measuring the impact of entities.
PLoS ONE, 8(8), e71416. http://doi.org/10.1371/journal.pone.0071416
Gutierrez, F. R. S., Beall, J., & Forero, D. A. (2015). Spurious alternative impact factors: The scale of the problem from an
academic perspective. BioEssays, n/a–n/a. http://doi.org/10.1002/bies.201500011
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research
metrics. Nature, 520(7548), 429–431. http://doi.org/10.1038/520429a
Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M. (2015). Relative Citation Ratio (RCR): A new metric that
uses citation rates to measure influence at the article level. bioRxiv, 029629. http://doi.org/10.1101/029629
Look out for Bogus Impact Factor Companies. (n.d.). Retrieved from http://scholarlyoa.com/2013/08/06/bogus-impact-
factor-companies/
Orduna-Malea, E., Ayllón, J. M., Martín-Martín, A., & López-Cózar, E. D. (2015). Improvements in Google Scholar
Citations are for the summer: Creating an institutional affiliation link feature. arXiv:1509.04515 [cs]. Retrieved from
http://arxiv.org/abs/1509.04515

23. References & Bibliography
Peekhaus, W., & Proferes, N. (2015). How library and information science faculty perceive and engage with open access.
Journal of Information Science, 41(5), 640–661. http://doi.org/10.1177/0165551515587855
Roemer, R. C., & Borchardt, R. (2015). Issues, controversies, and opportunities for altmetrics. Library Technology
Reports, 51(5), 20–30.
Suiter, A. M., & Moulaison, H. L. (2015). Supporting scholars: An analysis of academic library websites’ documentation
on metrics and impact. The Journal of Academic Librarianship, 41(6), 814–820.
http://doi.org/10.1016/j.acalib.2015.09.004
Sutton, S. W., & Miles, R. (2015, September). Using alternative metrics for collection development. Presented at the
Kansas Library Association / Missouri Library Association Joint Conference, Kansas City, MO.
Syn, S. Y., & Oh, S. (2015). Why do social network site users share information on Facebook and Twitter? Journal of
Information Science, 41(5), 553–569. http://doi.org/10.1177/0165551515585717
Wasserman, M., Zeng, X. H. T., & Amaral, L. A. N. (2015). Cross-evaluation of metrics to estimate the significance of
creative works. Proceedings of the National Academy of Sciences, 112(5), 1281–1286.
http://doi.org/10.1073/pnas.1412198112
Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the
presence of “alternative metrics” in scientific publications. Scientometrics, 101(2), 1491–1513.
http://doi.org/http://link.springer.com/article/10.1007%2Fs11192-014-1264-0