This presentation presents the findings of a usability study that was conducted as part of a discovery system redesign project. Recordings from the usability test are included.

1. How Users Conduct Research
Using Discovery Systems:
Results of a Usability Study
Kelsey Brett
Discovery Systems Librarian
University of Houston Libraries

2. Discovery System Redesign
Began in Fall of 2014

3. Discovery Redesign Activities
Primo Sandbox at time of usability testing
• Comparative
analysis - Fall of
2014
• Focus groups
with service desk
staff - Fall of 2014
• Usability Testing -
Spring of 2014

4. Usability Study: Methodology
• Received IRB approval prior to study – Exempt category
• Pop-Up Usability Study in the library atrium
• Alternated between two version of the tests, each with two tasks -
averaged test took 10 minutes
• Caller, Facilitator, and Note taker – Facilitator and Note taker
alternated roles
• Incentivized participation with library branded giveaways and snacks
• Recorded the audio and screen of each session using Morae
Usability Software

5. Usability Study: Test Version
1
Task 1
“You are trying to find an article that was cited in a paper you read
recently. You have the following citation:
Clapp, E., & Edwards, L. (2013). Expanding our vision for the
arts in education. Harvard Educational Review, 83(1), 5-14.
Please find this article using OneSearch”
Task 2
“You are doing a research project on the effects of video games on
early childhood development. Find a peer-reviewed article on this
topic, using OneSearch.”

6. Usability Study: Test Version
2
Task 1
“Recently your friend recommended the book “The Lighthouse” by
P.D. James. Use OneSearch to find out if you can check this book
out from the library.”
Task 2
“You are writing a paper about the drug cartels’ influence on
Mexico’s relationship with the United States. Find a newspaper
article on this topic, using OneSearch.”

7. Usability Study: Participation
• 15 Participants
– 15 undergraduate students
• 4 Freshmen
• 1 Sophomore
• 7 Juniors
• 2 Seniors
– 1 graduate student
– 1 post-baccalaureate student
• Disciplines represented
– 11 participants majoring in Sciences
– 2 majoring in Business
– 2 majoring in Communications
– No participation from Humanities students

8. Results at a Glance
Test 1, Task 1
Participant Time on Task Task Completion
1 1m 54s Y
2 4m 13s Y
3 1m 26s Y
4 1m 17s Y
5 1m 26s Y (required assistance)
6 1m 43s Y
7 1m 27s Y
8 1m 5s Y
Test 1, Task 2
Participant Time on Task Task Completion
1 3m 44s Y
2 2m 21s Y
3 5m 23s Y (required assistance)
4 2m 5s Y
5 3m 32s Y
6 2m 45s Y
7 3m 8s Y
8 3m 1s Y (required assistance)
Test 2, Task 1
Participant Time on Task Task Completion
1 1m 7s Y
2 56s Y
3 No recording Y
4 2m 21s Y
5 1m 8s Y
6 2m 14s Y
7 1m 15s Y
Test 2, Task 2
Participant Time on Task Task Completion
1 4m 45s Y (required assistance)
2 59s Y
3 No recording N
4 7m 47s Y
5 2m 52s Y
6 1m 33s Y
7 1m 30s Y

9. Users do not understand library or
academic jargon
-Terms such as Peer-reviewed, Digital Library, and citation cause
confusion

10. Pre-search filtering options
-Users appreciate them, and except more of them

11. Users modify their search terms
-If relevant results do not appear on the first page

12. Users do not easily distinguish
between scholarly and non-
scholarly resources
-When asked to find a newspaper article, users did not distinguish
them from other types of resources

13. Redesign Decisions
• Removed ‘UH Digital Library’ from Drop-down
menu and added ‘Articles +’
• Changed ‘Peer-reviewed journals’ facet to ‘Peer-
reviewed articles’
• Tabs were converted into buttons and renamed
• Facets were re-ordered to promote most useful
limiters
• Configured the title link to go directly to full-text or
catalog record
• More white space incorporated into display

14. Future Possibilities
• Improve integration with stack’s guide for
location information
• Remove Newspaper articles?
• Incorporate instructional information to
improve searching
• Create search scopes for specific
collections

15. Possibilities for improvements
on vendor side
• Improve relevancy ranking
• Improve autofill and ‘did you mean’
functionalities
• Support natural language searching
• Improve integration with competing ILS
and ERM systems

16. Interface prior to Redesign
May 2014 – May 2015

17. Redesigned Interface
May 2015 – Present

18. THANK YOU!
Kelsey Brett
krbrett@uh.edu