Here is a usability report on a transit mapping application.

1. Missouri Western State University
PSY 600
• Stewart Hutchison
• Santhosh Kasula

4. Usability Testing Report for One Bus Away,
a web-based bus routing application
http://onebusaway.org/
Date of Report: 10/18/2014
Date of Test: 10/07 – 10/08
Location of Test: MWSU Psychology Lab
Prepared for: OneBusAway Dev Team
Prepared by: Stewart Hutchison
Santhosh Kasula

5. Table of Contents
Executive Summary
Participants
Tasks & Procedures
Results
Usability Videos (4:55)
Major Findings
Conclusion

6. Executive Summary
The goal was to determine:
1. How do undergraduate students navigate a novel
online map system?
2. How do satisfaction ratings compare to experience
and frequency of use?

7. Participants
Four undergraduate students were recruited from MWSU:
User Age Gender
1 35 Female
2 20 Male
3 23 Female
4 22 Female

8. Tasks & Procedures
Each participant was given:
• Demographic questionnaire
• Tasks to complete with a laptop computer (recorded with Camtasia)
• Satisfaction survey

9. Tasks & Procedures
Demographic Questionnaire:
1. What is your age?
2. What is your gender?
3. Have you used an online map to plan a route for travel?
4. If so, how often do you plan trips with online maps?

10. Tasks & Procedures
Tasks:
1. Find the web application.
2. Locate the CAMLS building.
3. Identify bus route #30.
4. When will the next bus arrive at CAMLS?
5. Where is this bus currently?

11. Tasks & Procedures
Satisfaction Survey (SUS, Jeff Sauro 2011):
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
[http://www.measuringu.com/sus.php]

12. Methodology
• Sessions were recorded with Camtasia
• Videos were reviewed and measured:
Efficiency (Time on task)
Effectiveness (Errors and assists)
Satisfaction (SUS survey)

13. Results
(Time on task, seconds)
User Task #1
Find Application
Task #2
Locate CAMLS
Task #3
Identify route 30
Task #4
When next bus…?
Task #5
Where is the bus…?
#1 2 31 49 N/A N/A
#2 32 33 255 380 437
#3 82 185 170 285 402
#4 31 30 225 305 250

14. Results (Time on task, Errors, Assists)
Task # Time on Task
(seconds)
Errors Assists
1. Find Application x̅
= 36.75, s = 33.22 6 1
2. Locate CAMLS x̅
= 69.75, s = 76.84 8 1
3. Identify Route #30 x̅
= 174.75, s = 90.92 13 0
4. When next bus…? x̅
= 329.33, s = 96.67 21 1
5. Where bus currently…? x̅
= 363.0, s = 99.41 23 3

15. Results (Demographics and Satisfaction)
User Used online maps? How Often? Satisfaction?
(SUS)
#1 Yes Weekly 5
#2 No N/A 5
#3 Yes Monthly 25
#4 Yes Weekly 47.5

16. Satisfaction Survey (SUS, Jeff Sauro 2011)
[http://www.measuringu.com/sus.php]

17. Satisfaction Survey (SUS, Jeff Sauro 2011)
[http://www.measuringu.com/sus.php]

18. Task 1 – Find the web application
3/4 users made errors…

19. Task 3 – Identify Route #30
Feedback and Google Map comparisons…

20. Task 4 – When will the next #30 bus arrive?
Getting a little “flustered”…

21. Task 5 – Where is the bus currently?
Nope… 4/4 users were unable or required assistance

22. Major Findings
• 3/4 were unable to correctly locate the web application
• All users compared the system to Google Maps features
• None of the users were able to locate the bus on their own

23. Major Findings
Positive feedback:

24. Major Findings
Highest satisfaction level (x̅
= 2.25, s = 1.41)
was reported for survey question #5:
“I found the various functions in this system were well integrated”

25. Conclusion
Each successive task was met with more difficulty, as
both average time and errors increased.
Although based in a different location, these
undergraduate students are potential users who
could transfer to your area.

26. What We Learned…
Ask the user questions before offering help (don’t be the nice guy)
Define concepts before reviewing videos (such as “error” and “assist”)
Recruit participants and collect data as early as possible

28. Questions?