A presentation, with Christine Yates, for the Information Design Association, London, 1998

1. Usability Evaluation of Forms
Caroline Jarrett and Christine Yates
Presentation for the Information Design Association, London, 1998

2. Introductions
 Caroline Jarrett
 Forms specialist, usability consultant
 Christine Yates
 Head of usability, Inland Revenue Forms Unit

3. Background
 Needed a usable tax return
 Familiar with MUSiC methodology
 Measurement of Usability in Context
 National Physical Laboratory
 Developed for measuring computer usability
 Adapted methods for non-computer
 Used on Inland Revenue and other forms

4. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

5. Why evaluate?
 Quantitative techniques
 Traditional qualitative
 Interaction with the product

6. Video clip
 Calculation of tax
 Two different layouts
 Participant gave appropriate permissions

7. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

8. Product definition
 The form
 …. and the guidance
 …. in an envelope
 …. with a return envelope
 …. and a helpline
 …. the policy for dealing with it
 …. the source of the information

9. Why does it matter who the users are?
 You must complete your hardanger by 19th May
1999. Place your order for:
 22 count
 perle
 chart
 You must complete your annual return by 19th
May 1999. Place your order for:
 P14 (OCR)
 P11D
 Booklet 480

10. Who are the users?
 Recipient
 Advisor
 Friend or relative
 Customer assistance

11. Who are the users?

12. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

13. The tasks reflect the goals
 Find the questions
 Understand the questions
 Put answers in the boxes
 Send it back
 Comply with deadlines

14. Structure of evaluation
 A participant
 Attempting a task
 With the product

15. Types of evaluation
 Measurement
 Diagnostic
 Concept

16. Participants
 Real users
 Proxy users
 Experts

17. Number of participants
 Statistical sample
 One is enough

18. Number of participants
 Statistical sample
 One is enough
 5 for a typical user evaluation
 Double for comparative evaluation
 Avoid evaluator exhaustion

19. Recruitment
 Who do you know?
 Lists from the client
 Market research recruitment

20. Running an evaluation
 Preparing an example
 Run through is essential
 Pre- and post- interviews
 Observers

21. Role of the interviewer
 Think aloud
 More natural
 More surprises
 More convincing
 Can provide measures
 With an interviewer
 Better control
 Keep them talking
 More detail
 Less nerve-wracking

22. Observation
 Self-reporting
 Interviewer
 Video
 Remote link

23. Rights of participants
 MRS code of conduct
 ESOMAR video code

24. Analysis and reporting
 Log of events
 List of problems
 Full methodological report
 Video report
 Personal observation

25. Types of problems
 Layout
 Question and answer
 Task structure

26. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)