User Interface Design for Medical Devices - The Relationship Between Usability and Safety Rich Newman's presentation from the UXPA Boston 2016 conference

1. The relationship between usability and safety
29 April 2016
Rich Newman
Usability Engineering Manager
BlackHägen Design
rich.n@blackhagendesign.com
User Interface Design for Medical Devices

2. Impact of Poor Usability
• Web Site, Software Application, Consumer Product
– Poor conversion rate – loss of sales
– Mistakes on orders, reports, etc.
– User frustration
2

3. Impact of Poor Usability
• Medical Device
– Incorrect medication or dose
– Incorrect therapy setting
– Incorrect interpretation of data
resulting in misdiagnosis
3

4. Usability Directly Relates to Safety
• If a device is difficult to use, the user is more
likely to use it incorrectly
• Can result in harm or death
– to patient, operator, or bystanders
4

5. Example 1: Therac 25
• Radiation Therapy Device
– Used for both low and high dose therapy
• Incorrect key sequence caused machine to switch to high
dose mode without engaging safety hardware*
– User quickly corrected an incorrect key-press
– Slow response to key press caused device to respond
incorrectly
– Patients received 125x the normal dose of
radiation
• 6 patients received radiation overdose
– 5 deaths between 1985-1987
– Machine recalled after 5th incident
*The Therac disaster resulted from a combination of software and usability problems. It lead to
sweeping changes in safety critical software development. Major changes related to usability were
not addressed until more recently.
5

6. Example 2: Infusion Pumps
• Between 2005 and 2009, FDA reported:
– 56,000 infusion pump incidents
– 710 deaths
– 87 recalls
• Infusion devices account for up to 35% of all
medication errors that result in significant harm
• A large percentage of these adverse events are due to programming errors that can be
attributed to poor usability
– Entering weight in pounds instead of kg – results in 2.2x overdose
– Incorrectly placed decimal point – results in 10x under or over infusion
– Select incorrect dose mode – mg/kg/min instead of mcg/kg/min – results in 1000x
overdose
Source: FDA White Paper: Infusion Pump Improvement Initiative, April 2010
6

7. How big is this problem?
• Recalls
– 44% of medical device recalls due to design problems
– More than 1/3 involved user interface
• Medical Device Incident Reports
– FDA receives about 100,000 medical device incident reports
every year.
– More than 1/3 can be attributed to usability problems
– Small fraction of all the incidents that actually occur get
reported
7
Source: James Carstensen 1990 FDA presentation: “An internal FDA study some years ago showed that 44% of medical device recalls
were the result of design problems and more than 1/3 involved the device-user interface (device, it’s labeling or instructions for use).”

8. What is Considered a Medical device
• Physical Devices / Systems
– Robotic Surgery System
– Defibrillator
– Infusion pump
– Pulse oximeter
– Auto injector – Epi-Pen
– Oral thermometer (analog or digital)
• Software
– To monitor or configure devices
• Central monitoring station for a patient monitor
– Image storage and viewing systems (PACS, etc.)
• Mobile apps
– Used as an accessory to a regulated medical device
– Used for diagnosis of disease or other conditions
– Used for the cure, mitigation, treatment, or prevention of disease
8

9. Definition: User Interface
Anything the user interacts with
• Display and touch screen (GUI)
• Buttons and knobs
• Mounting hardware
• Cover, cap, or case
• Disposables (needles, tubing, etc.)
• Labeling
• Training materials
• Instructions for use (IFU)
Not just the GUI
9

10. Definition: USE ERROR
• Incorrect use called a USE ERROR
– NOT a User Error
– The User is not at Fault
• Device / UI Design is at fault
– The design enabled or tolerated the error
10

11. Causes of USE ERRORS
• Perception
– Failure to see or hear information
• Cognition (Processing)
– Forget information or step
– Forget or apply rule incorrectly
– Misinterpret information
• Action
– Inability to reach a control
– Use incorrect control
– Use a control incorrectly
– Failure to activate control
11
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011

12. Use Error = Device Defect
USE ERRORS treated the same as any
other device fault
• A use error is considered a device defect
– Same as hardware or software fault
• Must be reported to FDA or local regulatory
body
12

13. What is Being Done?
• Most countries now require proof that the UI is safe before
approving a medical device for sale
– A new international standard went into effect in 2010
• IEC 62366 - Application of Usability Engineering to Medical Devices
• Revised in 2015
– Outlines process for developing safe medical device user
interfaces
Devices will not be approved unless UI is shown to be safe
13

14. But UI Design is Subjective…
How can you prove the UI is safe?
14

15. Risk Based Process
• Adapted from proven processes used
for hardware and software
– Originated in aviation industry
• Integrate UCD methods to identify and
mitigate risks
• Iterative Process
– Loop back as new risks identified
15
Source: ANSI/AAMI HE75:2009
HF Risk Management Workflow

16. Define Intended Use, Users, Use Environment
16
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011

17. Factors that affect Safety: Users
• Abilities
– Physical size, strength, and stamina,
– Physical dexterity, flexibility, and coordination
– Sensory abilities (i.e., vision, hearing, tactile
sensitivity)
– Cognitive abilities, including memory
– Familiarity with similar devices
• Factors that affect abilities
– Age
– Medical conditions
– Literacy and language skills
– General health status
– Mental and emotional state
– Medical expertise
17

18. Factors that affect Safety: Use Environment
• Lighting
– Visibility of displays and indicators
• Noise
– Ability to hear audible cues and alarms
• Space
– Access to device, interference with cords and tubing
• Activity
– Distraction
18

19. Factors that affect Safety: User Interface
• Input
– Controls – type, size, spacing, actuation force, etc.
– Size and Configuration of device
• Processing
– Menu and control design
– How system responds to user action
– How feedback is provided to user
• Output
– Displays and Indicators
• Brightness, Off angle visibility, Glare resistance
• Font selection, size, color; screen layout
– Auditory
• Alarms, Beeps, Ambient Noise (motors, fans, etc.)
• Other
– Device labeling, packaging,
– Training materials, Instruction for Use (IFU), and other reference materials.
19

20. Identify Use Related Hazards
• Identify Known Problems
– Internal customer complaint files
– Government and trade group databases
• FDA, EU, and others maintain complaint, adverse event, recall, and other databases
• Analytical Approaches
– Interviews and focus groups
– Contextual inquiry
– Function and task analysis
– Heuristic analysis / Expert review
• Formative Testing
– Unanticipated Use Errors frequently found during formative testing
20

21. Estimate and Prioritize Risk
• Use established risk analysis methods
– Failure Mode and Effect Analysis (FMEA)
– Fault Tree Analysis (FTA)
– Details outside of the scope of this talk
• Use results to prioritize and address potential hazards
– Based mainly on level of harm
– Frequency of occurrence also considered but less of a factor
• Iterate
– Regularly review and update during development process
21

22. Implement Risk Controls
22
Source: AAMI/ IEC 62366 -1 Public Review Draft (AAMI/IEC DS-1 6236601-D)

23. Example – Displaying Units of Measure
• Hazardous situation
– User enters weight in pounds instead of kg
• Harm
– Weight based dose calculation is 2.2x too high resulting in overdose
• Fix
– Display units with similar prominence as numeric field
– Locate units adjacent to numeric entry
Weight (kg) Weight
100 45.4 kg
23
X Source: NPSA Design for
Patient Safety – A guide to the
design of Electronic Infusion
Devices, Edition 1, 2010

24. Example – Displaying Numeric Values
• Hazardous situation
– Misreading decimal numbers due to incorrect display of leading or trailing zeros
– Misreading of large numbers
• Harm
– Misdiagnosis to incorrect interpretation of data
– Failure to detect data entry error
• Fix
– Do not display trailing zeros (after the decimal point)
– Use a leading zero (before decimal point) for numbers less than 1
– Include thousands separators for large numbers
24
Source: NPSA Design for Patient
Safety – Guidelines for safe on-
screen display of medication
information, Edition 1, 2010
Incorrect Correct
Dose 5.0 mg Dose 5 mg
Dose 5.30 mg Dose 5.3 mg
Volume .6 mL Volume 0.6 mL
Dose 10000 u Dose 10,000 u

25. Example – Numeric Keypad Design
• Hazardous situation
– User presses wrong key resulting in incorrect numeric entry
• Harm
– Incorrect therapy, overdose, underdose
• Fix
– Always use the same keypad layout ; “Phone” recommended vs. “Calculator”
– Do not place ‘0’ and ‘.’ adjacent to each other
– Keypad layout is only part of the fix
25
X
Source: NPSA Design for
Patient Safety – A guide
to the design of Electronic
Infusion Devices, Edition
1, 2010
7 8 9
4 5 6
1 2 3
0 .
1 2 3
4 5 6
7 8 9
C 0 .
1 2 3
4 5 6
7 8 9
0 C .
X

26. Risk Control Example- The Total Solution
• Each of the previous examples is only part of the solution
– Combine risk control methods for maximum benefit
26
1 2 3
4 5 6
7 8 9
0 C .
Weight
4,540 kg
Error – Weight
Exceeds Limit

27. Formative Evaluations
• Purpose is to inform the design
– Explore strengths and weaknesses of UI
– Identify unanticipated USE ERRORS
• Test with representative users
– Users with similar age, physical and cognitive abilities
– Users with appropriate training and experience
• Simulate use environment
– The more realistic the environment, the better the results
• Various methods of simulating treatment – mannequins, “dummy tummy”, …
– Include other devices typically found in use environment
• Test labeling, training materials, IFU, etc.
– Anything used to mitigate risk
• Iterate
– Fix problems and test again
27

28. Summative (Validation) Testing
• Use production equivalent devices
• Test in realistic use environment
– No think out loud; moderator probing, etc.
• 15-25 participants per user group
• Test essential and safety critical tasks
– Leverage results of formatives to narrow scope
• Include training materials, IFU if normally used
– Approximate training actual users will receive
– Allow time between training and testing (“training decay”)
28

29. Summative Data Collection
• Objective data
– Task success or failure, USE ERROR, close call
– Reference to instructions for use
– Need for assistance, evidence of difficulty or confusion
– Unsolicited comments
• Subjective data (after use)
– Discuss perceived reasons for any essential and critical task errors, failures and
difficulties
– Solicit feedback on design of device, packaging, labeling and training
29
Source: Human Factors Engineering of Combination Products and the FDA, Molly Story, July 2012
http://www.aami.org/hfconnect/FDA/HF_Engineering_Combination_Products_071712.pdf

30. Analysis of Summative Data
• Analyze all USE ERRORS and failures
– Determine root cause and potential clinical consequences
– Determine need to modify device, labeling, or training
– Identify true residual risks
• USE ERRORS/failures are not of equal importance
– Some errors might be frequent but inconsequential
– Some errors might be rare but reveal a hazardous design deficiency that was not previously
recognized
• Perform Residual Risk Analysis
– Can have USE ERRORS and still pass summative
– USE ERRORS must pass residual risk analysis
• Is benefit greater than risk?
30
Source: Human Factors Engineering of Combination Products and the FDA, Molly Story, July 2012
http://www.aami.org/hfconnect/FDA/HF_Engineering_Combination_Products_071712.pdf


31. Documentation
• FDA Human Factors Report
– Summary of usability engineering inputs, activities, and outputs
1. Intended device users, uses, environments
2. Description and images of device user interface
3. Summary of known use problems
4. User task descriptions, risk priorities, success criteria
5. Summary of preliminary evaluations
6. Validation study protocol, results and analysis
7. Conclusion
31
Source: FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design, 2011

32. Further reading
WEB
• AAMI Human Factors Resources Page:
http://www.aami.org/hfconnect/
– Links to major standards and other valuable Medical HF
Resources
• FDA Human Factors Home Page:
http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/H
umanFactors/default.htm
• FDA Guidance –Applying Human Factors and Usability Engineering
to Medical Devices; February 3, 2016
http://www.fda.gov/downloads/MedicalDevices/.../UCM259760.pdf
STANDARDS
• ANSI/AAMI/IEC 62366-1:2015 (Available for Purchase)
Medical devices - Part 1: Application of usability engineering to
medical devices
http://my.aami.org/store/detail.aspx?id=6236601-PDF
• ANSI/ANSI HE75:2009(R)2013 (Available for Purchase)
Human factors engineering—Design of medical devices
http://www.aami.org/publications/standards/he75.html
– Specific design recommendations and related information
BOOKS
• Handbook of Human Factors in Medical Device Design
Edited by Matthew Weinger, Michael Wiklund, and Daryle Gardner-
Bonneau
• Designing Usability into Medical Products
By Michael Wiklund and Stephen Wilcox
• Usability Testing of Medical Devices
by Michael Wiklund, Jonathan Kendler, and Allison Strochlic
32

33. www.BlackHagenDesign.com | 727.736.0582
Questions?
33
29 April 2016
Rich Newman
rich.n@blackhagendesign.com

34. Other Sources of Hazard Data
• Consider entire product lifecycle
– Transport
– Storage
– Installation
– Operation
– Maintenance and Repair
– Disposal
• Reasonably Foreseeable Misuse
– Use of non-approved disposables
– Use by unintended user groups
– Use in unintended environment
– Off label uses
34