This document discusses various usability methods that can be used at different stages of electronic medical record (EMR) development to improve usability. It describes contextual inquiry, personas, use case scenarios, requirements gathering, user stories, prototyping, card sorting, concept exploration, usability testing, incorporating user feedback, and maintaining design intent. Implementing usability methods early in the development process is most cost-effective, as it allows findings to be incorporated before significant code is written. While usability adds costs, it can provide measurable benefits like improved productivity, satisfaction, and safety.

1. June 9, 2010 Usability Methodsto Improve EMRs Janey Barnes, PhDUser-View, Inc. Raleigh, NC Jeff Belden, MD Family &Community MedicineUniversity of Missouri 1

2. Conflict of Interest Disclosure Janey Barnes, PhD Contracted Research: Allscripts Patagonia Health Duke Health Systems 2

3. Conflict of Interest Disclosure Jeff Belden, MD Dr. Belden is on faculty at University of Missouri - Columbia, and an EMR usability consultant with the Tiger Institute, a technology collaborative between the University of Missouri and Cerner Corporation. Dr. Belden has no financial interest. 3

4. Session Objectives Describe stages of software development in which usability methods can be most effective. List and describe usability methods applicable to each stage of EMR development. Analyze the cost/benefit of implementing usability methods within a user-centered design process. 4

5. Software Development Process 5

6. Waterfall modelof software development 6

7. 7 Waterfall

8. “Agile” modelof software development 8

9. 9 Agile

10. Usability methods work for both 10

11. don’t forget the buyer’s sideof usability 11

12. 12 Agile Waterfall Selection Implementation

13. Want to be a change-agent? 13

15. User Centered Design Process 15

17. Card sorting

18. Concept exploration

19. Usability Testing

20. and testing

21. Keep testing

22. Requirements

23. User stories

24. Summative testing

25. Affinity Diagramming

26. Personas

27. Use case scenarios

28. User acceptance testing

29. Incorporating feedback

32. Performance testing

33. Observe one of their experienced (but not an ace) users

34. Do clinical scenario test with real data

35. Define target measures

36. Set goals18

37. Implementation Activities Usability Methods of Interest Task analysis & user observation Card sorting (picking categories, nomenclature) Design guidelines & Style guides Paper prototyping Have clinician create documents for several patients 19

40. Details for planning& executingeach method

41. Best timing the software development schedule

42. How to deliver impactto the product21

43. Contextual Inquiry Persona 22

45. tasks

46. workflows

47. environment

48. description of a specific person (a target user). Details make them real.23 Contextual Inquiry Persona

50. Make notes.

51. Stand, sit, to side, in middle.

52. Stay out of way

54. Requirements gathering

55. Before sprint planning

58. tasks

59. workflows

60. environment

62. Make notes.

63. Stand, sit, to side, in middle.

64. Stay out of way

66. Requirements gathering

67. Before sprint planning

70. Not sell, or teach, or fix

72. 33 Contextual Inquiry How long to compile findings? Compile into Personas Use case scenarios Work flows Task flows

75. Planning

76. Execution

77. Timing

78. Impact35

80. how to score effectiveness - agreeing what errors user might make

81. the effectiveness target

82. the satisfaction target.36

84. How to score effectiveness

85. Effectiveness target

87. Learnability

88. Operability

89. AttractivenessRecruiting physicians is challenging. 37

91. Provides concrete objectives & testable criteria for usability38

92. User Stories 39

94. Slim, give high-level requirements

95. Just a sentence or three.

96. “A surgeon needs latest evolving lab results while moving through the hospital, without stopping to log-in at a PC”.40

97. User Stories Planning Execution Arrange short meeting with customer Bring 3x5 cards for each story User writes user stories 3x5 note card Rewrite until clear Developer may use questions to get user going 41

99. Low maintenance

100. Keeps contact with user

101. Helps estimate the development work effort42

105. Design & User Feedback Activities Prototyping 45

106. Prototyping Goal clarify requirements create draft of interaction designs & screen designs to be rapidly simulated & tested 46

108. Developers

109. Facilitator & scribe

110. Four stages

111. Concept design

112. Interaction design

113. Screen design

114. Screen testing47

116. Sketch possible approaches

117. See if they meet agree objectives

118. Interaction design

119. Use Post-It notes, writing each suggested screen or activity

120. Group them, name clusters, arrange sequences48

122. Brainstorm screen designs with users

123. Ask user to carry out a realistic task, pointing among screen sketches

124. Screen testing

125. Mock-up rough designs.

126. Walk thru steps, explaining what happens, or get fancier with paper menus, dialog boxes, etc.49

127. Prototyping Timing Early Before code Impact Find usability problems early before code is written Communication between users & designers is promoted Paper is cheap & quick 50

128. Design & User Feedback Activities Card sorting 51

131. Open (user names groups)

132. Note results. Include names user gave groups, proximity info or comments user made.53

133. Card Sorting Timing Design phase Before coding Impact Helps discover users’ mental model Shows how ideas or concepts should be presented 54

134. Speaker handoff

136. Communicating information priority

137. Engaging brand56

138. Visual Design It’s not just pretty colors… 57

139. Visual Design Contrast | Repetition |Alignment |Proximity Contrast | Repetition | Alignment | Proximity Contrast | Repetition | Alignment | Proximity 58

140. Visual Design Good visual design communicates: Info organization Info priority and reduces cognitive load 59

141. Design & User Feedback Activities Visual Design Concept Exploration Planning: Iterative Process Execution: Research methods to quantify effectiveness of Visual Design Timing: In parallel with information & interaction design Impact: Critical! Visual Design is not just “make it pretty” 60

143. Affinity Diagramming

144. Prototyping Low-resolution High-resolution

145. Design & User Feedback Activities Usability Tests 64

146. Usability Tests 65

148. Design decisions

149. Identify usability issues

150. Summative

151. Evaluate usability goals

153. Test Design

155. Test Environment

156. Test Report

157. Pilot Test

158. Logistics

162. Enough

163. Stability

165. UX doing formative activities for next sprint and summative activities for current sprint 68

167. Actionable & Prioritized Recommendations

169. Development Activities User Acceptance Testing (Software) Incorporating user feedback Maintaining the intent of the design 71 Keep the door open between UX and Development!

171. Planning: Begins during Define Phase

172. Execution: Automatic versus Manual

173. Timing: Nearing the end

174. Impact: Too late to IMPACT the application?72

176. Planning: Little planning time – rely on process

177. Execution: SUPPORT

178. Timing: Nearing the end

179. Impact: CRITICAL time73

181. System-wide usability test

183. Test Design

185. Test Environment

186. Test Report

187. Pilot Test

188. Logistics

191. Live usability test to inform current application and next release

193. Test Design

195. Test Environment

196. Test Report

197. Pilot Test

198. Logistics

200. 80 Post-Launch Post-Launch Post Launch Activities Learnings Discover Activities Implementation Discover Define Launch Selection Design Beta Develop Implementation

201. 81 Cost / Benefit

202. Cost/Benefit Key to cost-effective product usability is to Plan & Manage the usability activities within the development process. 82

204. Card sorting

205. Concept exploration

206. Usability Testing

207. and testing

208. Keep testing

209. Requirements

210. User stories

211. Summative testing

212. User acceptance testing

213. Incorporating feedback

214. Maintain design intent

215. Affinity Diagramming

216. Personas

217. Use case scenariosspecific activity designed to gather data to address specific research question

218. http://bit.ly/UXmethods 84

219. 85 Costs Including Usability in Development Cycle Design Alternatives Cost of Changes Planning Design Development Release

220. CostsUsability in the Development Cycle 1 of 2 Discount Usability vs Premium Usability Time (labor) Employee vs Contractor (expendable) Design vs Evaluation Test system setup Participant recruitment and compensation 86

222. Sometimes too busy to stop and determine how to integrate usability during development.87

224. Fewer errors during tasks

226. Marketing voice88

228. Proprietary data used competitively and not available to the publicUsability community has made efforts to quantify benefit of Usability interventions 89

229. Tradeoffsbetween Usability Costs & Benefits Case scenarios presented as examples that hopefully demonstrate the cost-benefit tradeoffs across several types of HF activities. Data for case scenarios taken from: Good Ergonomics Is Good Economics (1996). Hal Hendrick. Proceedings of the HFES 40th. Usability Professionals’ Association (UPA) http://www.upassoc.org 90

231. Early enough to make changes91

233. Links Usability Methods Table | Usability.net HIMSS EHR Usability Links | Online resources

234. Questions? For further info, contact:Janey Barnes PhD | jbarnes@user-view.comJeff Belden MD | jeffbelden@gmail.com