The document is a presentation about developing a continuous UX improvement model. It proposes a 10-step UX optimization lifecycle that combines elements of traditional UX and continuous improvement processes. The lifecycle includes goals analysis, requirements gathering, design, testing, implementation, metrics collection, and reporting phases that form a repeating cycle of analysis and improvement. The presentation advocates for this approach to promote evidence-based practices, reduce rework, and institutionalize continuous optimization processes.

1. Towards a Continuous
UX Improvement Model
Leveraging an Usability Optimization Lifecycle
Jeffrey Ryan Pass, Solution Architect & Lead User Experience Consultant
Aquilent (we’re hiring)
jeff.pass@aquilent.com | @jeffpass | about.me/jeffpass
UXCamp DC 2015, 01/03/2014

2. Special thanks to Debra Martin who
created the original UX optimization
lifecycle that we have continued to
develop and refine at Aquilent. I
borrowed heavily and shamelessly
from her excellent work.
And to anyone who attended Debra’s UXCamp DC 2013 presentation
on continuous optimization… some of this will seem very familiar.

3. and you think, cube-dweller or drone. But hear me out.
Now I know, you hear ‘continuous improvement…

4. This is a good thing. (OK, maybe not this good, but still…)

5. Why continuous improvement and optimization
as opposed to continuous progress?

6. And why
this idiot?

7. But what is continuous improvement and optimization?

8. And what isn’t continuous improvement and optimization?

10. Today’s episode is brought to you by numbers, aka…

11. The (crucial) role of metrics

12. Why so crucial?
Data-driven, evidence-driven approach
Identify pain points
Triage then prioritize work
Justify:
- Usability studies
- IA changes
- New content/features
- Design updates
Measure success
Refine approach going forward
Plan, staff, and maximize IA/UX skillset
utilization
Identify key metrics supporting continuous
improvement/optimization
>_

13. Bear with me, this is where we go all PowerPointy…

15. 10. Report
and Advise 1. Goals
and Needs
2. Data
Analysis
3.
Require-
ments
4. User
Tasks
5. Design
Solution
6. Test
Acceptance
7. Metrics
and
Reporting
8.
Implement,
Test,
Release
9. User
Measures
1. Goals and Needs
2. Data Analysis
3. Requirements
4. User Tasks
5. Design Solution
6. Test Acceptance
7. Metrics & Reporting
8. Implement, Test, Release
9. User Measures
10. Report and Advise

16. 10. Report
and Advise 1. Goals
and Needs
2. Data
Analysis
3.
Require-
ments
4. User
Tasks
5. Design
Solution
6. Test
Acceptance
7. Metrics
and
Reporting
8.
Implement,
Test,
Release
9. User
Measures
Combines
UX and
continuous
improvement
lifecycles

17. 10. Report
and Advise 1. Goals
and Needs
2. Data
Analysis
3.
Require-
ments
4. User
Tasks
5. Design
Solution
6. Test
Acceptance
7. Metrics
and
Reporting
8.
Implement,
Test,
Release
9. User
Measures
Familiar UX
lifecycle
elements

18. 10. Report
and Advise 1. Goals
and Needs
2. Data
Analysis
3.
Require-
ments
4. User
Tasks
5. Design
Solution
6. Test
Acceptance
7. Metrics
and
Reporting
8.
Implement,
Test,
Release
9. User
Measures
Familiar
continuous
improvement
lifecycle
elements

19. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
1.
1. Goals and Needs
Business and user objectives change.
Confirm stakeholder as well as audience
wants and needs.
UX Roles Tools Opportunities
Business Analyst
Compliance Analyst
Requirements Analyst
Content inventory
Surveys and polls
Stakeholder interviews
User feedback
User interviews
Create relationships with
stakeholders and users.
Think ongoing dialogue, not
one-time intelligence
gathering.

20. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
2.
2. Data Analysis
Review and assess metrics, user behavior,
and feedback to understand issues and
opportunities.
UX Roles Tools Opportunities
Business Analyst
Content Strategist
Information Architect
Metrics/Analytics Analyst
Search Engine Analyst
Usability Analyst/Tester
Ad hoc reporting
Analytics
Content inventory
Dashboards
Ranking
Scheduled reporting
Search logs and stats
Surveys and polls
Usage logs (web, social)
User feedback
Identify and codify
meaningful metrics, key
performance indicators
(KPIs), and success criteria.
Instill a data-driven culture
that relies on evidence-
based practices.

21. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
3.
3. Requirements
Identify business and user requirements.
Group by priority, create a roadmap, and
maintain a backlog.
UX Roles Tools Opportunities
Business Analyst
Requirements Analyst
User Interface Designer
Heuristic review
Issue tracking tools
Requirements standards
Top task list
Use cases
User stories
Tell the stakeholder’s and
user’s stories. Document
needs in a way that is
meaningful to all involved
parties. Use a backlog to
capture low-priority or wish
list items.

22. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
4.
4. User Tasks
Focus your efforts and avoid unnecessary
effort and functionality. Think of the
80/20 rule and focus on vital tasks.
UX Roles Tools Opportunities
Information Architect
Requirements Analyst
Search Engine Analyst
Usability Analyst/Tester
Analytics
Search logs and stats
Surveys and polls
Usage logs (web, social)
User feedback
User interviews
Identify then focus on key
tasks and eliminate
unnecessary complexity.
Understand related tasks
and identify efficiencies
wherever possible.

23. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
5.
5. Design Solution
Address IA, interaction design, functional
concerns, visual design, and technical
issues. Create artifacts or prototypes that
communicate the solution.
UX Roles Tools Opportunities
Content Strategist
Information Architect
User Interface Designer
Usability Analyst/Tester
Analog and digital design
Development frameworks
Rapid prototyping software
Style and brand guides
Design for simplicity,
usability, scanability,
minimal cognitive load.
Think about the larger
domain and a consistent
user experience.

24. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
6.
6. Test Acceptance
Change involves risk. Conduct tests to
confirm the proposed design satisfies
requirements and user tasks.
UX Roles Tools Opportunities
Information Architect
Requirements Analyst
User Interface Designer
Usability Analyst/Tester
AB/multivariate testing
Automated usability testing
Card sorts
First-click testing
Moderated usability testing
Tree tests
User feedback
Testing in development or
staging environments can
minimize risk in production.
Live testing can then refine
(rather than fix). Free and
low-cost testing options
make it easier to create a
testing culture.

25. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
7.
7. Metrics & Reporting
Use meaningful metrics and reporting
approaches. Use KPIs and try to provide
context. Use tags and technologies to
improve data collection and analysis.
UX Roles Tools Opportunities
Business Analyst
Compliance Analyst
Metrics/Analytics Analyst
Project Manager
Quality Analyst
Search Engine Analyst
Usability Analyst/Tester
Ad hoc reporting
Analytics
Dashboards
Ranking
Scheduled reporting
Search logs and stats
Surveys and polls
Usage logs (web, social)
User feedback
Monitor domain health and
evaluate changes regularly.
Quantify key elements of
user experience and
reinforce a data-driven
culture and reliance on
evidence-based practices.

26. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
8.
8. Implement, Test, Release
Follow a standardized implementation,
testing, and release steps. Repeatable
process reduces risk and simplifies
debugging and future enhancements.
UX Roles Tools Opportunities
Business Analyst
Compliance Analyst
Content Strategist
Information Architect
Quality Analyst
Requirements Analyst
Usability Analyst/Tester
Automated testing
Heuristic reviews
Issue tracking tools
Test cases and scripts
Top task list
Unit testing
Use cases
User stories
User tasks
Involving UX in
implementation, testing,
and release can help quickly
identify bugs and short-cut
fixes. It also provides an
excellent opportunity to
flesh-out the backlog and to
prime the next iteration.

27. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
9.
9. User Measures
Use measurement tools to understand
adoption, customer satisfaction,
conversion, or other success measures
identified in step 7.
UX Roles Tools Opportunities
Metrics/Analytics Analyst
Quality Analyst
Search Engine Analyst
Usability Analyst/Tester
AB/multivariate testing
Analytics
Automated usability testing
Eye tracking
Heat mapping
User feedback
It is impossible for a UX
professional to completely
intuit user’s experiences;
measurement fills in the
gaps and identifies new or
hereto unknown
considerations.

28. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
10.
10. Report and Advise
Develop and discuss reported analysis
findings. Generally, it will be beneficial to
measure against baselines. Trends may
indicate more than a one-time snapshot.
UX Roles Tools Opportunities
Metrics/Analytics Analyst
Project Manager
Quality Analyst
Search Engine Analyst
Usability Analyst/Tester
Ad hoc reporting
Change control board
Dashboards
Scheduled reporting
State of the “…” reporting
Grow the relationships with
stakeholders and users by
continuing the dialogue.

29. 10.
9.
8.
7.
6. 5.
4.
3.
2.
1.
1.
1. Goals and Needs
Next iteration.
There is always room for improvement
and we have the success metrics to prove
value every step of the way.

30. Don’t worry, I’m not trying to lay some Sisyphus trip on you.

31. Instead, I’m trying to…

32. Promote a culture of
continuous improvement
and optimization…
• Selling optimization
• Selling continuous improvement
• Smaller improvements, but faster
• UX involvement at every project phase
• Evidence- and data-based approaches
• Easier tracking and measurement
• Improved governance
• Less rework
• Fewer redesigns

34. Take advantage of opportunities associated with
continuous improvement/optimization approaches:
• For the external UX consultant
• For embedded UX teams and project staff
• For project managers (especially of UX staff)
• For business development and posture
• For UX ‘insertion points’ throughout the SDLC and content lifecycles

35. Thank you!
And if you liked this presentation or have
any feedback, please don’t keep it to
yourself. This is a work in progress and any
input is appreciated.
Jeffrey Ryan Pass, Solution Architect & Lead User Experience
Consultant
Aquilent (we’re hiring)
jeff.pass@aquilent.com | @jeffpass | about.me/jeffpass

36. Image Sources
Debra Martin – Debra Martin & LinkedIn
https://media.licdn.com/media/p/2/000/038/101/09ac576.jpg
Brownies on the dock – Jeffrey Ryan Pass
Lost Highway – David Lynch & Timeout
http://now-here-this.timeout.com/wp-content/uploads/2014/03/lynch.jpg
Preparing to carve pumpkins – Jeffrey Ryan Pass
Shorted Oxford English Dictionary (5th Edition, A-M) – Jeffrey Ryan Pass
The Big Bang – NASA & GSFC
http://cosmictimes.gsfc.nasa.gov/universemashup/archive/complete_archive.html
Sesame Street – Children's Television Workshop
http://www.themarysue.com/wp-content/uploads/2011/04/sesamestreet.jpg
Binary Code – Wikimedia Commons
http://commons.wikimedia.org/wiki/File:Binary_Code.jpg
Le moulin de l’oubli (1999) – Gilbert Garcin
http://imgur.com/gallery/nyrpIEl
Playtime – Jacques Tati
http://acbatesy.com/2013/02/25/great-big-70mm-film-festival-highlights-from-chicago/urlot
Shamwari Wildlife: Dung Beatle – Rhino Africa Safaris
http://www.rhinoafrica.com/blog/2013/04/18/rhino-africa-in-talk-with-adrian-gardiner/
Bed - From Einstein on the Beach 2014 – Robert Wilson & Philip Glass & Brooklyn Academy of Music (BAM)
http://vimeo.com/user1387873
Furnityre disassembly + installation – AMG - Argosy Management Group
http://argosymg.com/furniture-disassembly-and-installation.php
'This American Life' host Ira Glass in 2012 – Chicago Tribune & Stephen J. Cohen/Getty Images
http://www.trbimg.com/img-5385c8de/turbine/chi-chicago-public-media-this-american-life-20-001/1200/1200x936
Opportunity views the empty lander, the Challenger Memorial Station – NASA & JPL & Cornell
http://en.wikipedia.org/wiki/Opportunity_%28rover%29#mediaviewer/File:Challenger_Memorial_Station_At_Meridiani_Planum.jpg
Jeff Waters – Jeffrey Ryan Pass & Movember