SlideShare a Scribd company logo
1 of 20
Download to read offline
Design Research for a quality experience
National Software Testing Conference 2016
Carmen Brion
Tech Reliability and Usability are basic
requirements in a digital product for customers
To find competitive advantage we need to understand the specific
Jobs To Be Done that solve problems for our core audience
Lack of Design Research, or leaving it too late,
increases the risk of failure
IMVU, Eric Ries’ first startup, was
initially a failure because the team
never undertook any research to
assess the idea and strategy.
“Listen old man, you don’t
understand. What is the
deal with this crazy
business of inviting friends
before I know if it is cool”
Source: Lean Startup by Eric Ries
Continuous Design Research increases likelihood
of success, even in difficult markets
Intuit became the leader in the
difficult market of personal finance
by investing in understanding the
jobs to be done for their core
audience.
Quicken was the result of:
● early generative
research
● recurrent usability
testing while building
● public beta versions
before launch
Source: The Lean Product Playbook by Dan Olsen
Postponing or not doing Design Research will
affect the product metrics
Acquisition
Activation
Retention
Revenue
Referral
How do customers find you?
Do they have a great first experience?
Do they come back?
How do you make money?
Do they tell others?
Becomes harder and more expensive
Reduces brand loyalty and likelihood of retention
“If you make a customer unhappy in the physical
world they might tell 6 friends, if you do so on the
internet, they can tell 6,000” Jeff Bezos
Unhappy customers are less likely to come back
Increase of costs by rework and customer support
Effects of bad experiences
Source: Lean Startup Pirate Metrics (AARRR)
“Your job isn’t to build more software faster: it’
s to maximize the outcome and impact you get
from what you choose to build” Jeff Patton
● QA and Usability Testing just guarantee usable software outputs
● To meet customer outcomes we need continuous and adequate
Design Research
Customers buy and use a product because
it does a job for them
Job To Be Done (JTBD)
Context of use
Desired OUTCOME
Product/Service
● What outcome is the customer trying to
achieve?
● What are the problems and pain points
getting in the way of the outcomes?
● How can they achieve outcomes in a
way that also allows the business to
reach its goals?
Digital
Ability
Expectations from
other solutions
JTBD
Ability
Who is the audience?
What are their JTBD?
What are they currently using?
How can we do the job better?
Which solutions do the job?
Which solution does the job better?
Is the solution easy to use?
How effective is the solution at
doing the job?
Which is the most efficient
variant to get the job done?
Why might one variant work
better than the other(s)?
Evaluative research
Generative research
Product Exploration
To get outcomes we need Continuous Design Research
Product Development Product Validation
To get outcomes we need Continuous Design Research
Ethnography
Diary studies
Contextual
inquiry
Interviews
Co-creation
workshop
Tree test
Card sorts
Click tests
Prototype testing
MVT, A/B testing
Path analysis, Heatmaps
Recorded sessions
Feedback surveys
Usability testing
Generative research
Evaluative research
Usability testing
Surveys
Benchmark
Desk research
Analytics
Evidence-based Output Outcome
Product Development Product ValidationProduct Exploration
Research/Design Framework
DEFINE
SOLUTION
ASSESSMENT
SOLUTION
EXPLORATION
PROBLEM
DEFINITION
PROBLEM
EXPLORATION
HYPOTHESES AND
JOBS TO BE DONE
DEFINITION
“Few ideas work on the first try. Iteration is key to innovation”
― Sebastian Thrun
“Design Research is
a team sport” GDS
1 32 4 5 6 7 8 9
Kick off meeting
Determine the
assumptions and
questions
Our research process for prototype testing
Prototype definition
Determine and agree
what to prototype
Recruitment spec
Agree participants
profiles and start
recruitment
Sessions plan
Define all hypotheses to
test
Prototype reviews
The core team assesses
progress and discusses
any changes
Discussion guide
Determine and
triangulate tasks
and questions for
participants
Run the sessions
Team and stakeholders
practice ‘Active
Observation’
Sessions washup
Review and agree the
main themes and insights
with the core team
Insights presentation
Present insights and
hypotheses outcomes
to the core team and
stakeholders
2 weeks 1 week
“[Both QA and UAT] follow repetitive and set
user paths that assume a level of rational
predictability which is rarely present in human
behaviour”
Jamie Mayers, Made by Many
How can we cater for the user paths we may not have
predicted?
QA that goes beyond checking that technical
requirements have been met
Product Development
USABILITY TESTING and QA
UAT
● Work with UX and Development to prioritise bug fixes taking into account
potential benefits to the personas and the impact to their JTBD
● Get ideas from the following in order to define test cases:
○ using the product personas to understand the context of use, JTBD,
ability and level of expertise
○ observe usability sessions to gather inspiration on how customers
use the product and potential unusual paths/usage
Product ValidationProduct Exploration
UAT that goes beyond an acceptable system
USABILITY TESTING and QA
UAT
Business Analysts can collaborate with UX to:
● Define key User Journeys to assess
● Define an experience rating that includes how well the product meets:
○ the experience principles
○ the JTBD for each User Journey
Product Development Product ValidationProduct Exploration
But it is really unlikely that we can
predict all unusual paths that
people might take
Give the product to your customers,
let them play, see where it breaks for
them
Photo by Storem
Product Development
USABILITY TESTING and QA
UAT
Product Validation
Evidence-based Output Outcome
Let your customers be the product testers
MVT, A/B testing
Path analysis, Heatmaps
Recorded sessions
Usability testing
Run usability sessions and complement with a Beta test or a product
validation phase.
“Alone we can do so little; together
we can do so much”
- Helen Keller
Photo by Easa Shamih
Thank you!
You can find me @Tea_monster
References
Irrational realism: on QA vs Testing, by Jamie Mayes
https://medium.com/the-many/irrational-realism-on-qa-vs-testing-7fd8618f7fdb#.q9b3wxosm
It’s not just semantics: managing outcomes vs outputs, by Deborah Mills-Scofield
https://hbr.org/2012/11/its-not-just-semantics-managing-outcomes
The neglected practice of iteration, by Jeff Patton
https://www.agileconnection.com/article/neglected-practice-iteration
Early usability testing decides the product’s destiny, by Tamara Rendulic
http://www.paulolyslager.com/early-usability-testing-decides-product-destiny/?
utm_content=buffer4482d&utm_medium=social&utm_source=twitter.
com&utm_campaign=buffer

More Related Content

What's hot

Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...
Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...
Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...Business of Software Conference
 
Introduction to Jobs to Be Done
Introduction to Jobs to Be DoneIntroduction to Jobs to Be Done
Introduction to Jobs to Be DoneMichele Hansen
 
Jobs-To-Be-Done Framework - An Introduction
Jobs-To-Be-Done Framework - An IntroductionJobs-To-Be-Done Framework - An Introduction
Jobs-To-Be-Done Framework - An IntroductionRizwan Javaid
 
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be Done
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be DoneBoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be Done
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be DoneBusiness of Software Conference
 
Jobs to Be Done Template
Jobs to Be Done TemplateJobs to Be Done Template
Jobs to Be Done TemplateSoren Kaplan
 
Product mentor jobs to be done session 3 amanda ralph 240615
Product mentor jobs to be done session 3 amanda ralph 240615Product mentor jobs to be done session 3 amanda ralph 240615
Product mentor jobs to be done session 3 amanda ralph 240615Amanda Ralph
 
Breakthrough Innovation with Jobs To Be Done
Breakthrough Innovation with Jobs To Be DoneBreakthrough Innovation with Jobs To Be Done
Breakthrough Innovation with Jobs To Be DoneClaudio Perrone
 
Jobs to be Done - Outcome-driven Innovation
Jobs to be Done - Outcome-driven InnovationJobs to be Done - Outcome-driven Innovation
Jobs to be Done - Outcome-driven InnovationThiago Bernardo Santos
 
Jobs to be Done Interview Technique
Jobs to be Done Interview TechniqueJobs to be Done Interview Technique
Jobs to be Done Interview TechniqueJason Hall
 
Practitioners guide to_product_strategy
Practitioners guide to_product_strategyPractitioners guide to_product_strategy
Practitioners guide to_product_strategyPandith Jantakahalli
 
Using Jobs to be Done to Create a Sustainable Growth Strategy
Using Jobs to be Done to Create a Sustainable Growth StrategyUsing Jobs to be Done to Create a Sustainable Growth Strategy
Using Jobs to be Done to Create a Sustainable Growth StrategyAlan Klement
 
Get More Traction for Your Product Using Jobs-To-Be-Done
Get More Traction for Your Product Using Jobs-To-Be-DoneGet More Traction for Your Product Using Jobs-To-Be-Done
Get More Traction for Your Product Using Jobs-To-Be-Donepascallaliberte
 
Jobs To Be Done Framework
Jobs To Be Done FrameworkJobs To Be Done Framework
Jobs To Be Done FrameworkRupali Shah
 
Berlin JTBD Meetup, March 24 2015 – On JTBD Interviews
Berlin JTBD Meetup, March 24 2015 – On JTBD InterviewsBerlin JTBD Meetup, March 24 2015 – On JTBD Interviews
Berlin JTBD Meetup, March 24 2015 – On JTBD InterviewsTor Bollingmo
 
Business Model Innovation via JTBD Methodology
Business Model Innovation via JTBD MethodologyBusiness Model Innovation via JTBD Methodology
Business Model Innovation via JTBD MethodologyPralabh Verma
 
Intro to Lean Startup and Customer Discovery for Agilists
Intro to Lean Startup and Customer Discovery for AgilistsIntro to Lean Startup and Customer Discovery for Agilists
Intro to Lean Startup and Customer Discovery for AgilistsShashi Jain
 
Jobs-To-Be-Done Theory
Jobs-To-Be-Done TheoryJobs-To-Be-Done Theory
Jobs-To-Be-Done TheoryAlex Kehr
 

What's hot (20)

Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...
Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...
Turn Jobs to Be Done Theory into Practice - Tony Ulwick at Business of Softwa...
 
Jobs to Be Done
Jobs to Be DoneJobs to Be Done
Jobs to Be Done
 
Introduction to Jobs to Be Done
Introduction to Jobs to Be DoneIntroduction to Jobs to Be Done
Introduction to Jobs to Be Done
 
Jobs-To-Be-Done Framework - An Introduction
Jobs-To-Be-Done Framework - An IntroductionJobs-To-Be-Done Framework - An Introduction
Jobs-To-Be-Done Framework - An Introduction
 
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be Done
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be DoneBoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be Done
BoSUSA13 | Bob Moesta Chris Spiek | Uncovering the Jobs to be Done
 
Jobs to Be Done Template
Jobs to Be Done TemplateJobs to Be Done Template
Jobs to Be Done Template
 
Product mentor jobs to be done session 3 amanda ralph 240615
Product mentor jobs to be done session 3 amanda ralph 240615Product mentor jobs to be done session 3 amanda ralph 240615
Product mentor jobs to be done session 3 amanda ralph 240615
 
Breakthrough Innovation with Jobs To Be Done
Breakthrough Innovation with Jobs To Be DoneBreakthrough Innovation with Jobs To Be Done
Breakthrough Innovation with Jobs To Be Done
 
Jobs to be Done - Outcome-driven Innovation
Jobs to be Done - Outcome-driven InnovationJobs to be Done - Outcome-driven Innovation
Jobs to be Done - Outcome-driven Innovation
 
Jobs to be Done Interview Technique
Jobs to be Done Interview TechniqueJobs to be Done Interview Technique
Jobs to be Done Interview Technique
 
Practitioners guide to_product_strategy
Practitioners guide to_product_strategyPractitioners guide to_product_strategy
Practitioners guide to_product_strategy
 
Using Jobs to be Done to Create a Sustainable Growth Strategy
Using Jobs to be Done to Create a Sustainable Growth StrategyUsing Jobs to be Done to Create a Sustainable Growth Strategy
Using Jobs to be Done to Create a Sustainable Growth Strategy
 
Get More Traction for Your Product Using Jobs-To-Be-Done
Get More Traction for Your Product Using Jobs-To-Be-DoneGet More Traction for Your Product Using Jobs-To-Be-Done
Get More Traction for Your Product Using Jobs-To-Be-Done
 
Market creation
Market creationMarket creation
Market creation
 
Jobs To Be Done Framework
Jobs To Be Done FrameworkJobs To Be Done Framework
Jobs To Be Done Framework
 
Berlin JTBD Meetup, March 24 2015 – On JTBD Interviews
Berlin JTBD Meetup, March 24 2015 – On JTBD InterviewsBerlin JTBD Meetup, March 24 2015 – On JTBD Interviews
Berlin JTBD Meetup, March 24 2015 – On JTBD Interviews
 
Personas 101
Personas 101Personas 101
Personas 101
 
Business Model Innovation via JTBD Methodology
Business Model Innovation via JTBD MethodologyBusiness Model Innovation via JTBD Methodology
Business Model Innovation via JTBD Methodology
 
Intro to Lean Startup and Customer Discovery for Agilists
Intro to Lean Startup and Customer Discovery for AgilistsIntro to Lean Startup and Customer Discovery for Agilists
Intro to Lean Startup and Customer Discovery for Agilists
 
Jobs-To-Be-Done Theory
Jobs-To-Be-Done TheoryJobs-To-Be-Done Theory
Jobs-To-Be-Done Theory
 

Similar to Design research for a quality product

Prototyping and Usability Testing your designs
Prototyping and Usability Testing your designsPrototyping and Usability Testing your designs
Prototyping and Usability Testing your designsElizabeth Snowdon
 
Leverage The Power of Small Data
Leverage The Power of Small DataLeverage The Power of Small Data
Leverage The Power of Small DataKaryn Zuidinga
 
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...BrittanyRubinstein
 
Usability Testing for Developers with No Budget
Usability Testing for Developers with No BudgetUsability Testing for Developers with No Budget
Usability Testing for Developers with No BudgetAshley Dzick
 
Product Development Demystified: Launching Faster with Confidence through Hum...
Product Development Demystified: Launching Faster with Confidence through Hum...Product Development Demystified: Launching Faster with Confidence through Hum...
Product Development Demystified: Launching Faster with Confidence through Hum...Aggregage
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely
 
How to Master Product Management Case Studies by fmr Groupon PM
How to Master Product Management Case Studies by fmr Groupon PMHow to Master Product Management Case Studies by fmr Groupon PM
How to Master Product Management Case Studies by fmr Groupon PMProduct School
 
NPD hypothesis validation toolkit
NPD hypothesis validation toolkit NPD hypothesis validation toolkit
NPD hypothesis validation toolkit CX NPD ltd
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye trackingObjective Experience
 
Usability Testing 101 - an introduction
Usability Testing 101 - an introductionUsability Testing 101 - an introduction
Usability Testing 101 - an introductionElizabeth Snowdon
 
How to Avoid Common Mistakes in Product by Cake Product Manager
How to Avoid Common Mistakes in Product by Cake Product ManagerHow to Avoid Common Mistakes in Product by Cake Product Manager
How to Avoid Common Mistakes in Product by Cake Product ManagerProduct School
 
Build Services that Fit: Product Discovery Lessons Learned
Build Services that Fit: Product Discovery Lessons LearnedBuild Services that Fit: Product Discovery Lessons Learned
Build Services that Fit: Product Discovery Lessons LearnedTremis Skeete
 
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventUsability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventKay Aubrey
 
Best Practices for Early Stage Product Development
Best Practices for Early Stage Product DevelopmentBest Practices for Early Stage Product Development
Best Practices for Early Stage Product DevelopmentAmr Basha
 
UX Lead Product Management
UX Lead Product ManagementUX Lead Product Management
UX Lead Product ManagementJohn Wyatt
 
How to Effectively Experiment in PM by LendingTree Sr PM
How to Effectively Experiment in PM by LendingTree Sr PMHow to Effectively Experiment in PM by LendingTree Sr PM
How to Effectively Experiment in PM by LendingTree Sr PMProduct School
 
Webinar: How to Conduct Unmoderated Remote Usability Testing
Webinar: How to Conduct Unmoderated Remote Usability TestingWebinar: How to Conduct Unmoderated Remote Usability Testing
Webinar: How to Conduct Unmoderated Remote Usability TestingUserZoom
 
Useagility Webinar - Automated User Testing
Useagility Webinar - Automated User TestingUseagility Webinar - Automated User Testing
Useagility Webinar - Automated User TestingUseagility
 

Similar to Design research for a quality product (20)

Prototyping and Usability Testing your designs
Prototyping and Usability Testing your designsPrototyping and Usability Testing your designs
Prototyping and Usability Testing your designs
 
Leverage The Power of Small Data
Leverage The Power of Small DataLeverage The Power of Small Data
Leverage The Power of Small Data
 
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...
Denver Startup Week 2019: Choosing a Direction Learning How to Test Ideas and...
 
Usability Testing for Developers with No Budget
Usability Testing for Developers with No BudgetUsability Testing for Developers with No Budget
Usability Testing for Developers with No Budget
 
Product Development Demystified: Launching Faster with Confidence through Hum...
Product Development Demystified: Launching Faster with Confidence through Hum...Product Development Demystified: Launching Faster with Confidence through Hum...
Product Development Demystified: Launching Faster with Confidence through Hum...
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation
 
How to Master Product Management Case Studies by fmr Groupon PM
How to Master Product Management Case Studies by fmr Groupon PMHow to Master Product Management Case Studies by fmr Groupon PM
How to Master Product Management Case Studies by fmr Groupon PM
 
NPD hypothesis validation toolkit
NPD hypothesis validation toolkit NPD hypothesis validation toolkit
NPD hypothesis validation toolkit
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye tracking
 
Usability Testing 101 - an introduction
Usability Testing 101 - an introductionUsability Testing 101 - an introduction
Usability Testing 101 - an introduction
 
Website Testing WINS!
Website Testing WINS!Website Testing WINS!
Website Testing WINS!
 
How to Avoid Common Mistakes in Product by Cake Product Manager
How to Avoid Common Mistakes in Product by Cake Product ManagerHow to Avoid Common Mistakes in Product by Cake Product Manager
How to Avoid Common Mistakes in Product by Cake Product Manager
 
Build Services that Fit: Product Discovery Lessons Learned
Build Services that Fit: Product Discovery Lessons LearnedBuild Services that Fit: Product Discovery Lessons Learned
Build Services that Fit: Product Discovery Lessons Learned
 
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventUsability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
 
Best Practices for Early Stage Product Development
Best Practices for Early Stage Product DevelopmentBest Practices for Early Stage Product Development
Best Practices for Early Stage Product Development
 
Mike Roe Creating Client Value
Mike Roe Creating Client ValueMike Roe Creating Client Value
Mike Roe Creating Client Value
 
UX Lead Product Management
UX Lead Product ManagementUX Lead Product Management
UX Lead Product Management
 
How to Effectively Experiment in PM by LendingTree Sr PM
How to Effectively Experiment in PM by LendingTree Sr PMHow to Effectively Experiment in PM by LendingTree Sr PM
How to Effectively Experiment in PM by LendingTree Sr PM
 
Webinar: How to Conduct Unmoderated Remote Usability Testing
Webinar: How to Conduct Unmoderated Remote Usability TestingWebinar: How to Conduct Unmoderated Remote Usability Testing
Webinar: How to Conduct Unmoderated Remote Usability Testing
 
Useagility Webinar - Automated User Testing
Useagility Webinar - Automated User TestingUseagility Webinar - Automated User Testing
Useagility Webinar - Automated User Testing
 

Recently uploaded

UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 

Recently uploaded (19)

UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 

Design research for a quality product

  • 1. Design Research for a quality experience National Software Testing Conference 2016 Carmen Brion
  • 2. Tech Reliability and Usability are basic requirements in a digital product for customers To find competitive advantage we need to understand the specific Jobs To Be Done that solve problems for our core audience
  • 3. Lack of Design Research, or leaving it too late, increases the risk of failure IMVU, Eric Ries’ first startup, was initially a failure because the team never undertook any research to assess the idea and strategy. “Listen old man, you don’t understand. What is the deal with this crazy business of inviting friends before I know if it is cool” Source: Lean Startup by Eric Ries
  • 4. Continuous Design Research increases likelihood of success, even in difficult markets Intuit became the leader in the difficult market of personal finance by investing in understanding the jobs to be done for their core audience. Quicken was the result of: ● early generative research ● recurrent usability testing while building ● public beta versions before launch Source: The Lean Product Playbook by Dan Olsen
  • 5. Postponing or not doing Design Research will affect the product metrics Acquisition Activation Retention Revenue Referral How do customers find you? Do they have a great first experience? Do they come back? How do you make money? Do they tell others? Becomes harder and more expensive Reduces brand loyalty and likelihood of retention “If you make a customer unhappy in the physical world they might tell 6 friends, if you do so on the internet, they can tell 6,000” Jeff Bezos Unhappy customers are less likely to come back Increase of costs by rework and customer support Effects of bad experiences Source: Lean Startup Pirate Metrics (AARRR)
  • 6. “Your job isn’t to build more software faster: it’ s to maximize the outcome and impact you get from what you choose to build” Jeff Patton ● QA and Usability Testing just guarantee usable software outputs ● To meet customer outcomes we need continuous and adequate Design Research
  • 7. Customers buy and use a product because it does a job for them Job To Be Done (JTBD) Context of use Desired OUTCOME Product/Service ● What outcome is the customer trying to achieve? ● What are the problems and pain points getting in the way of the outcomes? ● How can they achieve outcomes in a way that also allows the business to reach its goals? Digital Ability Expectations from other solutions JTBD Ability
  • 8. Who is the audience? What are their JTBD? What are they currently using? How can we do the job better? Which solutions do the job? Which solution does the job better? Is the solution easy to use? How effective is the solution at doing the job? Which is the most efficient variant to get the job done? Why might one variant work better than the other(s)? Evaluative research Generative research Product Exploration To get outcomes we need Continuous Design Research Product Development Product Validation
  • 9. To get outcomes we need Continuous Design Research Ethnography Diary studies Contextual inquiry Interviews Co-creation workshop Tree test Card sorts Click tests Prototype testing MVT, A/B testing Path analysis, Heatmaps Recorded sessions Feedback surveys Usability testing Generative research Evaluative research Usability testing Surveys Benchmark Desk research Analytics Evidence-based Output Outcome Product Development Product ValidationProduct Exploration
  • 10. Research/Design Framework DEFINE SOLUTION ASSESSMENT SOLUTION EXPLORATION PROBLEM DEFINITION PROBLEM EXPLORATION HYPOTHESES AND JOBS TO BE DONE DEFINITION “Few ideas work on the first try. Iteration is key to innovation” ― Sebastian Thrun
  • 11. “Design Research is a team sport” GDS
  • 12. 1 32 4 5 6 7 8 9 Kick off meeting Determine the assumptions and questions Our research process for prototype testing Prototype definition Determine and agree what to prototype Recruitment spec Agree participants profiles and start recruitment Sessions plan Define all hypotheses to test Prototype reviews The core team assesses progress and discusses any changes Discussion guide Determine and triangulate tasks and questions for participants Run the sessions Team and stakeholders practice ‘Active Observation’ Sessions washup Review and agree the main themes and insights with the core team Insights presentation Present insights and hypotheses outcomes to the core team and stakeholders 2 weeks 1 week
  • 13. “[Both QA and UAT] follow repetitive and set user paths that assume a level of rational predictability which is rarely present in human behaviour” Jamie Mayers, Made by Many How can we cater for the user paths we may not have predicted?
  • 14. QA that goes beyond checking that technical requirements have been met Product Development USABILITY TESTING and QA UAT ● Work with UX and Development to prioritise bug fixes taking into account potential benefits to the personas and the impact to their JTBD ● Get ideas from the following in order to define test cases: ○ using the product personas to understand the context of use, JTBD, ability and level of expertise ○ observe usability sessions to gather inspiration on how customers use the product and potential unusual paths/usage Product ValidationProduct Exploration
  • 15. UAT that goes beyond an acceptable system USABILITY TESTING and QA UAT Business Analysts can collaborate with UX to: ● Define key User Journeys to assess ● Define an experience rating that includes how well the product meets: ○ the experience principles ○ the JTBD for each User Journey Product Development Product ValidationProduct Exploration
  • 16. But it is really unlikely that we can predict all unusual paths that people might take Give the product to your customers, let them play, see where it breaks for them Photo by Storem
  • 17. Product Development USABILITY TESTING and QA UAT Product Validation Evidence-based Output Outcome Let your customers be the product testers MVT, A/B testing Path analysis, Heatmaps Recorded sessions Usability testing Run usability sessions and complement with a Beta test or a product validation phase.
  • 18. “Alone we can do so little; together we can do so much” - Helen Keller Photo by Easa Shamih
  • 19. Thank you! You can find me @Tea_monster
  • 20. References Irrational realism: on QA vs Testing, by Jamie Mayes https://medium.com/the-many/irrational-realism-on-qa-vs-testing-7fd8618f7fdb#.q9b3wxosm It’s not just semantics: managing outcomes vs outputs, by Deborah Mills-Scofield https://hbr.org/2012/11/its-not-just-semantics-managing-outcomes The neglected practice of iteration, by Jeff Patton https://www.agileconnection.com/article/neglected-practice-iteration Early usability testing decides the product’s destiny, by Tamara Rendulic http://www.paulolyslager.com/early-usability-testing-decides-product-destiny/? utm_content=buffer4482d&utm_medium=social&utm_source=twitter. com&utm_campaign=buffer