These are the slides of the talk Giovanni Puliti and I gave at Intersection Conference 2019 in Milan. We talked about retrospective and predictive metrics and tools to support Product Owners, Designers and Developers to increase the value to be delivered. Generally, the value of a product is represented by the balance between business outcomes and users’ needs. But how is this value actually measured? How is the user satisfaction measured? How can Product Owners, Designers and Developers validate their hypotheses regarding the product strategy? How can the team members agree that for each user story moved to “done” the next one moved to “doing” will increase the quality of the product and therefore the user satisfaction? In our talk, we gave some tips to measure the User Experience and help the team in finding the right answers to the questions above.

1. Measuring the user experience
and the product value
October 2nd 2019, Milan

2. Ilaria Mauric
Head of design
@ilariamauric
Giovanni Puliti
Agile coach and Trainer
@giovannipuliti

3. It’s undone
when it’s done.

4. It’s undone
when it’s done.

5. It’s undone
when it’s done.

6. It’s done when it’s done.
(aka “it works for me”)

7. Extended lifecycle
BUSINESSPROFITS
Development Introduction Growth Maturity Decline
TIME

8. from
backlog compiler
from “it’s done
when it’s done”
to
product strategist
to “create something
useful and valuable”

9. We need to measure
in a different way.

10. Direct external measurements
Indirect internal measurements

11. Indirect internal
measurements

12. Indirect internal measurement
Maintenance VS Dev

13. Indirect internal measurement
Lead time, cycle time, etc.
NUMBEROFACTIVITIES
DAYS, WEEKS, SPRINTS
Backlog
Requirements gathering
Development
Test
Deploy

14. No… I gave up by now…. When I call I’m
kept on hold, no one ever answers.
I don’t find you in the list…..
Did you call to reschedule your appointment?
We are here to answer every morning.
I don’t doubt it… but I hold indefinitely
and sometimes I got dropped.
You tell me if there’s another service
available every morning for 4 hours….
I don’t think that’s a metric….

15. Direct external
measurements.
Getting out of the
building.

17. 489 €
~650 €

19. B2 C
B2 B2 C
B2 Health Service (C)2 U

20. More a KPI than a User?
‣ Patients per room
‣ Staff-to-patient ratio
‣ Bed or room turnover
‣ Patient Waiting-Times By Process Step
‣ Time Between Symptom Onset & Hospitalisation
‣ Average Insurance Claim Processing Time & Cost
‣ …
Source: https://www.clearpointstrategy.com/25-healthcare-metrics-kpis/

21. Define which data you want to use.
Define how to use them.
Be open to review them.

22. What do we need?
exploratory
WHY
innovation
evaluative
HOW
maintenance
SolutionProblem

23. Qualitative design research
Through live observation, face-to face interviews and stories,
it aims to understand user’s behaviours, motivations, needs, pains,
emotions and real contexts.

24. Quantitative design research
Through numbers and graphs,
it gives a picture of the magnitudo of the “nitty-gritty” facts,
their distribution, their frequency, their tendency.

25. Qualitative
design research
Quantitative
design research

26. Measuring UX in a scrum team, by Mr. David Travis at BAD Conference 2019

27. 1
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG

28. 1
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG
FEEDBACK

29. 1
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG
FEEDBACK

30. 2
VISION,
USER NEEDS,
OPPORTUNITIES
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG
FEEDBACK

31. 2
VISION,
USER NEEDS,
OPPORTUNITIES
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG
FEEDBACK

32. FEEDBACK
3
VISION,
USER NEEDS,
OPPORTUNITIES
ITERATIVE
PRODUCT
DEVELOPMENT
BACKLOG
DELIVERED
VALUE
MEASURE

33. Framing a problem:
We believe that improving the product water resistancy
will help patients using it when swimming or training.
KPI: product usage over summertime
today: -40% - goal: same as winter 💪
Testing a solution:
As a T1D patient I want to edit carbos so that
I can correct them in case I forgot something I ate
KPI: ratings and reviews on the App Store
today: 1,8 - goal: 2 💪

35. References
[ITA] Product ownership e misurazione del valore del
prodotto
http://bit.ly/ABD19POUX
Quantitative versus Qualitativa Usability Testing
https://www.nngroup.com/articles/quant-vs-qual/
[BOOK] Observing the User Experience (2012)
https://www.amazon.it/Observing-User-Experience-
Practitioners-Research/dp/0123848695
Thank you! 👋

36. Thanks to our sponsor
Diamond
PLATINUM
Gold
silver
media
community
Intersection conference by