Presentation at Digital Elite Camp 2017 in Tallinn Estonia on June 17th about creating a company culture favorable to hypothesis experimentation, A/B testing and user research.

1. Optimization culture
@guido | x@gui.do​
a merchant's story

2. “ Sadly Thursday wasn't as good as we wanted in
terms of volumes...​
Standard deveation
...and we are now 20 orders behind target.​​
Issue seems to be related to CR. We are bringing more
traffic than last year but it's not converting as well.
= 46.9 orders

3. 200 Employees
35 Online
4+2 Optimization

4. Traditional roots
Online & Offline
B2C & B2B​​​

5. Started CRO
24 months ago​​​

6. Great KPI levels
for experimentation

7. Every test
UX research
UX design
Frontend developer
Web/data analyst
Psychologist
E-commerce manager
Optional
​Customer Service
Business intelligence/
Finance
Business sales (florists)
​External
People/skills involved

8. Customer journey phase
optimization couples:
1x Optimization team
1x Business user​​

9. Survey​

10. Q1: Company type​
Startup, online
Traditional, online + offline

11. Q2: How old is your
testing program?​
< 1 year​
1-3 years​
3 > years​
You work at Booking (10 > years)​

12. Q3: How big is
the testing team?​
< 1 (it's you!)
5 >
Multiple teams

13. Q4: MGMT support​
CEO is really enthusiastic about
testing
It's more a small (side) project

14. Q5: How many experiments
did you run this year?​
< 10
10 - 50
50 >
You work at Booking ( 10.000 > )

15. Platform
People
Process

16. Why do you need culture anyway?
Experimentation doesn't
come natural to everyone
Lack of trust/understanding about
data/stats/experiments undermines
your authority
Silo’s don’t let you connect the dots

17. Successful optimization cultures...​
Broad data literacy, leadership & trust
Inquisitive, questioning culture
Iterative, learning culture
Bottom-up (Anti-HIPPO)
Share learnings widely
Clear north star goal

18. ​Things that didn't work
Own internal "Which Test Won"
Tell once, expect everyone to "get it"
Getting older, offline people on-board
Employee form for experiment ideas

19. ​Things that do work
Take care of your shit umbrella
Explain all research/experiment learnings
Transparent workflow tool
Calculate ROI in kr./€/$
Shared cross-team KPIs

20. ​Things I hope will work
Onboarding for people
outside the online team
Move away from "CRO" and "A/B testing",
move to "Idea validation"
​Quantifying every step in
the customer journey

21. Final thought(s)

22. Create the shortest single RegEx that
shows all order confirmation pages
regardless of language, platform and
TLD. Exclude error pages.
RegEx challenge

23. “ Last week I did the challenge and I came up with
.*.../((en|EN)/)?(Pages/)?(c|C)?onfirmation.*
...but I couldn't stop thinking about it, got a
breakthrough and made it much shorter:
[a-zA-Z]/(c|C)onfirmation
What do you think?

24. ​@guido
x@gui.do​
gui.do/onculture