Here is how we approach experiments @ Vinted as well as what we have learnt during few years of practising it.

1. SPEAKER
What we have learnt @Vinted
Experiments
Tautvydas Gylys

2. Experiments - scientific
approach
Observe > Hypothesis > Test > Prove / Reject
photo:

3. 2 types of experimentation
Conversion optimisation Hypothesis driven development
To improve smth To create smth new
Will be talking about today

4. How do we experiment
@Vinted?
case study: Conversion to lister optimisation

5. case study: Conversion to lister optimisation

6. case study: Conversion to lister optimisation

7. 2. Ideas and assumptions
Data analysis

8. 2. Ideas and assumptions
Interviews
Data analysis

9. 2. Ideas and assumptions
Interviews
Data analysis
Brainstorm

10. case study: Conversion to lister optimisation

11. case study: Conversion to lister optimisation

12. 4. Experiment design

13. 4. Experiment design

14. case study: Conversion to lister optimisation

15. 7. Analysis and documentation
Dashboard
v1

16. Dashboard
v2
7. Analysis and documentation

17. results: Listers flow optimisation
~30 ideas
7 experiments
2 implemented
Learning!

18. Required registration
example

19. Conversion to lister: -
Enter product > Register > Listing form

20. Conversion to lister:
Enter product > Listing form > Register on upload

21. Conversion to lister:
Register > Enter product > Listing form

22. Conversion to lister:
Required registration V2

23. Required registration V2
Conversion to lister:
Take a way: in apps members are willing to invest time upfront to register. That way
they feel invested and explore the product with more attention.

24. Lessons learned

25. Idea: change to
“Buy now”
Goal: to improve
conversion to buyer
Is it a good idea?

26. Idea: change to
“Buy now”
Goal: improve
conversion to buyer
Goal to increase
conversion by 3%
Goal to increase
conversion by 2x
Yes!
No!
Is it a good idea?

27. Idea: change to
“Buy now”
Goal: improve
conversion to buyer
Goal to increase
conversion by 3%
Goal to increase
conversion by 2x
Yes!
No!
Is it a good idea?
Always have measurable goals before designing an experiment.

28. How many variables?
Control: Not required Test: Required

29. How many variables?
Control: Not required Test: Required
Picture
Copy
CTA
Screen size
Screen number

30. Control
Modal screen
/
Not required
Test A
Modal screen
/
Required
Test B
Full screen
/
Not required
Test C
Full screen
/
Required

31. Control
Modal screen
/
Not required
Test A
Modal screen
/
Required
Test B
Full screen
/
Not required
Test C
Full screen
/
Required
Be aware of how many variables you are testing.

32. Experiment culture changes
• Before: Dedicated experiment team (I have an idea, let’s test it!)
• Now: Each team runs experiments (I have a problem which potential solution I test)

33. Experiment culture changes
• Before: Dedicated experiment team (I have an idea, lets test it!)
• Now: Each team runs experiments (I have a problem which potential solution I test)
• Before: Test everything
• Now: Test only big things, trust gut feeling for small

34. Experiment culture changes
• Before: Dedicated experiment team (I have an idea, lets test it!)
• Now: Each team runs experiments (I have a problem which potential solution I test)
• Before: Test everything
• Now: Test only big things, trust gut feeling for small
• Before: Data only + Gut feeling
• Now: Data + Interviews + Gut feeling

35. Experiment culture changes
• Before: Dedicated experiment team (I have an idea, lets test it!)
• Now: Each team runs experiments (I have a problem which potential solution I test)
• Before: Test everything
• Now: Test only big things, trust gut feeling for small
• Before: Data only + Gut feeling
• Now: Data + Interviews + Gut feeling
• Before: Dig deep to learn the impact
• Now: Fail fast

36. Thanks!