This document outlines strategies for optimizing growth through conversion rate optimization (CRO) testing. It discusses: 1. CRO should focus on growth, not conversion rates alone. Goals and metrics like average uplift per test and number of tests per year should be set. 2. Good optimization ideas come from understanding user behavior through both quantitative data and qualitative research. A testing framework assesses how changes will impact user perception and behavior. 3. When testing, priorities should be set for hypotheses and economic metrics measured over statistical significance. Optimization is about effectiveness, not random testing.

1. © Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
Uh Oh.
The Million Dollar
Optimization Strategy

2. © Andre Morys, Web Arts AG www.web-arts.com3
André Morys
@morys

3. © Web Arts AG www.web-arts.com
5.8 bn ! Retail / Lead-Volume
4
0
1,5
3
4,5
6
2008 2010 2012
CRO Volumen (Mrd. !)
@morys

4. © Web Arts AG www.web-arts.com
115 Conversion Experts in 15 Offices worldwide
5
Web Arts is founder of the
worldwide alliance of leading
conversion optimization
companies
@morys

5. © Andre Morys, Web Arts AG www.web-arts.com
Agenda today
6
1.Don‘t call it „Conversion Rate Optimization“
2.Optimization is not Testing.
3.Don‘t let the numbers lie to you. (Case Study)
@morys

6. © Andre Morys, Web Arts AG www.web-arts.com
Growth of Top 10 e-Commerce Ventures in 5 Years (DE)
15

7. © Andre Morys, Web Arts AG www.web-arts.com16
Part I:
It‘s about Growth, stupid.

8. Who has a business plan for CRO?

9. © Andre Morys, Web Arts AG www.web-arts.com18
Do You Have a Business Plan for Your CRO Strategy?
@morys

10. © Andre Morys, Web Arts AG www.web-arts.com
Profit Margin
20
Opportunity Cost:
! 500‘ Profit
@morys

11. © Andre Morys, Web Arts AG www.web-arts.com
Challenge needed?
21
@morys

12. © Andre Morys, Web Arts AG www.web-arts.com
Compare the effect of two optimization strategies:
22
@morys

13. © Andre Morys, Web Arts AG www.web-arts.com
Profit Margin
24
Opportunity Cost:
! 2.2mProfit
@morys

14. © Web Arts AG www.web-arts.com
Now you can re-invest ! 2.2m in
optimization...
25
@morys

15. © Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #1
‣It‘s not about conversion rates. It‘s about
GROWTH.
‣ Set your goals properly.
‣ Measure your optimization efficiency.
‣ Average Uplift per Test
‣ Amount of Tests per Year
‣ Cumulated Uplift per Year
‣ Test Success Rate
‣ ROI per Test (%)
26
@morys

16. © Andre Morys, Web Arts AG www.web-arts.com27
„If you
double the number
of experiments
you do per year you're going to
double your
inventiveness“
Jeff Bezos, CEO amazon, 2004

17. © Web Arts AG www.web-arts.com
0
300
600
900
1200
2008 2009 2010 2011 2012
The Zalando Effect?
28
@morys

18. © Web Arts AG www.web-arts.com
So, what‘s the problem?
29

19. © Andre Morys, Web Arts AG www.web-arts.com
For a 25% Compounded Global CR Increase, You Need...
30
2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 2.5 10 Tests à 2.5%?
5 5 5 5 5 5 5 5 5 5 10 Tests à 5%?
5 5 5 5 5
20 Tests à 5%
and 50%
success rate?5 5 5 5 5
Problem #1:
Organizations are too lame
to execute 20 tests per year.
Problem #2:
This is really expensive and
complex. Organizations hate
both.
Problem #3:
Most CMOs still prefer to
buy 25% more traffic.
@morys

20. © Andre Morys, Web Arts AG www.web-arts.com
Why only 5% average uplift per test?
32
‣In the last case study, they talked about 45%
Uplift!
-> E-Commerce conversion rate is not email
subscription or download CR.
‣My last test had 33% Uplift!
-> Was it real? Don‘t confuse statistical
significance with validity. Tests should run
around 4 weeks for a sufficient sample size.

21. © Andre Morys, Web Arts AG www.web-arts.com33
A/B Test Result
Reality
„I can‘t feel the uplift!“
@morys

22. © Andre Morys, Web Arts AG www.web-arts.com34
Testresults - Significance versus Validity
1 week
500 vs. 550 conversions
+10% Uplift
significant - wohoo!
2 weeks
1000 vs. 1030 conversions
+3% Uplift
not significant - :-(
Run your experiments longer to get valid insights!
@morys

23. © Andre Morys, Web Arts AG www.web-arts.com43
A/B
Testing
70% don‘t know that there is a Phase 2
20% have no answer
10% have an idea how to reach Phase 3
@morys

24. © Andre Morys, Web Arts AG www.web-arts.com
KPI: Average E-Commerce CR Uplift per Template Type
44
0 %
2,5 %
5 %
7,5 %
10 %
Homepage Category Product Page Cart Checkout Site Wide
9,81 %
7,3 %
6,59 %
9,28 %
6,35 %
3,32 %
Average Uplift by Template Type (n= 134)
@morys

25. © Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #2
‣Growth is about effectiveness.
‣ To increase real CR, you need many really good test results.
‣ Average Uplift per Test influences your growth directly.
‣ So: Do the right things!
‣ Don‘t do it too fast.
‣ Don‘t test everything. It is expensive.
45
@morys

26. © Andre Morys, Web Arts AG www.web-arts.com46
Part II:
Where do good optimization ideas
come from?
@morys

27. © Andre Morys, Web Arts AG www.web-arts.com47

28. © Andre Morys, Web Arts AG www.web-arts.com50
where would you buy?
why?

29. © Andre Morys, Web Arts AG www.web-arts.com51

30. © Andre Morys, Web Arts AG www.web-arts.com52
quantitative qualitative
How much? Why?
♥︎

31. © Andre Morys, Web Arts AG www.web-arts.com53
Traffic
Website
Web
Analytics

32. © Andre Morys, Web Arts AG www.web-arts.com54
Traffic
Website
Web
Analytics
Click
Hand
Motor Control
prefrontal Cortex
Limbic System
Perception

33. © Andre Morys, Web Arts AG www.web-arts.com
Test: facebook Acount Deaktivierung
58
@morys

34. ROI
„I have to buy it.“
Brand, Habits
„I want to buy it.“
Motivation, Emotion
„I can buy it.“
Usability, Accessability,
Visibility
Focus on Consumer Behavior
@morys

35. © Web Arts AG www.web-arts.com60
Factors, that influence the chance of conversion, can be
identified with a conversion framework
7E Framework
@morys

36. You are too dumb to
buy here.

37. @morys

38. @morys

39. @morys

40. © Web Arts AG www.web-arts.com81
Conversion Optimization means:
„Get a relevant unique value
proposition in the users mind.“
good feeling
@morys

41. © Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #3
82
‣It is not hard to implement a testing tool. It‘s hard
to find the right things to optimize.
‣ Find out, what really motivates users.
‣ Use different qualitative methods, like user research, personas, etc.
@morys

42. © Andre Morys, Web Arts AG www.web-arts.com
Conversion DMAIC®
Lean Innovation Process by Web Arts AG www.web-arts.com
83
quantitative
qualtitative
LEARN
BACKLOG
CONTROL
IMPROVE ANALYSE
DEFINE
MEASURE
@morys

43. © Andre Morys, Web Arts AG www.web-arts.com84
#1
Is the change bold enough
that it will be perceived by
users?
#2
Does the variation have
enough contrast to change
the behavior of users?
#3
Does it use psychological /
social principles?
#4
Is it situated on the primary
path of conversion?
@morys

44. © Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #4
85
‣Always prioritize your testing hypothesis!
‣ Optimization is not trial and error.
@morys

45. © Andre Morys, Web Arts AG www.web-arts.com86
Part III:
Measure The Right Stuff.
@morys

46. © Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
Control Template Discount Value Prop.
143 %
114 %113 %
100 %
Visitor - Order Conversion (relative)

47. © Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
Control Basis Preis QualitätControl Template Discount Value Prop.
291 %
204 %
243 %
100 %
Return rate (relative)

48. © Andre Morys, Web Arts AG www.web-arts.comFRANKFURT - HAMBURG - MÜNCHEN
Control Basis Preis QualitätControl Basis Preis QualitätControl Template Discount Value Prop.
122,2 %
87,1 %
102,3 %
100 %
Profit Contribution (relative)

49. © Andre Morys, Web Arts AG www.web-arts.com
More-Growth-Through-Testing-Rule #5
96
‣Measure economical KPIs.
‣ Don‘t try to get them statistical significant.
‣ Conversion goal = organisations goal
@morys

50. © Andre Morys, Web Arts AG www.web-arts.com
Five Pillars of Growth Strategy:
1. CRO -> Growth.
2. Efficiency -> Effectiveness.
3. Prioritize Hypotheses.
4. Motivate users.
5. Measure the right stuff.
It makes a difference.
97
@morys

51. „If it doesn‘t sell,
it isn‘t creative.“
David Ogilvy
@morys

52. © Andre Morys, Web Arts AG www.web-arts.com99
André Morys
CEO
Web Arts AG
andre.morys@web-arts.com
Tel. +49.6172.68097-15
www.web-arts.com
Thank You!