The document discusses different strategies for innovative testing on websites, including iterative and innovative testing. Iterative testing involves changing small elements like button colors or copy, while innovative testing involves changing multiple elements or entire designs. It recommends using innovative testing when an iteration won't work, testing potential is small, or traffic is low. This allows uncovering bigger gains. Innovative testing carries more risk but also higher potential benefit than iterative testing. The document advises making sure the testing process is solid by asking questions like whether the test matches the hypothesis and goals are set up correctly. It also recommends continuing qualitative research for feedback.

1. WHEN, WHY & HOW TO DO
INNOVATIVE TESTING

2. 3 STRATEGIES
FOR BETTER DESIGN
BETTER CR, MORE SALES

4. ITERATIVE TESTING

5. ITERATIVE TESTING
‣ BUTTON COLOR CHANGE
‣ COPY CHANGE
‣ DIFFERENT HERO IMAGE
‣ REMOVING/ADDING A PIECE OF CONTENT

6. ITERATIVE TESTING
‣ BUTTON COLOR CHANGE
‣ COPY CHANGE
‣ DIFFERENT HERO IMAGE
‣ REMOVING/ADDING A PIECE OF CONTENT
ADD, CHANGE OR REMOVE ONE ELEMENT

7. “THERE'S NO REASON TO LIMIT
YOURSELF TO A SINGLE STRATEGY,
BECAUSE THE 3 APPROACHES
COMPLEMENT EACH OTHER WELL.”

8. INNOVATIVE TESTING

9. INNOVATIVE TESTING
‣ NAVIGATION CHANGES
‣ RADICAL REDESIGN
‣ NEW FUNCTIONALITIES, FEATURES
‣ COMBINATION OF MULTIPLE CHANGES

10. INNOVATIVE TESTING
‣ NAVIGATION CHANGES
‣ RADICAL REDESIGN
‣ NEW FUNCTIONALITIES, FEATURES
‣ COMBINATION OF MULTIPLE CHANGES
ADD, CHANGE OR REMOVE MULTIPLE ELEMENTS

11. “INNOVATION IS CHANGE THAT UNLOCKS
NEW VALUE”
JAMIE NOTTER

12. WHEN, WHY & HOW IS
INNOVATIVE TESTING A
GOOD IDEA?

13. #1 WHENAN ITERATION WON’T SUFFICE
#2 WHEN YOU’RE TESTING POTENTIAL IS SMALL
#3 WHEN YOU DON’T HAVE MUCH TRAFFIC

14. ‣ BASICS ARE OFF: New
customers don’t
understand what the site
is about, who is it for and
what’s the first thing they
should do there.
‣ MESSY NAVIGATION:
Visitors are unable to find
the product/service they
need
‣ etc

15. Original CXL Variation

16. CR UPLIFT OF 22.33%
Original CXL Variation

17. #1 WHEN AN ITERATION WON’T SUFFICE
#2 WHENYOU’RE TESTING POTENTIAL IS SMALL
#3 WHEN YOU DON’T HAVE MUCH TRAFFIC

18. YOU’VE TESTED EVERY ELEMENT ON THE SITE ALREADY
Time spent testing
CR increase
detected
CR DIFFERENCE <5%

19. STATISTICAL SIGNIFICANCE <1%
~100 000 VISITORS REMAINING
TEST DURATION: 6 WEEKS
+1.2%

20. VERY SMALL UPLIFTS (1-5%)
ARE ALMOST IMPOSSIBLE TO
MEASURE !

21. #1 WHEN AN ITERATION WON’T SUFFICE
#2 WHEN YOU’RE TESTING POTENTIAL IS SMALL
#3 WHENYOU DON’T HAVE MUCH TRAFFIC

23. +14.1%
STATISTICAL SIGNIFICANCE 36%
~81 481 VISITORS REMAINING
TEST DURATION: 6 WEEKS

24. WITH LOW TRAFFIC EVEN 10-30%
UPLIFTS ARE IMPOSSIBLE TO
MEASURE!

25. WHY
#2 IT’S STILL A SAFER CHOICE OVER A RADICAL REDESIGN
#1TO UNCOVER GROWTH POTENTIALS / BIGGER GAINS
#3 IT’S MORE FUN

26. #1 TO UNCOVER GROWTH POTENTIALS / BIGGER GAINS

27. TRANSFORM AND GROW YOUR
BUSINESS BY FIRST TESTING OUT
NEW IDEAS ONLINE

28. WHY
#2IT’S STILL A SAFER CHOICE OVER A RADICAL REDESIGN
#1 TO UNCOVER GROWTH POTENTIALS / BIGGER GAINS
#3 IT’S MORE FUN

29. #2 IT’S STILL A SAFER CHOICE OVER A RADICAL REDESIGN
Original CXL Variation

30. #2 IT’S STILL A SAFER CHOICE OVER A RADICAL REDESIGN
Original CXL Variation
CR UPLIFT = 61%

31. CXL Variation CXL Variation 2

32. CXL Variation CXL Variation 2
CR DECREASE = -7%

33. WHY
#2 IT’S STILL A SAFER CHOICE OVER A RADICAL REDESIGN
#1 TO UNCOVER GROWTH POTENTIALS / BIGGER GAINS
#3IT’S MORE FUN

34. #3 IT’S MORE FUN
HYPOTHESIS LIVE TEST RESULT

35. YOU’LL GET THE WHOLE TEAM
EXCITED AND ON BOARD

36. HOW
#1MAKE SURE YOUR TESTING PROCESS IS SOLID
#2 KEEP DOING QUALITATIVE RESEARCH

37. 4 QUESTIONS

38. 1. Is the test based on a data driven hypothesis?

39. 1. Is the test based on a data driven hypothesis?
2. Does the test design exactly match the
hypothesis?

40. 1. Is the test based on a data driven hypothesis?
2. Does the test design exactly match the
hypothesis?
3. Has the test been setup correctly with all
goals attached?
HINT: use event tracking and universal integration both

41. 1. Is the test based on a data driven hypothesis?
2. Does the test design exactly match the
hypothesis?
3. Has the test been setup correctly with all
goals attached?
HINT: use event tracking and universal integration both
4. Did you perform a proper Q&A check?

42. Original CXL Variation

43. Original CXL Variation

44. Original CXL Variation

45. HOW
#1 MAKE SURE YOUR TESTING PROCESS IS SOLID
#2KEEP DOING QUALITATIVE RESEARCH

46. #2 KEEP DOING QUALITATIVE RESEARCH
What’s holding you back from completing
your purchase today?
YOU’LL HAVE REAL TIME FEEDBACK
IF SOMETHING IS WRONG!

47. TESTING STRATEGIES
ITERATIVE INNOVATIVE
COST
RISK
POTENTIAL BENEFIT
SAMPLE SIZE
LOW
LOW
LESS < ~10%
BIGGER
HIGH (UNLESS LUCKY)
HIGH
~10% < MORE
SMALLER

49. People who don’t take risks
generally make about two big mistakes a year.
People who do take risks
generally make about two big mistakes a year.
Peter F. Drucker

50. Super profitable conversion advice FREE
every week!
Marie Polli
Twitter: @MariePolli
http://conversionxl.com/
Questions?