Class 07: Testing

1. TESTING

2. Facebook Advertising
WHO
WHAT
WHERE
WHEN
HOW
WHY
PERSONAS
COPY/IMAGES
PLATFORM
TIME/DATE/SEASON
UX/UI
BEHAVIOR

3. Facebook Advertising
LET’S GO BACK TO
YOUR MIDDLE SCHOOL
SCIENCE CLASSROOM
FOR A MINUTE...
...WHAT IS the
scientific method?

4. Facebook Advertising
Data-Driven Strategy

5. Facebook Advertising
Concept
Test
Analyze
Act Data-Driven Strategy

6. Facebook Advertising
Concept
Test
Analyze
Act Data-Driven Strategy
Strategize
Strategize
ExecuteExecute

7. Facebook Advertising
Concept
Test
Analyze
Act Data-Driven Strategy
Strategize
Strategize
ExecuteExecute
What To Do How To Do It
What HappenedWhy

8. A/B TESTING & MARKETING OPTIMIZATION
DESIGNING AN A/B TEST

9. DESIGNING AN A/B TEST
Conclusion
Hypothesis
Experiment
Analyze
Scientific Method
Observe, then…

10. EXAMPLE: LANDING PAGE VARIATIONS
A
B

11. CHANGE ONE THING ONLY
‣ Elements that are frequently tested:
‣ Offers
‣ Phrasing/Language
‣ Layout, Images
‣ # of Form Fields / Collected Data Points
‣ Elements that should remain constant:
‣ Time
‣ Other variables
‣ Any element leading up to the thing you're testing

12. TESTING RESULT SCENARIOS
‣ Not enough data or not extreme enough differences to reach confidence
‣ Too many success metrics
‣ Invalid experimental designs

13. WHAT
QUALIFIES AS
A VALID TEST?

14. STOLEN FROM DAN SIROKER

15. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

16. WHAT IS EXPERIMENTATION?
LET’S GO BACK TO
YOUR MIDDLE
SCHOOL SCIENCE
CLASSROOM FOR A
MINUTE...
...WHAT IS the
scientific
method?

17. WHAT IS EXPERIMENTATION?
Data-Driven Strategy

18. WHAT IS EXPERIMENTATION?
Concept
Test
Analyze
Act Data-Driven Strategy

19. WHAT IS EXPERIMENTATION?
Concept
Test
Analyze
Act Data-Driven Strategy
Strategize
Strategize
ExecuteExecute

20. WHAT IS EXPERIMENTATION?
Concept
Test
Analyze
Act Data-Driven Strategy
Strategize
Strategize
ExecuteExecute
What To Do How To Do It
What HappenedWhy

21. WHAT CAN YOU TEST: VARIABLES
WHO
WHAT
WHERE
WHEN
HOW
WHY
PERSONAS
COPY/IMAGES
PLATFORM
TIME/DATE/SEASON
UX/UI
BEHAVIOR

22. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

23. WHAT COULD YOU TEST WITH THIS AD?

24. WHAT COULD YOU TEST WITH THIS AD?
▪ Phrasing / Language
▪ Call to action
▪ Offers
▪ Targeting
▪ Keywords
▪ Location
▪ Age
▪ Language
▪ Day of Week, Time of Day
▪ Devices
▪ Ad destination (i.e. landing page)

25. WHAT COULD YOU TEST WITH THIS SITE?
https://www.warbyparker.com/eyeglasses/women/linwood/violet-magnolia

26. WHAT COULD YOU TEST WITH
THIS SITE?
‣ Check-out / sign-up process or form
‣ Button shapes, text, and color
‣ Site or product photography

27. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

29. DON’T FORGET THE DATA

30. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

31. EXPERIMENT DESIGN: TEST YOUR HYPOTHESIS
OPTION 1: Run “Free Shipping” ad on all keywords for 1
week & compare Conversion Rate to average
conversion rate.
OPTION 2: Run “free shipping” ad on half of
keywords for 1 week + CONTROL ad on other half &
compare conversion rates.

32. EXPERIMENT DESIGN: TEST YOUR HYPOTHESIS
OPTION 3: Run “free shipping” ad + CONTROL ad on
same keywords AT SAME TIME & compare conversion
rates.
OPTION 1: Run “Free Shipping” ad on all keywords for 1
week & compare Conversion Rate to average
conversion rate.
OPTION 2: Run “free shipping” ad on half of
keywords for 1 week + CONTROL ad on other half &
compare conversion rates.

33. PREMATURE
OPTIMIZATION IS THE
ROOT OF ALL EVIL.
- DONALD KNUTH

34. CHECK YOUR CURRENT DATA

35. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

36. MEASURING EXPERIMENTS
source: https://www.youtube.com/watch?v=Eh00PoR76NYyoutube.com/watch?v=Eh00PoR76NY

37. DESIGNING YOUR EXPERIMENTS: FRAMEWORK
1. Define quantifiable success metrics
2. Explore before you refine
3. Less is more. Reduce choices.
4. Words matter. Focus on your call to action.
5. Fail fast.
6. Start today.
STOLEN FROM DAN SIROKER

38. USER EXPERIENCE DESIGN
WHAT IS UX?

39. UX + DIGITAL MARKETING
USABILITY RULES THE WEB. SIMPLY STATED, IF THE CUSTOMER CAN’T
FIND A PRODUCT, THEN HE OR SHE WILL NOT BUY IT. JAKOB NIELSEN
IF THERE IS ANY ONE SECRET OF SUCCESS, IT LIES IN THE ABILITY TO
GET THE OTHER PERSON’S POINT OF VIEW AND SEE THINGS FROM
THAT PERSON’S ANGLE AS WELL AS FROM YOUR OWN. HENRY FORD

40. UX + DIGITAL MARKETING
THE RELATIONSHIP BETWEEN UX &
MARKETING
UX
MKTG

41. UX + DIGITAL MARKETING
UX & MARKETING ARE COMPLIMENTARY
Marketing is all about creating
the best and easiest path for your
customer to engage with your
product.
UX is about addressing user
problems and/or unmet needs to
help them achieve their goals.

42. WHAT IS UX
DESIGNPSYCHOLOGY
TECHNOLOGY
User Research
observing user
behaviors and
identifying user’s
needs
UI Design provide a
solution to get the
user to do what you
want them to do.
Prototyping - make things that
demonstrate the concepts to be shown
to potential users.

43. THE MAIN USER CONSIDERATIONS A MARKETER NEEDS TO KEEP IN
DESIGN
PSYCHOLOGY
TECHNOLOGY
Design the right ad and landing page experience and user
flow that speaks to your customer so they engage and stay
engaged with your product over time.
Understand what a potential and existing user expects,
needs, wants, feels about your product.
Build the right framework and platform to onboard users
that trigger points to re-engage at a later point.

44. UX + DIGITAL MARKETING

45. WEB PAGES
ANALYZING
WEB PAGES

46. WEB PAGES
LANDING PAGES
“Landing page experience refers to how good we think someone’s
experience will be when they get to your landing page” Google.
“User experience’s greatest impact to SEO is through the increase
it creates in organic sharing and distribution” Rand Fishkin, Moz.
“Pages with little content (aka thin content) are routinely ranked
poorly in their search results” Eric Enge, Stone Temple Consulting.

47. UX AND ITS ROLE IN MARKETING
WEBSITES
OBJECTIVES NAVIGATION LAYOUT
Your website should not
only “look good”, it should
also link back to your
objectives. What are your
business objectives?
You need to have a clear
path that is as
uncomplicated as
possible.
Being creative may not be
your friend.Your layout
should also be responsive
and work on all the devices
your customer is using
(desktop, tablet, mobile)

48. FOCUS IN ON THE VALUE PROPOSITION
WHY ARE PEOPLE HERE? WHAT IS THE CORE GOAL

49. UX FUNDAMENTALS
WHICH CHECKOUT EXPERIENCE IS MORE FOCUSED?
1 2

50. UX FUNDAMENTALS
WHAT IS THE CORE USER GOAL IN THIS PRODUCT?

51. UX CASE STUDIES: HTTP://UXTIMELINE.COM
2007 2015

52. LANDING PAGES, UX AND LEAD GEN
QUESTION
Look through 1 case study at uxtimeline.com
What ways has this brand used UX to be successful?
Why do you think those changes made the site better?
List as many as you can!

53. CONVERSION FUNNELS

54. CONVERSION FUNNELS
100%750,000
10%75,000
5%3,750
40%1,500

55. LANDING PAGES, UX AND LEAD GEN
ACTIVITY

56. Digital Marketing
DIRECTIONS
1. Find a partner, choose one of the three case study sets, then complete the objective of
the case study using this format:
1. Research Individually - 3 MIN
2. Swap Websites - 3 MIN
3. Discuss - 3 MIN
2. You will be given a website and a specific task. As you complete your task, write down
anything and everything that you find to be particularly helpful or intrusive.

57. VRBO.COM
SITE ANALYSIS: ROUND 1
FIND A COZY CABIN FOR 2 IN BOULDER,
COLORADO FOR NEXT WEEKEND.
AIRBNB.COM

58. KAYAK.COMHIPMUNK.COM
SITE ANALYSIS: ROUND 2
FIND THE FLIGHT WITH LEAST CONNECTIONS
TO ROME, ITALY A MONTH FROM NOW.

59. BOUNCEEXCHANGE.COMUNBOUNCE.COM
SITE ANALYSIS: ROUND 3
GO THROUGH THE EDUCATION AND SIGNUP
PROCESS.

60. 46SITE ANALYSIS
⌚ 3 ROUNDS: 15 MIN.
TOTAL: 45 MIN.
1. Grab (or share) a computer, a piece of paper, and a
pen.This activity has 3 rounds, each one is 15 minutes
long and consists of the following structure: research (5
min), swap websites (5 min), discuss (5 min)
2. Each group will be given a website and a specific task. As
you complete your task, write down anything that you find
to be particularly helpful or intrusive.
3. After 5 minutes, groups will switch websites and repeat.
STRUCTURE OBJECTIVES
2 GROUPS
ACTIVITY DEBRIEF

61. A/B TESTING & MARKETING OPTIMIZATION
DESIGN A USABILITY
OPTIMIZATION TEST

62. LOG IN TO GOOGLE ANALYTICS
NOT THE DEMO ACCOUNT!

63. Digital Marketing
DIRECTIONS
1. Create 1 A/B test in Google Optimize
2. Choose 1 objective
3. Choose 1 page
4. Choose an AdWords or Facebook Ads ad to link to the page
5. Write a SMART Objective, including the 1 test variable!
6. Use Google Optimize to make the changes
If you finish early, create an A/B test for one of your AdWords text ads!