Business Model Canvas (BMC)- A new venture concept
A/B Testing for Apps (Coca-Cola Mobile Innovation Workshop)
1. A/B Testing for Apps
How I Learned to Stop Worrying and Love
Data-Driven Decisions
Prepared for:
Coca-Cola Mobile Innovation Workshop
September 24-26, 2013
Shanghai, China
A/B Testing for Apps
2. User Base
Publish two different versions
of your app...
50% sees version A
...and see which one is driving
desirable user behavior.
50% sees version B
4. Version B: 114% Improvement
✗
Version A
Version B
Marketer’s Surprise: ‘FREE’ Loses
✔
5. Multivariate Testing
• Every screen has X components (ex: Marilyn’s hair)
• For each, we can test Y variations (ex.: Green)
• In total, we have [Y1 x Y2 x Y3] combinations
10. And the Winner is...
+40%
increase in conversion
rate
2.9 million
additional donators
$60 million
value of additional
donations
11. Obamalytics
• Original Conversion Rate:
• New Conversion Rate:
8.3%
11.6%
• 10 million signups from New Version would have been
7.12 million signups with the Original Version
• +2.88 million additional signups
• $21 average donation per signup
• Approximately $60 million in additional donations
13. Data-Driven App Marketing
• App marketing and analytics is in early stages
• App stores are opaque, approval processes slow down
iteration cycles
•
Few options for making decisions
• HiPPO: Highest Paid Person’s Opinion
• IHOP: In-House Optimization
• Scalable, reliable app measurement tools are emerging
