Mobile A/B Test

•

9 likes•1,093 views

Lucas Montano

A/B Testing for Mobile

Mobile

Inspired Dan Siroker & Wyatt Jenkins
A/B Testing

GROWTH
is simply a series of experiments
Wyatt Jenkins VP Product
at shutterstock

Idea Generation
Qualitative
Wyatt Jenkins VP Product
at shutterstock
Validation
Quantitative

You can not test what you don't meusure
Define quantifiable success metrics
50%
Users
50%
Users
Avg. Session Duration:
00:00:58
Avg. Session Duration:
00:01:28

You can not test what you don't meusure
Define quantifiable success metrics
100%
Users
Avg. Session Duration:
00:00:58
Avg. Session Duration:
00:01:28

Great tools to use
Define quantifiable success metrics

You can not test what you don't meusure
Define quantifiable success metrics

151
View Signup Page
105
Enter Email
78
Enter Password
46
Sign Up
69.54% 74.29%
58.97%
User flow
Understand Bottleneck

User flow
Understand Bottleneck
151
View Signup Page
105
Enter Email
78
Enter Password
46
Sign Up
69.54% 74.29%
58.97%

User flow
Understand Bottleneck
50%
Users
Repeat Password
50%
Users
Hide Repeat Password

User flow
Understand Bottleneck
151
View Signup Page
105
Enter Email
78
Enter Password
55
Sign Up
69.54% 74.29%
70.27%

User flow
Understand Oportunities
Download Content Revenue
Bound

User flow
Understand Oportunities
Download Content Revenue
Bound
50%
Users
50%
Users

User flow
Understand Oportunities
Download Content Revenue
+6%

Explore
Combining variants
Download Content Revenue
+9%

Words matter. Focus on your call to action.

Explore
Words Matters
Download Now Free Download Save to Later

Explore
Words Matters
Download Now Free Download Save to Later
+9%

Viewers also liked

Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing ...

Online Dialogue

The Best A/B Test Idea You Haven’t Thought Of

Kissmetrics on SlideShare

Chief Persuasion Officer Bart Schutz will be keynoting about his Wheel of Persuasion on marketing & optimization conferences throughout Europe in october & november 2014: Conversion Jam - Stockhom, Sweden Conversions@Google - Dublin, Ireland iLive - Riga, Latvia Conversion Conference - London, England Conversion Conference - Berlin, Germany Conversion Summit - Istanbul, Turkey Conversion Hotel - Texel, the Netherlands At these conferences he will present his Maxima (System 1) and Willem (System 2) story and inspire you how to persuade these two parts of your customers brains. This slideset is the main slideset for each conference and is extended with extra A/B-testcases. More info: http://wheel-of-persuasion.com (the knowledge source) http://online-dialogue.com (hire our team) http://testing.agency (fast and quality A/B-test development) http://abtestguide.com (you A/B-test workflow tool) and http://Persuasion.tips for an overview of Bart his recent work.

Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...

Online Dialogue

Auto aftermarket digital driver research studies

HouseLead Italia

“In God we trust, all others must bring data”. Intuition, experience and well known patterns may give us good indications of successful ideas and features, but nothing gets closer to the truth than data analysis and A/B testing. In this workshop, we’ll show how we do experimentation at Booking: what we test, how to get data through templates and JavaScript, and how we analyse the resulting metrics. We’ll live-code examples, see all potential caveats of dealing with the user tracking on the client-side, and show existent tools you can use to test your own ideas.

The anatomy of an A/B Test - JSConf Colombia Workshop

Eduardo Shiota Yasuda

Why everything is an A/B Test at Pinterest

Krishna Gade

What is A/B testing? A SCIENTIFIC METHOD OF PROOF WHAT IS A/B TESTING FOR MARKETING Where do you start testing? THE DON’TS WHERE DO YOU START TESTING? DON’T JUST GUESS SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing Where do you start testing? THE DOS DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights? A framework for success The basis of a successful testing program Research Know what to test and see the greatest improvements DIVE INTO YOUR DATA FIND YOUR BIGGEST OPPORTUNITIES INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT FORM A HYPOTHESIS By doing A, B will happen, Because of C BUILD THE TEST LAUNCH THE TEST VS. Control Variant THE TEST VS. +24.01% increase in registrations!! 100% Significance DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company REPEAT THE PROCESS AS OFTEN AS POSSIBLE Control vs. Variant “Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo* “The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0 Tooltips AND BURGERS How I like my testing burger Tooltips for your testing framework Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities Not everyone likes their burger the same way How do you take yours? Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS

How to Successfully Run Your First Website A/B Test

Kissmetrics on SlideShare

Presenter: Andrea Burbank, Pinterest A successful experimentation program consists of much more than mere randomization and measurement. How do you help stakeholders understand the right things to measure, avoid common pitfalls, and learn to rely on A/B tests as the best way to measure a new system or feature? In this talk, Andrea will explain how building a culture of experimentation and the right tools to support it is just as important as the statistics behind the comparisons themselves - and potentially much trickier to get right.

A/B Testing at Pinterest: Building a Culture of Experimentation

WrangleConf

Millward brown vermeer organizational design think brand 2014 snf

Vermeer

A Crash Course on A/B Testing

Stephen Edward Puiszis

Auto shoppers are open to influences. Today’s auto intenders are open-minded with 72% of search sessions involving cross-shopping. In-market shoppers are researching more than ever. While time in-market is roughly the same, these shoppers are turning to 24 research touch points on average. Connected devices are driving greater research activity. Mobile usage has increased 35% year over year, with a focus on research and comparison activities. Video influences auto brand discovery and consideration. Video research is on the rise and online video was the #1 format for encouraging brand consideration. Dealer interaction and post-purchase experiences matter. 62% of the vehicle owners said customer service could influence future purchases.

Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...

SL Ecommerce and ReviewsReputation.com

The Operating Model That is Eating the World

undercurrent

Always Be Testing - Learn from Every A/B Test (Hiten Shah)

Future Insights

[#500Distro] Measuring for Impact: Knowing When, What & How to A/B Test

500 Startups

How to A/B Test with WordPress: Conversions Aren’t Just for Landing Pages

Brendan Sera-Shriar

Viewers also liked (15)

Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing ...

The Best A/B Test Idea You Haven’t Thought Of

Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...

Auto aftermarket digital driver research studies

The anatomy of an A/B Test - JSConf Colombia Workshop

Why everything is an A/B Test at Pinterest

How to Successfully Run Your First Website A/B Test

A/B Testing at Pinterest: Building a Culture of Experimentation

Millward brown vermeer organizational design think brand 2014 snf

A Crash Course on A/B Testing

Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...

The Operating Model That is Eating the World

Always Be Testing - Learn from Every A/B Test (Hiten Shah)

[#500Distro] Measuring for Impact: Knowing When, What & How to A/B Test

How to A/B Test with WordPress: Conversions Aren’t Just for Landing Pages

Similar to Mobile A/B Test

Practical A/B Testing Statistics: 5 Tips To Help You Get Reliable Data

Hanapin Marketing

Growth hacking 101 - RET2082 POV

Roy Tertman

There's a Customer Out There with a Bullet for You: Understanding Your Customers

Evan Hamilton

How to Test Your Product Ideas by SoulCycle VP of Product

Product School

Overview Sales forecasting is a science and an art. It is the combination of information and metrics, intuition and best practices. However, sales forecasting is most commonly associated to the standard grading methodology of the particular customer relationship system that is being used (Salesforce.com, Oracle, Microsoft, etc.). In reality, how do key sales leaders become high performing accurate sales forecasters? In addition, how do companies effectively utilize sales forecasting information to increase overall organizational performance? Here’s what we’ll discuss in this session: State-of-the-art forecasting strategies, best practices, and key metrics The interconnection between product complexity, company lifecycle stage, and accurate forecasting Mitigating downside risk and triangulation strategies to determine the truth Deal inspection and vetting sales rep forecasts The different types of sales forecasters; exaggerators, sandbaggers, and Heavy Hitters The difference between snapshot, intra-department, and inter-department sales forecasting

The Art and Science of Sales Forecasting: A Webinar for Sales Managers and Co...

Birst

How to Use Customer Feedback on Your Product by UserVoice CEO

Product School

Palestra TDC POA - Trilha Gestão de Produto Link: http://www.thedevelopersconference.com.br/tdc/2017/portoalegre/trilha-gestao-de-produto A aquisição de um novo cliente custa de 5 a 10 vezes mais do que reter um cliente atual e um aumento em 5% de retenção pode refletir em ganhos de 25% até 95% no lucro da empresa. Se retenção é algo tão importante, por que as empresas dedicam pouco investimento e tem muitas dificuldades em mudar seus números de retenção? Atualmente, Churn Rate é a métrica mais famosa para retenção e tem uma correlação muito grande com 'growth' e lucro, porém ela nos diz pouco sobre qual é a deficiência de retenção do produto. Para entender melhor, é necessário aprofundar o conhecimento, entender como funciona o ciclo de retenção e implementar táticas, no produto, que irão abrangir todas as fases.

Palestra TDC POA - Seus clientes estão abandonando o seu produto? Aprenda com...

Gustavo Mendes

The minimum viable product (MVP) is that version of a product which allows a team to collect the maximum amount of validating insight about customers' needs with the least amount of product building. It's one of the most important principles of a lean startup. This deck provides you with insight into the value behind building an MVP, the overall process of achieving the maximum amount of validating learning, why in-depth customer interviews (and not just user testing) are crucial to testing your initial hypotheses about your business or product, and how you might collect or analyze qualitative and quantitative metrics to iterate your MVP. Josh Cyr, founder of Alpha Loft and web app developer, also shared real lessons learned from one startup while building their MVP.

Lean Startup Business Tactics - Identifying Your Minimal Viable Product

UNHInnovation

Infographic Presentation Template If you want to buy this presentation template, please visit http://punkl.com Creating a presentation from scratch can be quite labour-intensive. Starting with a presentation template from Punkl is beneficial. It saves time, provides good visual design and means that you can primarily spend your time and attention on the content of your presentation. Punkl Presentation Templates save you time, as they're a whole lot quicker than trying to design a deck from scratch. Also, starting with a template means that you can primarily spend your time and attention on the content of your presentation, while the visual style is already designed to be engaging. Typically, the only elements that are changed while working with a presentation template are colors, typography, copy and any visual assets such as photos for example.

Annya Presentation : Light Color Theme

punkl.

Annya Presentation : Dark Color Theme

punkl.

U certify earlybird sale

jasminesmith2111

By Srikant Kotapalli – VWO AB Testing has traditionally been one of the most commonly used methods to boost conversion rates. Many marketers have since used it tactically, that gets them some early wins but are unable to convert it into repeatable success. Building an effective optimization strategy needs an in-depth understanding of user behaviour, adopting a structured framework and constantly measuring and improving upon the results. This session will walk you through everything you need to know to transform your AB testing into a winning optimization engine.

Build a Winning Conversion Optimization Strategy

Savage Marketing

Mehrish-WPS Office.pptx

Mehrish7

Tec314

John Art

"You can download this product from SlideTeam.net" To help you conclude your next product and process innovation PowerPoint presentation on time, here we are showcasing complete PowerPoint slide deck that you can use for your next project. With help of this readymade PPT sample file you can motivate your employees to apply modifications in design of previous versions for new innovation and product development. Well motivation given in such a form of visual aid obviously gives a competitive advantage. Additionally, with our PPT model you can efficiently portray your innovation strategy so as to exert a pull on new potential partners or financers. Finally, to cover all related concepts our PPT presentation includes exclusive slides templates like of generate new product ideas, external sources of new product ideas, screening the ideas, competitor analysis etc. All in all, a complete PowerPoint show for product developers, product planners and innovators to depict different stages of innovation process. So why not download this complete stage gate product innovation process presentation deck to see its engaging impact. Express genuine gratitude with our Stage Gate Product Innovation Process Complete Powerpoint Deck With Slides. Be able to greet your guests heartily. https://bit.ly/3N8WbFA

Stage Gate Product Innovation Process Complete Powerpoint Deck With Slides

SlideTeam

Every organization needs to adapt to the ever-changing business environment. Sensing this need, we have come up with these content-ready change management PowerPoint presentation slides. These change management PPT templates will help you deal with any kind of an organizational change. Be it with people, goals or processes. The business solutions incorporated here will help you identify the organizational structure, create vision for change, implement strategies, identify resistance and risk, manage cost of change, get feedback and evaluation, and much more. With the help of various change management tools and techniques illustrated in this presentation design, you can achieve the desired business outcomes. This business transition PowerPoint design also covers certain related topics such as change model, transformation strategy, change readiness, change control, project management and business process. By implementing the change control methods mentioned in the presentation, you will be able to have a smooth transition in an organization. So, without waiting much, download our extensively researched change management framework presentation. With our Change Management Presentation slides, understand the need for change and plan to go through it without any hassles.

Stage Gate Product Innovation Process Complete PowerPoint Deck With Slides

SlideTeam

Beauty and Cosmetics Marketing Presentation Template If you want to buy this presentation template, please visit http://madlis.com Good design gets out of the way of the content you are sharing. It helps your audience focus on the content itself instead of the design. But, it's no secret that most people dislike giving presentations. The dread of public speaking consistently ranks among the greatest fears in public surveys. This presentation slides can help you reduce the anxiety involved with giving a presentation. Well-designed slides not only build your own confidence, they make your key points clearer to the audience.

Soroza Presentation : Light Color Version

Madlis

Soroza Presentation : Dark Color Version

Madlis

At Atlassian, support is essential to a successful customer experience. And, because of it's customer-facing position, support is also a great resource for product teams to continually use customer feedback to improve their product or service. Creating strong customer feedback loops means you are building a better experience for your customers day by day. In this talk, you'll learn 5 ways Atlassian tools facilitate the relationship between support and development and how you can establish these processes and metrics in your own organization. This talk is a follow on from "How Atlassian Support & Development Team Up to Release Software" from Summit 2016.

How Atlassian Broke Down the Walls Between Support & Development

Atlassian

How Can User Research Help PMs by Yelp Group Product Manager

Product School

Similar to Mobile A/B Test (20)

Practical A/B Testing Statistics: 5 Tips To Help You Get Reliable Data

Growth hacking 101 - RET2082 POV

There's a Customer Out There with a Bullet for You: Understanding Your Customers

How to Test Your Product Ideas by SoulCycle VP of Product

The Art and Science of Sales Forecasting: A Webinar for Sales Managers and Co...

How to Use Customer Feedback on Your Product by UserVoice CEO

Palestra TDC POA - Seus clientes estão abandonando o seu produto? Aprenda com...

Lean Startup Business Tactics - Identifying Your Minimal Viable Product

Annya Presentation : Light Color Theme

Annya Presentation : Dark Color Theme

U certify earlybird sale

Build a Winning Conversion Optimization Strategy

Mehrish-WPS Office.pptx

Tec314

Stage Gate Product Innovation Process Complete Powerpoint Deck With Slides

Stage Gate Product Innovation Process Complete PowerPoint Deck With Slides

Soroza Presentation : Light Color Version

Soroza Presentation : Dark Color Version

How Atlassian Broke Down the Walls Between Support & Development

How Can User Research Help PMs by Yelp Group Product Manager

Recently uploaded

99992-vip-18229 📞Mahipalpur (Delhi) Low price 100% genuine sexy VIP call girls are provided safe and secure service .call 📞,,24 hours 🕰️-- ✅100% gesnuine young RIYA SERVICE COMPANY ✔✔✔ ★ A-Level (5 star ) ★ Strip-tease ★ BBBJ (Bareback Blowjob) Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without a Condom) ★ DATING (Dinner At Night) ★ DSL (Dick Sucking Lips) ★ DT (Dining at the Toes English Spanking) ★ Doggie (Sex style from behind) ★ Duo (shot with two escorts; Threesome with the client) ★ S-GFE (Special Girl Friend Experience) ★ HJ (Hand Job) ★ Special Massage ★ O-Level (Oral sex) ★ Tour (International) ★ 69 (69 sex) ★ BJ (Blowjob With Condom) ★ GFE (Girl Friend Experience) ★ CBJ (Covered Blow Job; Oral sex with a condom _ LOW PRICE V I P MODEL FULL SAFE AND SECURE CALL MExxxs CALL ME

FULL ENJOY - 9999218229 Call Girls in {Mahipalpur}| Delhi NCR

nishacall1

Powerful Love Spells in Arkansas, AR (310) 882-6330 Bring Back Lost Lover

PsychicRuben LoveSpells

Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)

Cara Menggugurkan Kandungan 087776558899

Leading Mobile App Development Companies in India (2).pdf

CWS Technology

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

BDSM⚡Call Girls in Sector 71 Noida Escorts >༒8448380779 Escort Service

Delhi Call girls

99992-vip-66834 📞 Noida Sector 52 Low price 100% genuine sexy VIP call girls are provided safe and secure service .call 📞,,24 hours 🕰️-- ✅100% gesnuine young RIYA SERVICE COMPANY ✔✔✔ ★ A-Level (5 star ) ★ Strip-tease ★ BBBJ (Bareback Blowjob) Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without a Condom) ★ DATING (Dinner At Night) ★ DSL (Dick Sucking Lips) ★ DT (Dining at the Toes English Spanking) ★ Doggie (Sex style from behind) ★ Duo (shot with two escorts; Threesome with the client) ★ S-GFE (Special Girl Friend Experience) ★ HJ (Hand Job) ★ Special Massage ★ O-Level (Oral sex) ★ Tour (International) ★ 69 (69 sex) ★ BJ (Blowjob With Condom) ★ GFE (Girl Friend Experience) ★ CBJ (Covered Blow Job; Oral sex with a condom _ LOW PRICE V I P MODEL FULL SAFE AND SECURE CALL MExxxs CALL ME

9999266834 Call Girls In Noida Sector 52 (Delhi) Call Girl Service

nishacall1

Recently uploaded (6)

FULL ENJOY - 9999218229 Call Girls in {Mahipalpur}| Delhi NCR

Powerful Love Spells in Arkansas, AR (310) 882-6330 Bring Back Lost Lover

Obat Penggugur Kandungan Di Apotik Kimia Farma (087776558899)

Leading Mobile App Development Companies in India (2).pdf

BDSM⚡Call Girls in Sector 71 Noida Escorts >༒8448380779 Escort Service

9999266834 Call Girls In Noida Sector 52 (Delhi) Call Girl Service

Mobile A/B Test

1. Inspired Dan Siroker & Wyatt Jenkins A/B Testing

2. GROWTH is simply a series of experiments Wyatt Jenkins VP Product at shutterstock

3. 50% Users 50% Users

4. Idea Generation Qualitative Wyatt Jenkins VP Product at shutterstock Validation Quantitative

5. You can not test what you don't meusure Define quantifiable success metrics 50% Users 50% Users Avg. Session Duration: 00:00:58 Avg. Session Duration: 00:01:28

6. You can not test what you don't meusure Define quantifiable success metrics 100% Users Avg. Session Duration: 00:00:58 Avg. Session Duration: 00:01:28

7. Great tools to use Define quantifiable success metrics

8. You can not test what you don't meusure Define quantifiable success metrics

9. Understand Bottleneck

10. 151 View Signup Page 105 Enter Email 78 Enter Password 46 Sign Up 69.54% 74.29% 58.97% User flow Understand Bottleneck

11. User flow Understand Bottleneck 151 View Signup Page 105 Enter Email 78 Enter Password 46 Sign Up 69.54% 74.29% 58.97%

12. User flow Understand Bottleneck 50% Users Repeat Password 50% Users Hide Repeat Password

13. User flow Understand Bottleneck 151 View Signup Page 105 Enter Email 78 Enter Password 55 Sign Up 69.54% 74.29% 70.27%

14. Understand Oportunities

15. User flow Understand Oportunities Download Content Revenue Bound

16. User flow Understand Oportunities Download Content Revenue Bound

17. User flow Understand Oportunities Download Content Revenue Bound 50% Users 50% Users

18. User flow Understand Oportunities Download Content Revenue +6%