SlideShare a Scribd company logo
1 of 34
How, what & why of mobile game
A/B testing and optimization
Based on http://appiterate.com/how-whatand-why-of-ab-testing-and-optimization-ofandroid-and-ios-mobile-games/
Game Developer Myth #253:

•

A/B testing is change

•

CHANGE IS DANGEROUS!!

• Thus,

A/B testing is DANGEROUS!!!
But…
Morpheus is right…
Game developers have it easier than app developers
Game developers have it easier than app developers

Why? You asked?
Game developers have it easier than app developers

Why? You asked?
Stat #1:

93% of paid app downloaders pay for a mobile gaming app
http://www.socialnomics.net/2012/01/20/social-gaming-infographic-81-million-play-each-day-more-stats/
This is because a game is the CORE product and not a supplement to
another product as might be the case for other apps
This is because a game is the CORE product and not a supplement to
another product as might be the case for other apps

Also, you have to give it your all BECAUSE it
is your core product!
Stat #2:
80% of all revenue generated by mobile applications in 2012 was from games
http://www.wired.com/opinion/2013/01/how-mobile-kills-the-console-but-advances-the-gaming-industry/
Stat #2:
80% of all revenue generated by mobile applications in 2012 was from games
http://www.wired.com/opinion/2013/01/how-mobile-kills-the-console-but-advances-the-gaming-industry/

But, this stat is skewed by the Zyngas and Rovios of the world (The top 25
developers alone took in half of mobile development revenue in 2012)
http://skillz.com/blog/2013/03/05/65-mobile-gaming-stats-to-impress-your-friends/www.gamesindustry.biz/articles/2012-12-06-top-25-us-mobile-devs-take-in-halfof-app-revenue
So, what do you do if you are a small
developer working out of your garage?
This perfectly portrays game developers!
•

Acquiring the user is just getting off the blocks

•

In fact, you did well to get another download

But… you still haven’t finished the race...
So, what do you need to make some sweet moolah??
So, what do you need to make some sweet moolah??

You need to be a NINJA. A R-E-M ninja!
R-Retention
E-Engagement
M-Monetization

Let’s have a look at what you can do to be one…
ENGAGEMENT

Create targeted experiences for gamer segments
•

Tailor your game’s event flow based on previous interactions

•

If a gamer is unable to complete certain levels, you can dynamically change the
difficulty level from our dashboard

•

Use preset conditions over a user group to achieve this!
ENGAGEMENT

Game features
•

Common sense says - higher the number of unlocked levels, more would be the
amount of time spent in a freemium game
•

But, what if you are keeping money on the table by giving your gamers too many free
levels?

•

Possible in case of an engaging game

•

Developers shouldn’t lose out on what they deserve.

•

A/B test how many free levels should be there

•

Treat advanced gamers differently from a novice gamer
•

Different feature launches - another way to provide a more tailored gaming experience
ENGAGEMENT

In-app messaging
•

Powerful medium of
•

Cross selling other games in your kitty

•

Selling your wares in an in-game store

•

Test which levels are the sweetest to send an in-app message and for what
category of messages

•

Use user segments (both behavioural and demographic) here to create an instant
win
RETENTION

Start position
•

How many free credits do you give out at the start?
•

Too many means you lose out on later credit purchases

•

Too few means a higher chance of an uninstall

•

Can only be found out from rigorous experimentation and putting your numerous
hypotheses to test

•

Segment the results using analytics to find what works for which user base
RETENTION

Registration screens
•

These are a way to hold user information in games where you build your profile
and earn reputation

•

Once you a gamer has come to this screen after downloading your game (which
already means a WIN for you), you should not piss him off with a badly designed
form

•

Test different variations of fields, form layouts and designs

•

Experiment using social sign-up plugins as well (Facebook, Twitter and the like).
RETENTION

Characters in Tutorial
•

First ask yourself - do you even need a tutorial at first launch of your game on a
device?

•

Separate the advanced gamers from the novices

•

Determine your target audience for a tutorial

•

Show different tutorial characters depending on gender or age of the gamer
RETENTION

Theme layouts
•

Change game screen depending on
•
•

An approaching festival or event

•

•

Time of the year

Use different layouts by country

Use a visual mobile editor platform to do all that and much more.
MONETIZATION

In-game ads
•

Advertisements are the most common form of earning revenue for free games

•

Test when maximum Click Through Rates (CTR) happen
•
•

Or is it the text size or a different way in which the ad was worded?

•

•

Is it a certain size, layout, format, on-screen position of an ad?

Why not create different ad texts for gamers speaking different languages?

Although interstitial ads are hated, you can always test delay times to increase
CTRs
MONETIZATION

In game purchases (1/2)
•

These form the biggest revenue earner for a freemium game
•

•

•

Virtual goods alone make $12.92+ per user per month for such a game

That’s a lot of money, considering majority of gamers do not bother buying

Iteratively find the sweet spot in pricing
•

Test different pricing strategies for various goods to find which goods are elastic and
inelastic for you*

•

Test the amount of free currency you want to give out to gamers

•

Give out free currency to re-engage a person who isn’t able to progress in a game
because he/she doesn’t want to buy

+ http://iqu.com/blog/mobile-game-stats-you-need-to-know-infographic
* http://en.wikipedia.org/wiki/Elasticity_%28economics%29#Elasticities_of_demand
MONETIZATION

In game purchases (2/2)
•

Targeted pricing
•

Increase your prices before weekends/holidays

•

Give discounts to a light user

Play around pricing a lot
MONETIZATION

In-game store design
•

Store layouts can help a lot in increasing purchases in a game

•

Answers to questions like these can be found out from real gamer data:
•

Which goods come before others?

•

How big should the picture of an item be?

•

Should you use descriptive text or does it clutter your screen?

•

Test where the link to the store should be present in the game menu!

All this is to improve your ARPAU (another fancy term for
Average Revenue Per Active User) – the raison d’être of a game
developer/marketer
Maybe the greatest tribute to mobile game A/B testing was paid by Mark Pincus,
CEO, Zynga, when he said, “We no longer need to build it for three months and
hope and pray.”

http://grattisfaction.com/2010/01/how-zynga-does-customer-development-minimum-viable-product/
To sign off, don’t forget what your purpose as a game developer is!
SOME ADDITIONAL TIPS (1/3)

Remove bottlenecks in the testing process:
•

Use Appiterate’s code-free visual platform to eliminate approvals from your
coders and save precious developer time

•

Create and deploy tests on the fly

•

Use live user data to deploy winning variations

Appiterate is so easy even marketers can do it!
SOME ADDITIONAL TIPS (2/3)

Remove failures:
•

Too much experimenting gives rise to experiments that don’t work

•

Appiterate’s dashboard helps you eliminate failures easily by letting you disable live
tests

Don’t change everything:
•

Split an experiment into two tests so that you can pinpoint the reason of the
change in your goal metric

•

Too many types of changes in a multivariate test can spoil the experiment and you
will end up without any insight
SOME ADDITIONAL TIPS (3/3)

Segment your tests
•

Run some types of tests only on first-time users of your app

•

They have not seen your layout before and the UI would be new for them

•

Resulting data would be clean, unbiased and insightful!
www.appiterate.com
info@appiterate.com

Visit http://appiterate.com/how-what-and-why-of-ab-testing-and-optimizationof-android-and-ios-mobile-games/ for more

More Related Content

Recently uploaded

Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 

Recently uploaded (20)

Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 

Featured

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by HubspotMarius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTExpeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsPixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Mobile game A/B testing and optimization

  • 1. How, what & why of mobile game A/B testing and optimization Based on http://appiterate.com/how-whatand-why-of-ab-testing-and-optimization-ofandroid-and-ios-mobile-games/
  • 2. Game Developer Myth #253: • A/B testing is change • CHANGE IS DANGEROUS!! • Thus, A/B testing is DANGEROUS!!!
  • 4.
  • 6. Game developers have it easier than app developers
  • 7. Game developers have it easier than app developers Why? You asked?
  • 8. Game developers have it easier than app developers Why? You asked? Stat #1: 93% of paid app downloaders pay for a mobile gaming app http://www.socialnomics.net/2012/01/20/social-gaming-infographic-81-million-play-each-day-more-stats/
  • 9. This is because a game is the CORE product and not a supplement to another product as might be the case for other apps
  • 10. This is because a game is the CORE product and not a supplement to another product as might be the case for other apps Also, you have to give it your all BECAUSE it is your core product!
  • 11. Stat #2: 80% of all revenue generated by mobile applications in 2012 was from games http://www.wired.com/opinion/2013/01/how-mobile-kills-the-console-but-advances-the-gaming-industry/
  • 12. Stat #2: 80% of all revenue generated by mobile applications in 2012 was from games http://www.wired.com/opinion/2013/01/how-mobile-kills-the-console-but-advances-the-gaming-industry/ But, this stat is skewed by the Zyngas and Rovios of the world (The top 25 developers alone took in half of mobile development revenue in 2012) http://skillz.com/blog/2013/03/05/65-mobile-gaming-stats-to-impress-your-friends/www.gamesindustry.biz/articles/2012-12-06-top-25-us-mobile-devs-take-in-halfof-app-revenue
  • 13. So, what do you do if you are a small developer working out of your garage?
  • 14. This perfectly portrays game developers!
  • 15. • Acquiring the user is just getting off the blocks • In fact, you did well to get another download But… you still haven’t finished the race...
  • 16. So, what do you need to make some sweet moolah??
  • 17. So, what do you need to make some sweet moolah?? You need to be a NINJA. A R-E-M ninja! R-Retention E-Engagement M-Monetization Let’s have a look at what you can do to be one…
  • 18. ENGAGEMENT Create targeted experiences for gamer segments • Tailor your game’s event flow based on previous interactions • If a gamer is unable to complete certain levels, you can dynamically change the difficulty level from our dashboard • Use preset conditions over a user group to achieve this!
  • 19. ENGAGEMENT Game features • Common sense says - higher the number of unlocked levels, more would be the amount of time spent in a freemium game • But, what if you are keeping money on the table by giving your gamers too many free levels? • Possible in case of an engaging game • Developers shouldn’t lose out on what they deserve. • A/B test how many free levels should be there • Treat advanced gamers differently from a novice gamer • Different feature launches - another way to provide a more tailored gaming experience
  • 20. ENGAGEMENT In-app messaging • Powerful medium of • Cross selling other games in your kitty • Selling your wares in an in-game store • Test which levels are the sweetest to send an in-app message and for what category of messages • Use user segments (both behavioural and demographic) here to create an instant win
  • 21. RETENTION Start position • How many free credits do you give out at the start? • Too many means you lose out on later credit purchases • Too few means a higher chance of an uninstall • Can only be found out from rigorous experimentation and putting your numerous hypotheses to test • Segment the results using analytics to find what works for which user base
  • 22. RETENTION Registration screens • These are a way to hold user information in games where you build your profile and earn reputation • Once you a gamer has come to this screen after downloading your game (which already means a WIN for you), you should not piss him off with a badly designed form • Test different variations of fields, form layouts and designs • Experiment using social sign-up plugins as well (Facebook, Twitter and the like).
  • 23. RETENTION Characters in Tutorial • First ask yourself - do you even need a tutorial at first launch of your game on a device? • Separate the advanced gamers from the novices • Determine your target audience for a tutorial • Show different tutorial characters depending on gender or age of the gamer
  • 24. RETENTION Theme layouts • Change game screen depending on • • An approaching festival or event • • Time of the year Use different layouts by country Use a visual mobile editor platform to do all that and much more.
  • 25. MONETIZATION In-game ads • Advertisements are the most common form of earning revenue for free games • Test when maximum Click Through Rates (CTR) happen • • Or is it the text size or a different way in which the ad was worded? • • Is it a certain size, layout, format, on-screen position of an ad? Why not create different ad texts for gamers speaking different languages? Although interstitial ads are hated, you can always test delay times to increase CTRs
  • 26. MONETIZATION In game purchases (1/2) • These form the biggest revenue earner for a freemium game • • • Virtual goods alone make $12.92+ per user per month for such a game That’s a lot of money, considering majority of gamers do not bother buying Iteratively find the sweet spot in pricing • Test different pricing strategies for various goods to find which goods are elastic and inelastic for you* • Test the amount of free currency you want to give out to gamers • Give out free currency to re-engage a person who isn’t able to progress in a game because he/she doesn’t want to buy + http://iqu.com/blog/mobile-game-stats-you-need-to-know-infographic * http://en.wikipedia.org/wiki/Elasticity_%28economics%29#Elasticities_of_demand
  • 27. MONETIZATION In game purchases (2/2) • Targeted pricing • Increase your prices before weekends/holidays • Give discounts to a light user Play around pricing a lot
  • 28. MONETIZATION In-game store design • Store layouts can help a lot in increasing purchases in a game • Answers to questions like these can be found out from real gamer data: • Which goods come before others? • How big should the picture of an item be? • Should you use descriptive text or does it clutter your screen? • Test where the link to the store should be present in the game menu! All this is to improve your ARPAU (another fancy term for Average Revenue Per Active User) – the raison d’être of a game developer/marketer
  • 29. Maybe the greatest tribute to mobile game A/B testing was paid by Mark Pincus, CEO, Zynga, when he said, “We no longer need to build it for three months and hope and pray.” http://grattisfaction.com/2010/01/how-zynga-does-customer-development-minimum-viable-product/
  • 30. To sign off, don’t forget what your purpose as a game developer is!
  • 31. SOME ADDITIONAL TIPS (1/3) Remove bottlenecks in the testing process: • Use Appiterate’s code-free visual platform to eliminate approvals from your coders and save precious developer time • Create and deploy tests on the fly • Use live user data to deploy winning variations Appiterate is so easy even marketers can do it!
  • 32. SOME ADDITIONAL TIPS (2/3) Remove failures: • Too much experimenting gives rise to experiments that don’t work • Appiterate’s dashboard helps you eliminate failures easily by letting you disable live tests Don’t change everything: • Split an experiment into two tests so that you can pinpoint the reason of the change in your goal metric • Too many types of changes in a multivariate test can spoil the experiment and you will end up without any insight
  • 33. SOME ADDITIONAL TIPS (3/3) Segment your tests • Run some types of tests only on first-time users of your app • They have not seen your layout before and the UI would be new for them • Resulting data would be clean, unbiased and insightful!