Have you ever burnt money on unprofitable UA? Was there an opportunity to scale that you missed? Identifying potentially long-term profitable campaigns as soon as possible using LTV prediction can double or triple profits of the mobile game. In this talk, we will look at how to effectively reallocate user acquisition investments early in campaigns’ lifetime using Machine Learning. We will shed light on some of the challenges game studios face when building a fully automatic LTV prediction pipeline for games with investment optimisation in mind.

1. Optimising user acquisition through LTV prediction
Róbert Magyar
Warsaw 14 -15 October 2019

2. Warsaw | 14-15 October 2019
Róbert Magyar
Data Science Lead
robert.magyar@superscale.com

3. “We’re forming growth partnerships with world’s top developers
to scale their games to maximum potential.”

4. Warsaw | 14-15 October 2019
● 70+ World Class Experts In-House
○ (Self-)Publishing Infrastructure
i. Business Intelligence
ii. Analytics & Data Science
○ Monetization
i. LiveOps Optimization
ii. Game & Monetization Design
○ User Acquisition
i. Creatives
ii. UA Campaign Management
iii. ASO
● Founded in 2016
● Bratislava, London, Berlin, Helsinki &
Prague offices
First Light Games
Our partners
Games

5. Warsaw | 14-15 October 2019
UA Optimization
Motivation

6. Warsaw | 14-15 October 2019
How can we improve User Acquisition?
Huge topic - lots of angles
● Lookalike redesign
● Different UA channels
● New creatives
● Different spending strategy
● Passively through game related efforts e.g. Economy Balancing
...
● Optimization of UA investments through LTV prediction - focus of this talk

7. Warsaw | 14-15 October 2019
LTV predictions as an actionable insight for UA team
Comes early in the
campaign lifecycle
Helps to identify
opportunities
Accuracy is
stable over time
Supports decision
making
Defines strategy of
UA investment
optimization

8. Warsaw | 14-15 October 2019
Waiting for data is not fun
Usual approach for building LTV prediction model:
● wait several weeks or months to gather data about the campaigns
● create LTV model
● use the model to estimate UA performance
.. but market changes over time, new competitors arise etc

9. Warsaw | 14-15 October 2019
The Question
How can we improve UA efforts with LTV predictions
● early in campaigns lifecycle?
● without need of waiting several months to understand performance trend?
=> by understanding monetization of the game and building predictions from
bottom-up
How to actually do it?
● utilizing cloud machine learning

10. Warsaw | 14-15 October 2019
Some games have steep early monetization which creates an advantage in
modeling.
Game monetization defines data needed for predictions
= Game monetization => defines how many days of data we
need and helps to understand the payback of UA
50% of revenue in first couple of days =
enough data to work with

11. Warsaw | 14-15 October 2019
Variation in campaign cohorts performance = accuracy loss
=> Accuracy of models is impacted by daily market changes,
this can be seen on significant movements in campaign’s cohorts performances
over time

12. Warsaw | 14-15 October 2019
Building accurate predictions
Focusing on daily cohorts => cohort’s distribution of
payers does not change over time which improves
accuracy while building up the predictions:
1. Predictions based on cohort level not on campaign
level
2. Aggregate predictions for each campaign
Idea is to go hierarchically to the lowest level possible
(cohort/ad/adset etc) while:
- Keeping enough players in cohorts
= Daily cohorts => logarithmic growth can be leveraged => no need to wait
several months for data

13. Warsaw | 14-15 October 2019
Keeping enough players in cohorts
Number of players needed is based on:
- Conversion and number of payers
Unusual growth of revenue in the cohort can be the clue of not having enough players to
work with.
Looks like step function - reason could be:
1. Game relying mainly on Liveops offers (not so good design)
2.Conversion for the cohort is not high enough

14. Warsaw | 14-15 October 2019
UA Optimization
LTV prediction in Action

15. Warsaw | 14-15 October 2019
Information we are able to use:
- Monetary : Conversion, ARPU, ARPPU, # of purchases, revenue, payers
distribution, probability of 2nd purchase etc
- Absolute value
- Relative change/growth over first couple of days
- per cohort or campaign and aggregated
- Behavioral (Engagement metrics) : playtime, retention, first day
drop-off, percentage of one-shots etc
Leveraging all the data

16. Warsaw | 14-15 October 2019
Business optimization goal

17. SuperScale Analytics + UA Stack
Data Sources
(downloaded daily)
Game Data
(Appsflyer, Google Firebase, ...)
Attribution Platform
(Appsflyer, ...)
UA Channel Data
(Facebook, Google UAC, Ad-n...)
Store Revenue Data
(G Play, iTunes)

18. SuperScale Analytics + UA Stack
Data Sources
(downloaded daily)
Data Storage Materialized Views
Google BigQuery
UA Campaign
Evaluation MVs
Game Data
(Appsflyer, Google Firebase, ...)
Attribution Platform
(Appsflyer, ...)
UA Channel Data
(Facebook, Google UAC, Ad-n...)
Store Revenue Data
(G Play, iTunes)
Google Cloud Platform Machine Learning
(stochastic algorithms, random forest,
k-means clustering, …)
SBDW ETL Processing
(User States,
Materialized Views)

19. SuperScale Analytics + UA Stack
Data Sources
(downloaded daily)
Data Storage Materialized Views Application
Google BigQuery
Data Validator
(store revenue vs app revenue,
AF revenue vs store)
ROI/ROAS Spreadsheets
UA Campaign
Evaluation MVs
Game Data
(Appsflyer, Google Firebase, ...)
Attribution Platform
(Appsflyer, ...)
UA Channel Data
(Facebook, Google UAC, Ad-n...)
Store Revenue Data
(G Play, iTunes)
FB Campaign Optimization
Data Visualisation
Campaign
LTV/ROI prediction
Google Cloud Platform Machine Learning
(stochastic algorithms, random forest,
k-means clustering, …)
SBDW ETL Processing
(User States,
Materialized Views)

20. Warsaw | 14-15 October 2019
Building predictions from a bottom-up
Daily cohort 1
Daily cohort 2
Daily cohort 3
..
UA DATA PREDICTIONS
for each cohort
AGGREGATION of cohorts predictions on a
higher level
..
Weekly
Monthly
Quarterly
Campaign
Overall
CPI
LTV
Conversion,
ARPU, ARPPU,
# purchases,
revenue,
payers
distribution
(absolute values,
relative values)
Campaign success
prediction
Weighting of data
points based on
recency

21. Warsaw | 14-15 October 2019
Cloud pipeline takes care of automatic scaling of projects.
Cloud prediction pipeline
Data warehouse Batch preprocessing Machine learning
BigQuery
Creating models for:
Daily, weekly, monthly cohorts
and campaigns
Additional analysis:
Geo analysis
Break-even analysis
Business Intelligence
Data Studio
Scheduler
Cloud Functions
Models storage
Google Cloud
Storage
Google
Dataflow
ML Engine on
AI platform
Raw GAME Data +
Campaign data +
LKLs data
Periscope

22. Warsaw | 14-15 October 2019
Supporting decision making - examples
Weekly
cohort
estimate
d LTV
IAP D7 ROAS performance per country Weekly cohort LTV performance estimation Campaigns breakeven analysis
Top country to target for LKLs Which strategy was the best When can we expect payback

23. Warsaw | 14-15 October 2019
- LTV/ROAS predictions can help to identify opportunities but timing is the
factor that makes or breaks any modelling
- Aggregating weekly and monthly predictions from daily ones can improve
accuracy significantly
- Weighting data points based on recency can improve accuracy of
estimations
Takeaways

24. Warsaw | 14-15 October 2019
Thank you for attention.
Questions?