Delivered at Casual Connect Europe 2016 One of the key elements of user acquisition in mobile is to make sure that the creative (either videos or static/animated images) performs well. At Peak Games, while growing our products globally through paid acquisition, the importance given to creative optimization is huge. For some of our products we create more than 500 different patterns for different creatives, test, iterate and improve them constantly to make sure a certain performance standard holds. This session will be on how to create a good performing set of creatives for a certain product, from qualitative and quantitative analysis on understanding the users, to AB testing different patterns of creatives. The session will also try to inform the audience on how to analyze a good performing creative, how to optimize the campaigns and iterate the visuals according to previous AB test results, and implement a similar approach to App Store pages with a detailed explanation on why creative optimization is one of the key elements during this process.

1. AB Testing & Optimization for Creatives in
Mobile User Acquisition
Erdem İnan
Director, BI & Marketing
Peak Games

2. Overview
• Peak Games & Marketing Team & Myself
• Why Creative Optimization is important?
• Creative Testing Process: From Planning to Execution
• Real Life Cases on Creative Optimization
• Q&A

3. Peak Games
• Founded in October 2010, based in Istanbul / Turkey, a team of 100 people
• 300M+ Installs globally, 2 Games in Top 100 Grossing in US Stores, 2
Games in Top 5 Grossing in TR Stores.
• Active in 150+ Countries with 10+ Titles in Synchronous Multiplayer &
Casual Categories
• Flagship Titles: Toy Blast (US), Spades Plus (US), 101 Okey Plus (TR), Okey
Plus (TR)

4. Marketing Team & Myself
• BI & Marketing Team
– A Team of 15 People: Analytics, User Acquisition & Art, Big Data Team
– Providing services to 10+ products in User Acquisition, CRM and
Analytics
– 10+ Game Launches; Regular User Acquisition Campaigns in many
countries at large scale.
• Myself:
– 3.5 years at Peak Games.
– Leading the User Acquisition + CRM + Analytics teams
– Engineering Background with an Economics MSc.

5. Why Creative Optimization is important?
Ad Impression
Ad Click
Ad Install
CTR
CVR
cpm
cpc
cpi
• Publishers want to
maximize the revenue
from their ad inventory,
advertisers should be
giving them highest cpms.
• The system is based on
eCPM maximization
Advertisers /
Metrics
Advertiser A Advertiser B Advertiser C
Target CPI 3 USD 2 USD 10 USD
Impressions 1000 1000 1000
CTR 10% 20% 5%
Clicks 100 200 50
CVR 15% 25% 10%
Installs 15 50 5
Expected Cost 45 USD 100 USD 50 USD
Calculated CPM 45 USD 100 USD 50 USD
• Advertiser B wins the auction, Advertiser C gets the
second place, Advertiser A gets the third place.

6. Why Creative Optimization is important?
• Better creatives can lead to higher eCPMs =>
a. Can lead to higher scale at same rates
b. Can lead to cheaper installs.
c. Can put you in an earlier impression in most ad networks: (Earlier
rewarded video impressions have higher CTRs and CVRs which will lead
even better eCPMs and create a self-fulfilling loop)
d. Can even lead to higher quality installs.

7. Creative Testing Process: Understanding Users
• Research:
– Facebook Audience Insights
– Google Search Trends
– Twitter Hashtags
– Profiling Competitors’ Audiences
– Searching Web
• Competitors’ Creatives:
– Tools to check competitors’ best
performing creatives
• Creating a Typical Player:
• Geographics: US, UK, CA
• Demographics: F, 45+
• Other Interests:
• Country Music, Bingo, Slots, Puzzle
Games
• Music Shows, TV Shows, Celebrities
• Things to consider
• Politically conservative
• Raising Grandkids
• Middle Income
• More affinity: IL, TX, OH
• Religious…

8. Creative Testing Process: Executing Tests
Ad Pattern 1 v1.0
Ad Pattern 2 v1.0
Ad Pattern 3 v1.0
Ad Pattern 4 v1.0
Ad Pattern 5 v1.0
Ad Pattern 2 v1.0
Ad Pattern 2 v1.1
Ad Pattern 2 v1.2
Ad Pattern 5 v1.2
Ad Pattern 5 v1.0
Ad Pattern 5 v1.1
Ad Pattern 5 v1.2.3
Ad Pattern 5 v1.2
Ad Pattern 5 v1.2.2
• Some Ground Rules:
– Total image test budget should not exceed
x% of the total budget
– ~30 installs per creative is usually enough for
a rough evaluation.
– Use the previous test know-how in creating
next creatives. (buttons, cta, banners,
backgrounds etc.)
– All creatives should run at the same time, for
the same duration, either with same budget,
or with same impressions

9. Creative Testing Process: Reporting & Analysis
• Finding an exceptional creative instead of many good creatives
• Mind the “risky” inverse correlation between CTR and CVR
• Remember the second and third best performing: Ad Fatigue
• Checking Significance
• Chartboost, Unity Ads, Facebook for easy & proper AB Testing.

10. Case: Image Optimization towards CPI
• Gin Rummy Plus => Image Optimization towards lower CPI
• Fail: Lower CPI does not necessarily mean lower CPE.
• Above three has better CTRxCVR and lower CPI to our winning creatives
• Misleading creatives can lead to lower % Actives thus higher CPEs
• 30% lower performance in CPEs compared to our regular creatives.

11. Creative Testing: Iterating Images
1
2
4
3
5
• Discussing each experiments
within the team.
• Decomposing Images: What
made this image work?
1. CTA in the image
2. Rubic Cube Item
3. Game Play Background
4. Shiny Play Button
5. Contrast between background
and foreground items
• Create iterations of each
element.
• Test again!

12. Case: Calibrating between CTR and CVR
Base Image: Very Good CVR 1st Iteration: Increasing CTR 2nd Iteration: Further Increasing CTR
Pattern 1: Good CTR, Good CVR Pattern 2: High CTR, Low CVR Pattern 3: Above Average CTR & CVR

13. Case: Reaction to Different Creatives (1/2)
United States
CTR: 1.55%
CR: 33%
Turkey
CTR: 1.58%
CR: 28%
Netherlands
CTR: 2.03%
CR: 34%
United States
CTR: 0.88%
CR: 36%
Turkey
CTR: 3.44%
CR: 33%
Netherlands
CTR: 1.33%
CR: 37%
• Cultural Similarity & Localization in Ads are crucial in creative performance
• Finding the right creatives for a segment of users may cut costs easily by 40-50%.

14. Case: Reaction to Different Creatives (2/2)
Good Performers Bad Performers
Turkey
Italy
US
• There are absolute winners & losers in creatives; but cultural effects can kick in.
• Expectations & Prejudices might work: But A/B Testing for each region is critical

15. Case: Preparing Creatives for Different Times of the Year
• Easter, Valentine’s Day, 4th July, Month of Eid, Halloween Christmas
are extraordinary days for users. Make sure you respond that.
• Such specific days in calendar might affect costs, available traffic and
virality of ads

16. Creative Testing: Things to Know
• 500+ creatives for a mid-large scale game
• Different ad units, channels & sizes might have different winning creatives.
• Interstitials and banners usually have different winning patterns.
• Dropping CTR with the same volume of installs is a warning sign to change
creatives.
• In creative testing the prices will be higher than an optimal campaign.
• Emerging markets might have bandwidth problems => Make sure you have
lower sized creatives as well.
• Video Testing:
– Video networks are extra sensitive to creative performance
– Videos are difficult to test.
– Test concepts => Make sure execution is as good as possible in each concept.
– Thumbnail image has a high effect on performance.

17. Wrapping Up
• Creative Optimization brings greater scale at a cheaper price
• Success in creative optimization needs a systematic approach
and a constant effort with a strong team.
• There are more failures than successes in creative optimization.
• Ad Creatives are cultural, contextual and usually a composition
of certain elements.

18. Questions?
Erdem Inan
Director, BI & Marketing @ Peak Games
erdem@peakgames.net