This document discusses predictive marketing and machine learning models for predicting user behavior in mobile apps. It describes common challenges with predictive models, including consistency and noise in datasets. It also outlines the process of developing a machine learning model, including collecting event data, cleaning the dataset, developing the model, and challenges during production implementation and maintenance of the model over time.

1. Predictive marketing in a
nutshell

2. Coupons Shopping listsDiscounts nearby
Edadeal

3. WAU

4. WAU
something
went wrong

5. Reasons
• CPI increased rapidly

6. Reasons
• CPI increased rapidly
• Pricy and time-consuming to test new hypotheses

7. Reasons
• CPI increased rapidly
• Pricy and time-consuming to test new hypotheses
• Target event happens on 14th day since install

8. The funnel

9. The funnel

10. Target metric

13. Data
• Over 500 distinct in-app events
• Lots of custom in-app event parameters
• Lots of successful user scenarios

14. 1 Session Logs Machine Learning
RR1w
predicted
Blackbox

15. Problem #1
Consistency, necessity and sufficiency

17. Problem #2
Dataset cleansing

18. Dataset cleansing
• Reduce the number of noisy features
• Get rid of compromised users
• Balance the dataset

19. Problem #3
Changing over time

20. Changing over time

21. Problem #4
Which actually doesn’t exist

22. ML Model
* Zalando blog

23. ML Model
* Zalando blog

24. Process

25. Traffic
Source
subpub_id

26. Traffic
Source
subpub_id user

27. Traffic
Source
subpub_id user
events

28. Traffic
Source
subpub_id user
events
events

29. Traffic
Source
subpub_id user
events
events
First session events

30. Traffic
Source
subpub_id user
events
events
First session events

31. Traffic
Source
subpub_id user
events
events
First session events
postback

32. Production

33. Production. Challenge the model

34. Production
Start of the
campaigns

35. Production
Downtime
for a week
Start of the
campaigns

36. Production
Downtime
for a week
Start of the
campaigns
Fine-tuning

37. Production
Downtime
for a week
Start of the
campaigns
Fine-tuning
Stability

38. The result
что-то пошло
не так
something
went wrong

39. The result
no, np
something
went wrong

40. Homework
• YDF & World of Tanks case

41. Homework
• Zalando case

42. Homework
• Jelly Splash! case

43. Q&A
@maratz