Machine learning at Criteo - Paris Datageeks
•Download as PPTX, PDF•
18 likes•20,983 views
Criteo is an advertising company that displays ads on websites for its partners. It uses machine learning for click prediction models and product recommendations. Criteo has a large infrastructure across multiple data centers and handles over 30 billion HTTP requests per day. The presentation discusses the challenges of building machine learning models at Criteo's scale for real-time bidding and recommendations, such as dealing with many variables, changing products, and different timeframes.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Machine learning at Criteo - Paris Datageeks
Similar to Machine learning at Criteo - Paris Datageeks (20)
Recently uploaded
Recently uploaded (20)
Machine learning at Criteo - Paris Datageeks
- 1. Shallow learning @ Criteo Nicolas Le Roux Scientific Program Manager – R&D
- 2. I N T R O D U C T I O N Copyright © 2013 Criteo
- 3. WHAT DOES CRITEO DO? 3 • We buy advertising space on websites • We display ads for our partners • We get paid if the user clicks on the ad Copyright © 2013 Criteo
- 4. HOW DO WE BUY ADVERTISING SPACE? 4 • Real-time bidding (RTB): it is an auction • Second-price • Optimal strategy: bid the expected gain • Expected gain = CPC * CTR Copyright © 2013 Criteo
- 5. WHAT TO DO ONCE WE WIN THE DISPLAY? 5 • Choose the best products • Choose the color, the font, the layout… • Generate the banner Copyright © 2013 Criteo
- 6. WHERE IS THE ML AT CRITEO? 6 • Click prediction model • Product recommendation • Testing, … Copyright © 2013 Criteo
- 7. WHERE IS THE ML AT CRITEO? 7 • Click prediction model • Product recommendation • Testing, … Copyright © 2013 Criteo
- 8. INFRASTRUCTURE AT CRITEO 8 • 5 data centers on 3 continents • More than 3 500 servers • More than 30B HTTP requests per day • More than 2B banners displayed per day • Peak traffic: 500K HTTP requests per second Copyright © 2013 Criteo
- 9. C L I C K P R E D I C T I O N M O D E L Copyright © 2013 Criteo
- 10. CTR PREDICTION MODEL 10Copyright © 2013 Criteo
- 11. CTR PREDICTION MODEL 11Copyright © 2013 Criteo
- 12. CTR PREDICTION MODEL 12Copyright © 2013 Criteo
- 13. CTR PREDICTION MODEL 13Copyright © 2013 Criteo
- 14. CTR PREDICTION MODEL 14Copyright © 2013 Criteo
- 15. 15-SECOND ML CRASH COURSE 15Copyright © 2013 Criteo
- 16. NEW HIRE REACTION 16Copyright © 2013 Criteo
- 17. NEW HIRE REACTION 17Copyright © 2013 Criteo
- 18. NEW HIRE REACTION 18Copyright © 2013 Criteo
- 19. NEW HIRE REACTION 19Copyright © 2013 Criteo
- 20. 20Copyright © 2013 Criteo
- 21. 21Copyright © 2013 Criteo
- 22. 22 • Merchant ID is too raw • We can learn fancy features for each merchant: demographics, size, ... • Hard to not penalize large merchants Copyright © 2013 Criteo
- 23. OPTIMIZING THE MODEL 23Copyright © 2013 Criteo
- 24. DEALING WITH MANY VARIABLES 24Copyright © 2013 Criteo
- 25. DEALING WITH MANY VARIABLES 25Copyright © 2013 Criteo
- 26. OTHER CHALLENGES 26 • How long do you wait for sales after a click? • How much in the past do you go to learn the model? • Do you value more long term or short term variables? • How to choose the best variables? Copyright © 2013 Criteo
- 27. P R O D U C T R E C O M M E N D A T I O N Copyright © 2013 Criteo
- 28. THE CHALLENGES OF PRODUCT RECOMMENDATION 28Copyright © 2013 Criteo
- 29. THE CHALLENGES OF PRODUCT RECOMMENDATION 29Copyright © 2013 Criteo
- 30. NEW HIRE REACTION 30 • « You should use [his PhD topic], it’s awesome! » Copyright © 2013 Criteo
- 31. NEW HIRE REACTION 31 • « You should use [his PhD topic], it’s awesome! » • « It just takes a week to train. » Copyright © 2013 Criteo
- 32. NEW HIRE REACTION 32 • « You should use [his PhD topic], it’s awesome! » • « It just takes a week to train. » • « In real-time, you just have to score every product. » Copyright © 2013 Criteo
- 33. MAJOR HURDLES 33 • Products come and go • There might be a sale (Black Friday) • The « panties syndrome » • Complementary vs. similar products Copyright © 2013 Criteo
- 34. OTHER CHALLENGES 34 • Multiple products in a banner • Interaction between products and layout • Different timeframes for different products Copyright © 2013 Criteo
- 35. A FIRST RECIPE FOR SUCCESS 35 • There are many sources of success/failure • It is often suboptimal to focus on one • The first step to address each source is often manual Copyright © 2013 Criteo
- 36. CONCLUSION 36 • Prediction is at the core of our business • Huge engineering constraints • Different bottlenecks than in academia • Build from the ground up, not the other way around Copyright © 2013 Criteo
- 37. Nicolas: n.leroux@criteo.com Lisa: l.sarny@criteo.com Clem: c.daines@criteo.com QUESTIONS ?
Editor's Notes
- We’re really happy to have you all here. On a completely unrelated note, we’re hiring. If you’d like to know more, you’re welcome to chat with the two lovely ladies: Lisa and Clementine.Avant de commencer, je voulaisvousremercier d’être venus, on estravis de vousavoirici. A ce propos, je tenais à signaler qu’onrecrute. Si vousêtesintéressé, n’hésitez pas à discuter avec les deuxcharmantespersonnes qui sontlà pour ça: Lisa et Clementine.
- An example of phi is: rather than pick the time of day, split between “morning, work hours” and “evening”.
- I’m using the masculine form because Criteo is unfortunately slightly gender-biased.
- Every time your bid is slower, you lose money: timeout, RTB platform does not want to deal with you anymore, ...This is not image recognition. Keep it simple: a tree, a linear model, ...
- SGD is not advantageous because, when you have 10 times more data, you don’t have that much more information
- SGD is not advantageous because, when you have 10 times more data, you don’t have that much more information
- SGD is not advantageous because, when you have 10 times more data, you don’t have that much more information
- Go in the past: business team started a new campaign with high CPC which flooded your dataset.Long term vs short term: expensive computations vs cheap onesChoose the best variables: each variable adds latency.Conclusion: there are many more questions than “What is the best f I can think of?”
- I’m using the masculine form because Criteo is unfortunately slightly gender-biased.
- I’m using the masculine form because Criteo is unfortunately slightly gender-biased.
- SVD, NNMF, LDA, Bayesian Matrix Factorization, Sparse Coding, ...
- Panties: if you create a startup, display panties.
- There are instant buys and other products which need a bigger thought. DVD vs. car
- There are instant buys and other products which need a bigger thought. DVD vs. car
- It is not about “How fast can you go with a complicated model?” but “How complex can you go within the resource budget?”