Winning Data Science Competitions
•
24 likes•13,876 views
Presented by Jeong-Yoon Lee at Microsoft on 3/29/2017.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to Winning Data Science Competitions
Similar to Winning Data Science Competitions (20)
Recently uploaded
Recently uploaded (20)
Winning Data Science Competitions
- 1. Winning Data Science Competitions 3. 29. 2017 Jeong-Yoon Lee, Ph.D.
- 2. Chief Data Scientist, Conversion Logic 70+ Competitions 6 Times Prize Winner (KDD Cup 2012 & 2015) 8 Top 10 Finishes (Deloitte, AARP, Liberty Mutual) Top 10, Kaggle 2015 Father of 4 boys Jeong-Yoon Lee, Ph.D.
- 3. About Conversion Logic 3 Advanced Marketing Attribution For Diverse Customers
- 4. Why Data Science Competition
- 5. Why Compete For fun For experience For learning For networking 5
- 6. Fun Competing with others Continuous improvement 6
- 7. Experience 7
- 8. Learning 8
- 9. Learning 9
- 10. Networking 10
- 11. 11
- 13. Data Science Competitions Since 1997 2006 - 2009 Since 2010
- 14. Competition Structure Training Data Test Data Feature Label Provided Submission Public LB Score Private LB Score
- 15. Kaggle 250+ competitions since 2010 900K users 50K+ competitors $3MM+ prize paid out
- 16. Kaggle
- 17. Kaggle
- 19. Misconceptions on Competitions No ETL No EDA Not worth it Not for production 19
- 20. No ETL? - Deloitte Western Australia Rental Prices 20
- 21. No ETL? - Outbrain Click Prediction 21 2B page views. 16.9MM clicks. 700MM users. 560 sites
- 22. No ETL? - YouTube-8M Video Understanding Challenge 22 1.7TB feature-level data. 31GB video-level data.
- 23. No ETL? 23
- 24. No EDA? Most of competitions provide actual labels - typical EDA Anonymized data - more creative EDA o People decode age, states, time intervals, income, etc. 24
- 25. No EDA? Anonymized data - more creative EDA 25
- 26. Not worth it? Performance matters You walk easier when you can run 26
- 27. Not for Production? Kaggle Kernel o Max execution time:10 minutes o Max file output: 500MB o Memory limit: 8GB 27
- 28. Ensemble Pipeline at Conversion Logic 28
- 29. Best Practices
- 30. Best Practices Feature Engineering Diverse Algorithms Cross Validation Ensemble Collaboration 30
- 31. Feature Engineering 31 Types Note Numerical Log, Log2(1 + x), Box-Cox, Normalization, Binning Categorical One-hot-encoding, Label-encoding, Count, Weight-of-Evidence Text Bag-of-Words, TF-IDF, N-gram, Character-n-gram, K-skip-n-gram Timeseries/ Sensor data Descriptive Statistics, Derivatives, FFT, MFCC, ERP Network Graph Degree, Closeness, Betweenness, PageRank Numerical/ Timeseries Convert to categorical features using RF/GBM Dimensionality Reduction PCA, SVD, Autoencoder, Hashing Trick Interaction Addition/substraction/mutiplicaiton/division. Hashing Trick * More comprehensive overview on feature engineering by HJ van Veen: https://www.slideshare.net/HJvanVeen/feature-engineering-72376750
- 32. Diverse Algorithms Algorithm Tool Note Gradient Boosting Machine XGBoost, LightGBM The most popular algorithm in competitions Random Forests Scikit-Learn, randomForest Used to be popular before GBM Extremely Random Trees Scikit-Learn Neural Networks/ Deep Learning Keras, MXNet, Torch, CNTK Blends well with GBM. Best at image and speech recognition competitions Logistic/Linear Regression Scikit-Learn, Vowpal Wabbit Fastest. Good for ensemble. Support Vector Machine Scikit-Learn FTRL Vowpal Wabbit Competitive solution for CTR estimation competitions Factorization Machine libFM, fastFM Winning solution for KDD Cup 2012 Field-aware Factorization Machine libFFM Winning solution for CTR estimation competitions (Criteo, Avazu) 32
- 33. Cross Validation Training data are split into five folds where the sample size and dropout rate are preserved (stratified). 33
- 35. Ensemble - Stacking * for other types of ensemble, see http://mlwave.com/kaggle-ensembling-guide/ 35
- 37. Collaboration
- 38. Collaboration – Git Repo + S3/Dropbox 38
- 39. Collaboration – Common Validation 39
- 40. Collaboration – Internal Leaderboard 40
- 41. Best Practices For fun For experiences For learning For networking 41 Feature Engineering Diverse Algorithms Cross Validation Ensemble Collaboration Why Competition
- 42. Things That Help 42 Keep competition journals and repos – both during and after competitions Build and improve the automated pipeline and library for competitions • https://github.com/jeongyoonlee/Kaggler • https://gitlab.com/jeongyoonlee/allstate-claims-severity/tree/master • http://kaggler.com/kagglers-toolbox-setup/ Be humble, and ready to try and learn something new Make a commitment and work on competitions no matter what on a regular basis
- 43. Resources 43 No Free Hunch by Kaggle Winning Tips on Machine Learning Competitions by Marios Michailidis (KazAnova) Feature Engineering, mlwave.com by HJ van Veen (Triskelion) fastml.com by Zygmunt Zając (Foxtrot) kaggler.com, facebook.com/Kaggler by Jeong-Yoon Lee @ CL and Hang Li @ Hulu Tianqi Chen @ UW – Won KDDCup 2012, DSB 2015. Author of XGBoost, MXNet Gilberto Titericz Junior in San Francisco - #1 at Kaggle
- 44. Active Competitions 44 Kaggle – 6 Featured, 1 Job Competitions KDD Cup 2017 RecSys Challenge 2017 CIKM AnalytiCup 2017
- 45. Thank You