Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Takeaways from ICML 2019 Hong Kong Machine Learning Meetup Season 1 Episode 12 – Season Finale Gautier Marti Gautier Marti...
Table of contents 1 Day 1 - Tutorials 2 Day 2 - U.S. Census, Time Series, Hawkes Processes, Shapley values, Topological Da...
Disclaimer I was sent to the ICML 2019 conference by my employer Shell Street Labs. However, the opinions expressed in thi...
Section 1 Day 1 - Tutorials Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 4 / 41
Subsection 1 Safe Machine Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 5 / 41
Safe Machine Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 6 / 41
COMPAS - An example of ML biases & misspecification We show, however, that the widely used commercial risk assessment softw...
RL in the Wild - Other examples of misspecifications Reinforcement learning algorithms can break in surprising, counterintu...
Robustness - Adversarial attacks Robust Physical-World Attacks on Deep Learning Visual Classifica- tion https://arxiv.org/p...
Subsection 2 Active Learning: From Theory to Practice Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 10 / 41
Active Learning: From Theory to Practice Slides for the tutorial: http://nowak.ece.wisc.edu/ActiveML.html Active Learning ...
Active Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 12 / 41
Rethinking classical model generalization Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 13 / 41
Subsection 3 A Tutorial on Attention in Deep Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 14 / 41
A Tutorial on Attention in Deep Learning Slides: http://alex.smola.org/talks/ICML19-attention.pdf https://www.d2l.ai/ http...
Section 2 Day 2 - U.S. Census, Time Series, Hawkes Processes, Shapley values, Topological Data Analysis, Optimal Transport...
Subsection 1 Differential privacy Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 17 / 41
U.S. Census & Differential privacy Good related read: https://www.sciencemag.org/news/2019/01/ can-set-equations-keep-us-ce...
Subsection 2 Time Series Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 19 / 41
Deep Factors for Forecasting https://arxiv.org/pdf/1905.12417.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 20...
Hawkes Processes ICML 2018 tutorial: http://learning.mpi-sws.org/tpp-icml18/ http://proceedings.mlr.press/v97/trouleau19a/...
Subsection 3 Shapley values: Explainability & Data Valuation Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 22...
Shapley values A new trend in ML based on: Shapley, Lloyd S. A value for n-person games. Contributions to the Theory of Ga...
Subsection 4 Topological Data Analysis Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 24 / 41
Topological Data Analysis Already a tutorial at ICML 2013: Topological Data Analysis and Machine Learning http://www2.stat...
Subsection 5 Optimal Transport Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 26 / 41
Optimal Transport Trending in the ML community (at least 5 ICML 2019 papers) since Cuturi’s 2013 NIPS paper: Sinkhorn Dist...
Section 3 Day 3 - Robotics, Gaussian Processes, Learning with noisy labels Gautier Marti (Shell Street Labs) Takeaways fro...
Subsection 1 Machine Learning with Application to Robotics Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 29 /...
Machine Learning with Application to Robotics Solving complex PDEs and other stochastic control problems in real time is n...
Subsection 2 Gaussian Processes Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 31 / 41
Gaussian Processes One flagship project: The Automatic Statistician https://www.automaticstatistician.com/index/ Discoverin...
Subsection 3 Labels. . . Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 33 / 41
Labels. . . Learning Dependency Structures for Weak Supervision Models https://arxiv.org/pdf/1903.05844.pdf Gautier Marti ...
Section 4 Day 4 - Interpretability, Natural Language Processing Gautier Marti (Shell Street Labs) Takeaways from ICML 2019...
Subsection 1 Interpretability Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 36 / 41
Interpretability Towards a Deep and Unified Understanding of Deep Neural Models in NLP http://proceedings.mlr.press/v97/gua...
Subsection 2 Natural Language Processing Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 38 / 41
Natural Language Processing MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization https://arxi...
Section 5 Day 5 - Workshop Time Series Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 40 / 41
GluonTS GluonTS: Probabilistic Time Series Models in Python https://arxiv.org/pdf/1906.05264.pdf https://github.com/awslab...
Upcoming SlideShare
Loading in …5
×

Takeaways from ICML 2019, Long Beach, California

1,068 views

Published on

A few slides that highlight some of my personal takeaways from the ICML 2019 conference. I tried to identify niche trends such as Shapley values, topological data analysis, Hawkes processes...

Published in: Data & Analytics
no profile picture user

  • Login to see the comments

  • Be the first to like this

Takeaways from ICML 2019, Long Beach, California

  1. 1. Takeaways from ICML 2019 Hong Kong Machine Learning Meetup Season 1 Episode 12 – Season Finale Gautier Marti Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 1 / 41
  2. 2. Table of contents 1 Day 1 - Tutorials 2 Day 2 - U.S. Census, Time Series, Hawkes Processes, Shapley values, Topological Data Analysis, Optimal Transport for Graphs 3 Day 3 - Robotics, Gaussian Processes, Learning with noisy labels 4 Day 4 - Interpretability, Natural Language Processing 5 Day 5 - Workshop Time Series Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 2 / 41
  3. 3. Disclaimer I was sent to the ICML 2019 conference by my employer Shell Street Labs. However, the opinions expressed in this presentation are my own and do not reflect in any ways the view of my employer. Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 3 / 41
  4. 4. Section 1 Day 1 - Tutorials Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 4 / 41
  5. 5. Subsection 1 Safe Machine Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 5 / 41
  6. 6. Safe Machine Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 6 / 41
  7. 7. COMPAS - An example of ML biases & misspecification We show, however, that the widely used commercial risk assessment software COMPAS is no more accurate or fair than predictions made by people with little or no criminal justice expertise. The accuracy, fairness, and limits of predicting recidivism https://advances.sciencemag.org/content/4/1/eaao5580 Our analysis of Northpointe’s tool, called COMPAS (which stands for Correctional Offender Management Profiling for Alternative Sanctions), found that black defendants were far more likely than white defendants to be incorrectly judged to be at a higher risk of recidivism, while white defendants were more likely than black defendants to be incorrectly flagged as low risk. How We Analyzed the COMPAS Recidivism Algorithm https://www.propublica.org/article/ how-we-analyzed-the-compas-recidivism-algorithm GitHub: https://github.com/propublica/compas-analysis Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 7 / 41
  8. 8. RL in the Wild - Other examples of misspecifications Reinforcement learning algorithms can break in surprising, counterintuitive ways. Faulty Reward Functions in the Wild https://openai.com/blog/faulty-reward-functions/ Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 8 / 41
  9. 9. Robustness - Adversarial attacks Robust Physical-World Attacks on Deep Learning Visual Classifica- tion https://arxiv.org/pdf/ 1707.08945.pdf Fooling automated surveillance cameras: adversarial patches to attack person detection https://arxiv.org/pdf/ 1904.08653.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 9 / 41
  10. 10. Subsection 2 Active Learning: From Theory to Practice Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 10 / 41
  11. 11. Active Learning: From Theory to Practice Slides for the tutorial: http://nowak.ece.wisc.edu/ActiveML.html Active Learning tries to answer the question: Can we train machines with less labeled data and less human supervision? Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 11 / 41
  12. 12. Active Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 12 / 41
  13. 13. Rethinking classical model generalization Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 13 / 41
  14. 14. Subsection 3 A Tutorial on Attention in Deep Learning Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 14 / 41
  15. 15. A Tutorial on Attention in Deep Learning Slides: http://alex.smola.org/talks/ICML19-attention.pdf https://www.d2l.ai/ https://github.com/d2l-ai Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 15 / 41
  16. 16. Section 2 Day 2 - U.S. Census, Time Series, Hawkes Processes, Shapley values, Topological Data Analysis, Optimal Transport for Graphs Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 16 / 41
  17. 17. Subsection 1 Differential privacy Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 17 / 41
  18. 18. U.S. Census & Differential privacy Good related read: https://www.sciencemag.org/news/2019/01/ can-set-equations-keep-us-census-data-private Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 18 / 41
  19. 19. Subsection 2 Time Series Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 19 / 41
  20. 20. Deep Factors for Forecasting https://arxiv.org/pdf/1905.12417.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 20 / 41
  21. 21. Hawkes Processes ICML 2018 tutorial: http://learning.mpi-sws.org/tpp-icml18/ http://proceedings.mlr.press/v97/trouleau19a/trouleau19a.pdf cf. tick for practitioners: https://github.com/X-DataInitiative/tick Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 21 / 41
  22. 22. Subsection 3 Shapley values: Explainability & Data Valuation Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 22 / 41
  23. 23. Shapley values A new trend in ML based on: Shapley, Lloyd S. A value for n-person games. Contributions to the Theory of Games 2.28 (1953). 158 citations. φi (v) = S⊆N{i} |S|!(N − |S| − 1)! N! (v(S ∪ {i}) − v(S)) Recent applications: Explainability: A Unified Approach to Interpreting Model Predictions http://papers.nips.cc/paper/ 7062-a-unified-approach-to-interpreting-model-predictions. pdf Data valuation: Data Shapley: Equitable Valuation of Data for Machine Learning http://proceedings.mlr.press/v97/ghorbani19c/ghorbani19c.pdf Since computationally intensive, many papers try to approximate these values fast... Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 23 / 41
  24. 24. Subsection 4 Topological Data Analysis Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 24 / 41
  25. 25. Topological Data Analysis Already a tutorial at ICML 2013: Topological Data Analysis and Machine Learning http://www2.stat.duke.edu/~sayan/Primoz/ICML.pdf A round-up of TDA papers at ICML 2019: https://bastian.rieck.me/blog/posts/2019/icml_tda_roundup/ (6 TDA-related papers) Take-home message: TDA can provide robust features to Machine Learning models. Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 25 / 41
  26. 26. Subsection 5 Optimal Transport Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 26 / 41
  27. 27. Optimal Transport Trending in the ML community (at least 5 ICML 2019 papers) since Cuturi’s 2013 NIPS paper: Sinkhorn Distances: Lightspeed Computation of Optimal Transport, which made Optimal Transport for Machine Learning possible in practice. Way too slow before! Theoretical contributions: On Efficient Optimal Transport: An Analysis of Greedy and Accelerated Mirror Descent Algorithms Methodology: Optimal Transport for structured data with application on graphs Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 27 / 41
  28. 28. Section 3 Day 3 - Robotics, Gaussian Processes, Learning with noisy labels Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 28 / 41
  29. 29. Subsection 1 Machine Learning with Application to Robotics Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 29 / 41
  30. 30. Machine Learning with Application to Robotics Solving complex PDEs and other stochastic control problems in real time is not really feasible as of today. Machine Learning (supervised learning of trajectories, learning from demonstration, etc.) can help robotics. Recorded talk: https://www.facebook.com/icml.imls/videos/2368059266588651/ Lab: http://lasa.epfl.ch/ Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 30 / 41
  31. 31. Subsection 2 Gaussian Processes Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 31 / 41
  32. 32. Gaussian Processes One flagship project: The Automatic Statistician https://www.automaticstatistician.com/index/ Discovering Latent Covariance Structures for Multiple Time Series https://arxiv.org/pdf/1703.09528.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 32 / 41
  33. 33. Subsection 3 Labels. . . Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 33 / 41
  34. 34. Labels. . . Learning Dependency Structures for Weak Supervision Models https://arxiv.org/pdf/1903.05844.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 34 / 41
  35. 35. Section 4 Day 4 - Interpretability, Natural Language Processing Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 35 / 41
  36. 36. Subsection 1 Interpretability Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 36 / 41
  37. 37. Interpretability Towards a Deep and Unified Understanding of Deep Neural Models in NLP http://proceedings.mlr.press/v97/guan19a/guan19a.pdf Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation http://proceedings.mlr.press/v97/ancona19a/ancona19a.pdf https://icml.cc/media/Slides/icml/2019/grandball(13-09-00) -13-09-25-4776-explaining_deep.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 37 / 41
  38. 38. Subsection 2 Natural Language Processing Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 38 / 41
  39. 39. Natural Language Processing MeanSum: A Neural Model for Unsupervised Multi-Document Abstractive Summarization https://arxiv.org/pdf/1810.05739.pdf https://icml.cc/media/Slides/icml/2019/104(13-11-00) -13-12-10-4891-meansum_a_neur.pdf Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 39 / 41
  40. 40. Section 5 Day 5 - Workshop Time Series Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 40 / 41
  41. 41. GluonTS GluonTS: Probabilistic Time Series Models in Python https://arxiv.org/pdf/1906.05264.pdf https://github.com/awslabs/gluon-ts Gautier Marti (Shell Street Labs) Takeaways from ICML 2019 41 / 41

×