SlideShare a Scribd company logo

NIPS 2016 Highlights - Sebastian Ruder

•
•
S
Sebastian Ruder

Talk at 4th NLP Dublin meetup (https://www.meetup.com/NLP-Dublin/events/233314527/) on highlights of NIPS 2016.

Science
1 of 46
Download to read offline
Sebastian Ruder
 PhD Candidate, Insight Centre
 Research Scientist, AYLIEN @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup NIPS 2016 Highlights
Agenda 1. NIPS overview 2. Generative Adversarial Networks 3. Building applications with Deep Learning 4. RNNs 5. Improving classic algorithms 6. Reinforcement Learning 7. Learning-to-learn / Meta-learning 8. General AI 9. NLP 10. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NIPS Overview @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: http://www.tml.cs.uni-tuebingen.de/team/luxburg/misc/nips2016/index.php
Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks
Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf
Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 • Mostly used in CV for image generation; discriminator can be used as feature extractor • Less work in NLP and other areas; promising directions at GAN workshop (1, 2, 3, 4) 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf 5https://arxiv.org/abs/1611.01144
Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
 being rejected from NIPS 1996
 — perseverance pays off
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for- long-or-event-based-sequences.pdf
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3
RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 •General DL improvements • Weight normalisation4 • Multinomial dropout5 •Edward Grefenstette’s talk at NAMPI workshop6 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf 4https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural- networks.pdf 5https://papers.nips.cc/paper/6561-improved-dropout-for-shallow-and-deep-learning.pdf 6https://uclmr.github.io/nampi/talk_slides/grefenstette_limitations_of_rnns.pdf
Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity3 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf 3https://papers.nips.cc/paper/6449-clustering-with-same-cluster-queries.pdf
Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN • New research environments: • OpenAI’s Universe • Deep Mind Lab • FAIR’s Torchcraft 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
Learning-to-learn / Meta-learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Several papers, e.g.
 Learning to learn by gradient descent by gradient descent1 • Meta-learning panel at RNN symposium • Sutskever: No good meta-learning models so far • Schmidhuber: Focus on universal model • Focus at Neural Abstract Machines workshop 1https://papers.nips.cc/paper/6461-learning-to-learn-by-gradient-descent-by-gradient-descent.pdf
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
General AI @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: Drew Purves
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain • Machine Intelligence workshop: • FAIR’s CommAI-env • Hierarchical representations, identifying patterns • Panel: Unclear if focus on specific tasks brings us closer to General AI.
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 • Dialogue modelling workshop: • End-to-end systems, linguistics, and ML methods 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere!
Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics
Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing
Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence
Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence • RocketAI “launch” party
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Thanks for your attention! Questions?

Recommended

Transfer Learning for Natural Language Processing
Transfer Learning for Natural Language ProcessingTransfer Learning for Natural Language Processing
Transfer Learning for Natural Language ProcessingSebastian Ruder
 
Successes and Frontiers of Deep Learning
Successes and Frontiers of Deep LearningSuccesses and Frontiers of Deep Learning
Successes and Frontiers of Deep LearningSebastian Ruder
 
Transfer Learning -- The Next Frontier for Machine Learning
Transfer Learning -- The Next Frontier for Machine LearningTransfer Learning -- The Next Frontier for Machine Learning
Transfer Learning -- The Next Frontier for Machine LearningSebastian Ruder
 
Challenges in transfer learning in nlp
Challenges in transfer learning in nlpChallenges in transfer learning in nlp
Challenges in transfer learning in nlpLaraOlmosCamarena
 
Transfer learning-presentation
Transfer learning-presentationTransfer learning-presentation
Transfer learning-presentationBushra Jbawi
 
Apply chinese radicals into neural machine translation: deeper than character...
Apply chinese radicals into neural machine translation: deeper than character...Apply chinese radicals into neural machine translation: deeper than character...
Apply chinese radicals into neural machine translation: deeper than character...Lifeng (Aaron) Han
 
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...Lifeng (Aaron) Han
 
ODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPindico data
 

Recommended

Transfer Learning for Natural Language Processing
Transfer Learning for Natural Language ProcessingTransfer Learning for Natural Language Processing
Transfer Learning for Natural Language ProcessingSebastian Ruder
 
Successes and Frontiers of Deep Learning
Successes and Frontiers of Deep LearningSuccesses and Frontiers of Deep Learning
Successes and Frontiers of Deep LearningSebastian Ruder
 
Transfer Learning -- The Next Frontier for Machine Learning
Transfer Learning -- The Next Frontier for Machine LearningTransfer Learning -- The Next Frontier for Machine Learning
Transfer Learning -- The Next Frontier for Machine LearningSebastian Ruder
 
Challenges in transfer learning in nlp
Challenges in transfer learning in nlpChallenges in transfer learning in nlp
Challenges in transfer learning in nlpLaraOlmosCamarena
 
Transfer learning-presentation
Transfer learning-presentationTransfer learning-presentation
Transfer learning-presentationBushra Jbawi
 
Apply chinese radicals into neural machine translation: deeper than character...
Apply chinese radicals into neural machine translation: deeper than character...Apply chinese radicals into neural machine translation: deeper than character...
Apply chinese radicals into neural machine translation: deeper than character...Lifeng (Aaron) Han
 
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than C...Lifeng (Aaron) Han
 
ODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPindico data
 
Document Classification with Neo4j
Document Classification with Neo4jDocument Classification with Neo4j
Document Classification with Neo4jKenny Bastani
 
Intro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringIntro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringTraian Rebedea
 
From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?Constantin Orasan
 
Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Tasnim Ara Islam
 
Practical machine learning - Part 1
Practical machine learning - Part 1Practical machine learning - Part 1
Practical machine learning - Part 1Traian Rebedea
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesMatthew Lease
 
Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Rich Heimann
 
Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Rich Heimann
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Rich Heimann
 
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Ahmed Magdy Ezzeldin, MSc.
 
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Association for Computational Linguistics
 
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systemsQi He
 
Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)YerevaNN research lab
 
Aspects of NLP Practice
Aspects of NLP PracticeAspects of NLP Practice
Aspects of NLP PracticeVsevolod Dyomkin
 
Practical NLP with Lisp
Practical NLP with LispPractical NLP with Lisp
Practical NLP with LispVsevolod Dyomkin
 
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systemsQi He
 
SF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonSF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonPaco Nathan
 
Recurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovRecurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovBhaskar Mitra
 
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsText, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsFred Morstatter
 
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisMicrosoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisAlex Polozov
 
Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05tracykteal
 
DMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can UseDMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can UseUniversity of California Curation Center
 

More Related Content

What's hot

Document Classification with Neo4j
Document Classification with Neo4jDocument Classification with Neo4j
Document Classification with Neo4jKenny Bastani
 
Intro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringIntro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringTraian Rebedea
 
From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?Constantin Orasan
 
Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Tasnim Ara Islam
 
Practical machine learning - Part 1
Practical machine learning - Part 1Practical machine learning - Part 1
Practical machine learning - Part 1Traian Rebedea
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesMatthew Lease
 
Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Rich Heimann
 
Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Rich Heimann
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Rich Heimann
 
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Ahmed Magdy Ezzeldin, MSc.
 
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Association for Computational Linguistics
 
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systemsQi He
 
Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)YerevaNN research lab
 
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systemsQi He
 
SF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonSF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonPaco Nathan
 
Recurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovRecurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovBhaskar Mitra
 
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsText, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsFred Morstatter
 
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisMicrosoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisAlex Polozov
 

What's hot (20)

Document Classification with Neo4j
Document Classification with Neo4jDocument Classification with Neo4j
Document Classification with Neo4j
 
Intro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringIntro to Deep Learning for Question Answering
Intro to Deep Learning for Question Answering
 
From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?
 
Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.
 
Practical machine learning - Part 1
Practical machine learning - Part 1Practical machine learning - Part 1
Practical machine learning - Part 1
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
 
Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)
 
Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
 
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
 
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
 
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
 
Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)
 
Aspects of NLP Practice
Aspects of NLP PracticeAspects of NLP Practice
Aspects of NLP Practice
 
Practical NLP with Lisp
Practical NLP with LispPractical NLP with Lisp
Practical NLP with Lisp
 
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
 
SF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonSF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in Python
 
Recurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovRecurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas Mikolov
 
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsText, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
 
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisMicrosoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
 

Similar to NIPS 2016 Highlights - Sebastian Ruder

Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05tracykteal
 
Building a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationBuilding a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationKe Tao
 
DIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDigCurV
 
DIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansDIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansEDINA, University of Edinburgh
 
DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14Carly Strasser
 
Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...LIBER Europe
 
Mid-term presentation.pdf
Mid-term presentation.pdfMid-term presentation.pdf
Mid-term presentation.pdfZixunZhou
 
Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24tracykteal
 
Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Lynn Connaway
 
Librarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineLibrarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineArden Kirkland
 
Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Shawna Reibling
 
OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC
 
Assessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandAssessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandPetri Linna
 
Building Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataBuilding Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataMarcia Zeng
 

Similar to NIPS 2016 Highlights - Sebastian Ruder (20)

Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05
 
DMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can UseDMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can Use
 
Building a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationBuilding a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result Diversification
 
DIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for Librarians
 
DIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansDIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for Librarians
 
Exploratory Analysis of User Data
Exploratory Analysis of User DataExploratory Analysis of User Data
Exploratory Analysis of User Data
 
DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14
 
Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...
 
Mid-term presentation.pdf
Mid-term presentation.pdfMid-term presentation.pdf
Mid-term presentation.pdf
 
Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24
 
Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.
 
Librarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineLibrarians Learning Online to Teach Online
Librarians Learning Online to Teach Online
 
Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013
 
OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.
 
DMPTool Webinar 3: Customizing the DMPTool
DMPTool Webinar 3: Customizing the DMPToolDMPTool Webinar 3: Customizing the DMPTool
DMPTool Webinar 3: Customizing the DMPTool
 
Research Data Management and Spatial Data
Research Data Management and Spatial DataResearch Data Management and Spatial Data
Research Data Management and Spatial Data
 
Assessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandAssessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finland
 
Charleston usa
Charleston usaCharleston usa
Charleston usa
 
Building Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataBuilding Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked Data
 
DMPTool Webinar 11: Complementary Tools
DMPTool Webinar 11: Complementary ToolsDMPTool Webinar 11: Complementary Tools
DMPTool Webinar 11: Complementary Tools
 

More from Sebastian Ruder

Frontiers of Natural Language Processing
Frontiers of Natural Language ProcessingFrontiers of Natural Language Processing
Frontiers of Natural Language ProcessingSebastian Ruder
 
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftStrong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftSebastian Ruder
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionSebastian Ruder
 
Neural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftNeural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftSebastian Ruder
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoHuman Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoSebastian Ruder
 
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiMachine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiSebastian Ruder
 
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimHashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimSebastian Ruder
 
Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Sebastian Ruder
 
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSpoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSebastian Ruder
 
Modeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverModeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverSebastian Ruder
 
Multi-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoMulti-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoSebastian Ruder
 
Funded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENFunded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENSebastian Ruder
 
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...Sebastian Ruder
 
Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Sebastian Ruder
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Sebastian Ruder
 
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Sebastian Ruder
 
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisA Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisSebastian Ruder
 
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Sebastian Ruder
 

More from Sebastian Ruder (19)

Frontiers of Natural Language Processing
Frontiers of Natural Language ProcessingFrontiers of Natural Language Processing
Frontiers of Natural Language Processing
 
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftStrong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary Induction
 
Neural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftNeural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain Shift
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep Learning
 
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoHuman Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
 
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiMachine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
 
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimHashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
 
Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...
 
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSpoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
 
Modeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverModeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John Glover
 
Multi-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoMulti-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer Calixto
 
Funded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENFunded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIEN
 
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
 
Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
 
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
 
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisA Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
 
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
 

Recently uploaded

Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPirithiRaju
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRlizamodels9
 
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfBehavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfSELF-EXPLANATORY
 
Pests of jatropha_Bionomics_identification_Dr.UPR.pdf
Pests of jatropha_Bionomics_identification_Dr.UPR.pdfPests of jatropha_Bionomics_identification_Dr.UPR.pdf
Pests of jatropha_Bionomics_identification_Dr.UPR.pdfPirithiRaju
 
《Queensland毕业文凭-昆士兰大学毕业证成绩单》
《Queensland毕业文凭-昆士兰大学毕业证成绩单》《Queensland毕业文凭-昆士兰大学毕业证成绩单》
《Queensland毕业文凭-昆士兰大学毕业证成绩单》rnrncn29
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.PraveenaKalaiselvan1
 
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptxTHE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptxNandakishor Bhaurao Deshmukh
 
Davis plaque method.pptx recombinant DNA technology
Davis plaque method.pptx recombinant DNA technologyDavis plaque method.pptx recombinant DNA technology
Davis plaque method.pptx recombinant DNA technologycaarthichand2003
 
Pests of safflower_Binomics_Identification_Dr.UPR.pdf
Pests of safflower_Binomics_Identification_Dr.UPR.pdfPests of safflower_Binomics_Identification_Dr.UPR.pdf
Pests of safflower_Binomics_Identification_Dr.UPR.pdfPirithiRaju
 
Neurodevelopmental disorders according to the dsm 5 tr
Neurodevelopmental disorders according to the dsm 5 trNeurodevelopmental disorders according to the dsm 5 tr
Neurodevelopmental disorders according to the dsm 5 trssuser06f238
 
Speech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxSpeech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxpriyankatabhane
 
Pests of soyabean_Binomics_IdentificationDr.UPR.pdf
Pests of soyabean_Binomics_IdentificationDr.UPR.pdfPests of soyabean_Binomics_IdentificationDr.UPR.pdf
Pests of soyabean_Binomics_IdentificationDr.UPR.pdfPirithiRaju
 
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxMurugaveni B
 
Radiation physics in Dental Radiology...
Radiation physics in Dental Radiology...Radiation physics in Dental Radiology...
Radiation physics in Dental Radiology...navyadasi1992
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingNetHelix
 
FREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naFREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naJASISJULIANOELYNV
 
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptx
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptxLIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptx
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptxmalonesandreagweneth
 
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxGenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxBerniceCayabyab1
 
OECD bibliometric indicators: Selected highlights, April 2024
OECD bibliometric indicators: Selected highlights, April 2024OECD bibliometric indicators: Selected highlights, April 2024
OECD bibliometric indicators: Selected highlights, April 2024innovationoecd
 
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...D. B. S. College Kanpur
 

Recently uploaded (20)

Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdf
 
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRCall Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR
 
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfBehavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
 
Pests of jatropha_Bionomics_identification_Dr.UPR.pdf
Pests of jatropha_Bionomics_identification_Dr.UPR.pdfPests of jatropha_Bionomics_identification_Dr.UPR.pdf
Pests of jatropha_Bionomics_identification_Dr.UPR.pdf
 
《Queensland毕业文凭-昆士兰大学毕业证成绩单》
《Queensland毕业文凭-昆士兰大学毕业证成绩单》《Queensland毕业文凭-昆士兰大学毕业证成绩单》
《Queensland毕业文凭-昆士兰大学毕业证成绩单》
 
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.
 
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptxTHE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx
THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx
 
Davis plaque method.pptx recombinant DNA technology
Davis plaque method.pptx recombinant DNA technologyDavis plaque method.pptx recombinant DNA technology
Davis plaque method.pptx recombinant DNA technology
 
Pests of safflower_Binomics_Identification_Dr.UPR.pdf
Pests of safflower_Binomics_Identification_Dr.UPR.pdfPests of safflower_Binomics_Identification_Dr.UPR.pdf
Pests of safflower_Binomics_Identification_Dr.UPR.pdf
 
Neurodevelopmental disorders according to the dsm 5 tr
Neurodevelopmental disorders according to the dsm 5 trNeurodevelopmental disorders according to the dsm 5 tr
Neurodevelopmental disorders according to the dsm 5 tr
 
Speech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptxSpeech, hearing, noise, intelligibility.pptx
Speech, hearing, noise, intelligibility.pptx
 
Pests of soyabean_Binomics_IdentificationDr.UPR.pdf
Pests of soyabean_Binomics_IdentificationDr.UPR.pdfPests of soyabean_Binomics_IdentificationDr.UPR.pdf
Pests of soyabean_Binomics_IdentificationDr.UPR.pdf
 
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
 
Radiation physics in Dental Radiology...
Radiation physics in Dental Radiology...Radiation physics in Dental Radiology...
Radiation physics in Dental Radiology...
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
 
FREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by naFREE NURSING BUNDLE FOR NURSES.PDF by na
FREE NURSING BUNDLE FOR NURSES.PDF by na
 
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptx
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptxLIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptx
LIGHT-PHENOMENA-BY-CABUALDIONALDOPANOGANCADIENTE-CONDEZA (1).pptx
 
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptxGenBio2 - Lesson 1 - Introduction to Genetics.pptx
GenBio2 - Lesson 1 - Introduction to Genetics.pptx
 
OECD bibliometric indicators: Selected highlights, April 2024
OECD bibliometric indicators: Selected highlights, April 2024OECD bibliometric indicators: Selected highlights, April 2024
OECD bibliometric indicators: Selected highlights, April 2024
 
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
Fertilization: Sperm and the egg—collectively called the gametes—fuse togethe...
 

NIPS 2016 Highlights - Sebastian Ruder

  • 1. Sebastian Ruder
 PhD Candidate, Insight Centre
 Research Scientist, AYLIEN @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup NIPS 2016 Highlights
  • 2. Agenda 1. NIPS overview 2. Generative Adversarial Networks 3. Building applications with Deep Learning 4. RNNs 5. Improving classic algorithms 6. Reinforcement Learning 7. Learning-to-learn / Meta-learning 8. General AI 9. NLP 10. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 3. NIPS Overview @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: http://www.tml.cs.uni-tuebingen.de/team/luxburg/misc/nips2016/index.php
  • 4. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks
  • 5. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 6. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf
  • 7. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 8. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 • Mostly used in CV for image generation; discriminator can be used as feature extractor • Less work in NLP and other areas; promising directions at GAN workshop (1, 2, 3, 4) 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf 5https://arxiv.org/abs/1611.01144
  • 9. Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 10. Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 11. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
  • 12. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
 being rejected from NIPS 1996
 — perseverance pays off
  • 13. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
  • 14. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for- long-or-event-based-sequences.pdf
  • 15. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf
  • 16. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3
  • 17. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 •General DL improvements • Weight normalisation4 • Multinomial dropout5 •Edward Grefenstette’s talk at NAMPI workshop6 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf 4https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural- networks.pdf 5https://papers.nips.cc/paper/6561-improved-dropout-for-shallow-and-deep-learning.pdf 6https://uclmr.github.io/nampi/talk_slides/grefenstette_limitations_of_rnns.pdf
  • 18. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
  • 19. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
  • 20. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity3 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf 3https://papers.nips.cc/paper/6449-clustering-with-same-cluster-queries.pdf
  • 21. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
  • 22. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 23. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN • New research environments: • OpenAI’s Universe • Deep Mind Lab • FAIR’s Torchcraft 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
  • 24. Learning-to-learn / Meta-learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Several papers, e.g.
 Learning to learn by gradient descent by gradient descent1 • Meta-learning panel at RNN symposium • Sutskever: No good meta-learning models so far • Schmidhuber: Focus on universal model • Focus at Neural Abstract Machines workshop 1https://papers.nips.cc/paper/6461-learning-to-learn-by-gradient-descent-by-gradient-descent.pdf
  • 25. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
  • 26. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
  • 27. General AI @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 28. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
  • 29. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
  • 30. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: Drew Purves
  • 31. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
  • 32. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
  • 33. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain • Machine Intelligence workshop: • FAIR’s CommAI-env • Hierarchical representations, identifying patterns • Panel: Unclear if focus on specific tasks brings us closer to General AI.
  • 34. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
  • 35. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 36. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 37. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
  • 38. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 39. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 40. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 • Dialogue modelling workshop: • End-to-end systems, linguistics, and ML methods 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
  • 41. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere!
  • 42. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics
  • 43. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing
  • 44. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence
  • 45. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence • RocketAI “launch” party
  • 46. @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Thanks for your attention! Questions?