This document discusses multitask learning in neural networks. It explains that multitask learning can help neural networks learn better by solving multiple tasks using a single model, allowing more knowledge to be contained within the model without additional inputs. Multitask learning also acts as a form of regularization that can improve generalization. The document reviews several examples and use cases of multitask learning, including computer vision tasks, natural language processing, and signal processing. It identifies some benefits of multitask learning as regularization, representation bias, feature selection, and transfer learning.

1. Multitask learninglearn more to learn better
Oleksandr Honchar
Mawi solutions

2. • AI @ Mawi Solutions
• Masters in maths candidate @
University of Verona
• Blogging @ Medium
• Calling myself an “AI expert” @
LinkedIn

3. • multitasking for humans
• multitasking for neural nets
• research use cases
• auxiliary losses
• takeaways and homework

4. humans and multitasking

5. https://bitesizebio.com/27766/multitasking-lab-not-multitasking/

6. multitasking and neural nets

7. http://colah.github.io/posts/2015-01-Visualizing-Representations/

8. Main optimization criteria
Tikhonov,
smoothing an ill-
posed problem
Zaremba, model
complexity
minimization
Bayes: priors over
parameters
Andrew Ng: need no
maths, but it prevents
overfitting!

9. Multitask Learning, Rich Caruana
Main optimization criteria (y, w, x) +α * L1(w) + β* L2(w) -γ * Entropy(y, w)

10. Multitask Learning, Rich Caruana
Main optimization criteria (y, w, x) +α * L1(w) + β* L2(w) -γ * Entropy(y, w)

11. Multitask Learning, Rich Caruana
Loss(y2, w, x) Loss(y3, w, x) Loss(y4, w, x)Loss1 (y1, w, x)

12. http://ruder.io/multi-task/
Motivation:
1. Solving a lot of tasks with a single model
2. More knowledge “inside” of a model
without additional inputs
3. Regularization and generalization as a
bonus
Loss(y2, w, x) Loss(y3, w, x)Loss1 (y1, w, x)

13. someone really does it?

14. http://taskonomy.stanford.edu/

15. http://decanlp.com/

16. Signal quality classification Rhythm anomaly detection
Morphological tagging

17. why can it work at all?

18. 1.Regularization
Multitask Learning, Rich Caruana | http://ruder.io/multi-task/
https://www.cs.umd.edu/~tomg/projects/landscapes/

19. 1.Regularization
2.Representation bias
Multitask Learning, Rich Caruana | http://ruder.io/multi-task/

20. 1.Regularization
2.Representation bias
3.Feature selection double check
Multitask Learning, Rich Caruana | http://ruder.io/multi-task/

21. 1.Regularization
2.Representation bias
3.Feature selection double check
4.“Transfer learning”
Multitask Learning, Rich Caruana | http://ruder.io/multi-task/

22. 1.Regularization
2.Representation bias
3.Multiple metrics
4.“Transfer learning”
5.Outputs that can’t be inputs
Multitask Learning, Rich Caruana | http://ruder.io/multi-task/
PredNet

23. auxiliary tasks

25. NLP
1. Language modeling
2. Relations, NER, linguistics…
3. DecaNLP
Computer vision
1. Taskonomy
2. OpenCV
Signal processing
1. Statistics / features modeling
2. Predictive modeling
3. Denoising
In general
1. Autoencoders / Generative
models
2. Hints (predict important feature)

26. takeaways

28. 1.Multitask learning is natural in terms of human-like learning
2.Multitask learning is natural in terms of “normal” machine
learning
3.Maybe you don’t need more data, but you need more losses?

29. What to google:
1. Taskonomy (CVPR 18)
2. DecaNLP (SalesForce)
3. Distral (DeepMind)
4. http://ruder.io/multi-task/
5. Attend my workshop today :)
FB: @rachnogstyleblog
MEDIUM: @alexrachnog