Interest in Deep Learning has been growing in the past few years. With advances in software and hardware technologies, Neural Networks are making a resurgence. With interest in AI based applications growing, and companies like IBM, Google, Microsoft, NVidia investing heavily in computing and software applications, it is time to understand Deep Learning better! In this lecture, we will discuss the basics of Neural Networks and discuss how Deep Learning Neural networks are different from conventional Neural Network architectures. We will review a bit of mathematics that goes into building neural networks and understand the role of GPUs in Deep Learning. We will also get an introduction to Autoencoders, Convolutional Neural Networks, Recurrent Neural Networks and understand the state-of-the-art in hardware and software architectures. Functional Demos will be presented in Keras, a popular Python package with a backend in Theano. This will be a preview of the QuantUniversity Deep Learning Workshop that will be offered in 2017.

1. Location:
QuantUniversity Meetup
December 21st 2016
Boston MA
Deep Learning : An introduction
2016 Copyright QuantUniversity LLC.
Presented By:
Sri Krishnamurthy, CFA, CAP
www.QuantUniversity.com
sri@quantuniversity.com

2. 2
Slides and Code will be available at:
http://www.analyticscertificate.com

3. - Analytics Advisory services
- Custom training programs
- Architecture assessments, advice and audits

4. • Founder of QuantUniversity LLC. and
www.analyticscertificate.com
• Advisory and Consultancy for Financial Analytics
• Prior Experience at MathWorks, Citigroup and
Endeca and 25+ financial services and energy
customers.
• Regular Columnist for the Wilmott Magazine
• Author of forthcoming book
“Financial Modeling: A case study approach”
published by Wiley
• Charted Financial Analyst and Certified Analytics
Professional
• Teaches Analytics in the Babson College MBA
program and at Northeastern University, Boston
Sri Krishnamurthy
Founder and CEO
4

5. 5
Quantitative Analytics and Big Data Analytics Onboarding
• Trained more than 500 students in
Quantitative methods, Data Science
and Big Data Technologies using
MATLAB, Python and R
• Launching the Analytics Certificate
Program in September

6. 6
• January 2017
▫ 19th, Deep Learning Lecture Part II
• February 2017
▫ Deep Learning Workshop (Date TBD)
Events of Interest

7. 7
Dr. Victor Shnayder
Fellow
QuantUniversity
Prior Experience:
Product Manager EdX (March 2013-June 2016)
Harvard University
PhD, Computer Science

8. 8
Start with labeled pairs (Xi, Yi)
( ,“kitten”),( ,“puppy”)
…

9. 9
Success: predict new examples
( ,?)

10. 10
https://commons.wikimedia.org/wiki/Neural_network
“kitten”
“puppy”
“has fur?”
“pointy ears?”
“dangerously cute?”

11. 11

12. 12
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double

13. 13
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Weighted sum

14. 14
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Non-linear “activation” function

15. 15
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learning = “find good weights”

16. 16
http://stackoverflow.com/questions/40537503/deep-neural-networks-precision-for-image-recognition-float-or-double
Learning = “find good weights”
How? Gradient descent!

17. 18
1. Our labeled datasets were thousands of times too small.
2. Our computers were millions of times too slow.
3. We initialized the weights in a stupid way.
4. We used the wrong type of non-linearity.
- Geoff Hinton
Neural nets were tried in the 1980s. What changed?
https://youtu.be/IcOMKXAw5VA?t=21m29s

18. 19
http://www.rsipvision.com/exploring-deep-learning/

19. 20
http://www.asimovinstitute.org/neural-network-zoo/

20. 21

21. 22
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html

22. 23
https://research.googleblog.com/2014/09/building-deeper-understanding-of-images.html

23. 24
Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary
Nodules in CT Scans http://www.nature.com/articles/srep24454/figures/1

24. 25
Towards End-to-End Speech Recognition with Recurrent Neural Networks
http://www.jmlr.org/proceedings/papers/v32/graves14.pdf

25. 26
https://www.technologyreview.com/s/544651/baidus-deep-learning-system-rivals-people-at-speech-recognition/

26. 27
https://research.googleblog.com/2014/11/a-picture-is-worth-thousand-coherent.html

27. 28
http://cs.umd.edu/~miyyer/data/deepqa.pdf

28. 29

29. 30
http://blog.ventureradar.com/2016/03/11/10-hot-startups-using-artificial-intelligence-in-cyber-security/

30. 31
https://www.youtube.com/watch?v=H4V6NZLNu-c

31. 32
https://www.engadget.com/2016/03/12/watch-alphago-vs-lee-sedol-round-3-live-right-now/

32. 33
https://www.youtube.com/watch?v=kMMbW96nMW8

33. 34

34. 35
How is deep learning special?
Given (lots of) data, DNNs learn useful input
representations.
D. Erhan et al. ‘09
http://www.iro.umontreal.ca/~lisa/publications2/index.php/publications/show/247

35. 36

36. 37
Hardware

37. 38
Data
http://www.theneweconomy.com/strategy/big-data-is-not-without-its-problems

38. 39
New Approaches
http://deeplearning.net/reading-list/

39. 40

40. 41
• Theano is a Python library that allows you to define, optimize, and
evaluate mathematical expressions involving multi-dimensional
arrays efficiently
• Performs efficient symbolic differentiation
• Leverages NVIDIA GPU (Claim 140X faster than CPU)
• Developed by University of Montreal researchers and is open-source
• Works on Windows/Linux/Mac OS
• See https://arxiv.org/abs/1605.02688
Theano

41. 42
• GPU vs CPU
▫ Theano Test
▫ See Theano Test.ipyb
Demo

42. 43
• Logistic Regression
Theano
See Theano-Logistic Regression.ipyb

43. 44
MLP

44. 45
Convolutional Neural Networks
Convolution

45. 46
Convolutional Neural Networks
Sparse connectivity
Weight sharing-Max-pooling layer
See Theano-Conv-Net.ipynb

46. 47
• Keras is a high-level neural networks library, written in Python and
capable of running on top of either TensorFlow or Theano. It was
developed with a focus on enabling fast experimentation.
• Allows for easy and fast prototyping (through total modularity,
minimalism, and extensibility).
• Supports both convolutional networks and recurrent networks, as
well as combinations of the two.
• Supports arbitrary connectivity schemes (including multi-input and
multi-output training).
• Runs seamlessly on CPU and GPU.
Keras

47. 48
• Keras Examples
▫ Testing Keras: See KerasPython.ipynb
▫ Running Convolutional NN on Keras with a Theano Backend
 See Keras-conv-example-mnist.ipynb
Demo

48. 49
• A case study for Convolutional Neural Networks
• Recurrent Neural Networks
• Auto Encoders
• Best Practices
Coming on January 21st - Part II

49. 50
Q&A

50. Thank you!
Members & Sponsors!
Sri Krishnamurthy, CFA, CAP
Founder and CEO
QuantUniversity LLC.
srikrishnamurthy
www.QuantUniversity.com
Contact
Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be
distributed or used in any other publication without the prior written consent of QuantUniversity LLC.
51