Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ai - A Practical Approach


Published on

Exploring Artificial Intelligence and in practice, with several network architectures implemented on the fly in a live coding session. In this talk we went through Shallow / Intermediate / Deep / Convolutional and Residual Networks. Showing the differences in hyperparameters tuning, which to use and when.

In addition, also how they came to life though the mind of computer scientists like Geoffrey Hinton, Yoshua Bengio, Xavier Glorot and Adam Coates.

The code presented in this talk can be found here:

Published in: Technology
  • Login to see the comments

Ai - A Practical Approach

  1. 1. Artificial Intelligence A Practical Approach
  2. 2. Who am I? Software & Machine Learning Engineer; City.AI Ambassador; IBM Watson AI XPRIZE contestant; Kaggler; Guest attendee at AI for Good Global Summit at the UN; X-Men geek; family man and father of 5 (3 kids and 2 cats). @wilderrodrigues
  3. 3. Machine Learning without a Ph.D.
  4. 4. What is it About? Source: World Wide Web Foundation, June 2017
  5. 5. Points to Focus On Number and Types of Layers; Initialisation; Regularisation; Cost Functions; Optimisation; Data Augmentation.
  6. 6. Shallow Neural Networks Random Initialisation Mean Squared Error Stochastic Gradient Descent Sigmoid
  7. 7. Initialisation and Loss Function Zero initialisation ☹ W1 = np.zeros((layers_dims[l], layers_dims[l - 1])) Random initialisation 😐 W1 = np.random.randn(layers_di ms[l], layers_dims[l - 1]) * 10 Mean Squared Error ☹ 1/m.sum(Y_hat, Y)**2
  8. 8. Xavier Glorot L2 regularisation Cross Entropy Adam Tanh Intermediate Neural Networks
  9. 9. Initialisation and Loss Function Xavier Glorot 😄 W1 = np.random.randn(layers[l], layers[l - 1]) * np.sqrt(2 / (layers[l - 1] + layers[l])) L2 regularisation ☺ (ƛ/2.m).sum(W**2) Adam Exponentially Weighted Averages Vt = βVt-1 + (1 - β)𝛳t RMSProp Cross Entropy 😄 1/m.sum(Y.log(a)+(1-Y).log(1-a))
  10. 10. Deep Neural Networks He Cross Entropy Dropout Adam ReLU
  11. 11. Initialisation and Cost Function He 😄 W1 = np.random.randn(layers[l], layers[l - 1]) * np.sqrt(2 / layers[l - 1]) Dropout 😄 ❌ ✅
  12. 12. Convolutional Networks Xavier Glorot Layers: Convolutional Layers Max Pooling Layers Fully Connected Layers Cross Entropy Dropout Adam ReLU
  13. 13. Convolutional Networks * = 6x6x3 3x3x16 4x4x16 4x4x16 2x2x16 2x2x16 * =
  14. 14. Residual Networks Xavier Glorot Layers: Convolutional Layers Bottleneck Layers Max Pooling Layers Fully Connected Layers Cross Entropy Dropout Adam ReLU
  15. 15. Residual Networks using Inception 5x5 Same 32 28x28x192 28x28x32 28x28x192 * 5x5x32 = 120m 1x1 16 1x1x192 28x28x192 28x28x16 5x5 Same 32 28x28x32 28x28x16 * 192 = 2.4m 28x28x32 * 5x5x16 = 10m
  16. 16. Capsule Networks The is no pose (translational and rotational) relationship between simpler features. Successive convolutional or max pooling layers to reduce spacial size.
  17. 17. Capsule Networks
  18. 18. Resources and References Machine Learning: Andrew Ng, Stanford University, Coursera. Neural Networks for Machine Learning: Geoffrey Hinton, University of Toronto, Coursera. Computational Neuroscience: Rajesh Rao & Adrienne Fairhall, Washington University, Coursera. Neural Networks and Deep Learning: Andrew Ng,, Coursera. Structuring Machine Learning Projects: Andrew Ng,, Coursera. Improving Deep Neural Networks with Hyperparameter Tuning, Regularisation and Optimisation: Andrew Ng,, Coursera. Convolutional Neural Networks: Andrew Ng,, Coursera. Calculus I: Jim Fowler, Ohio State University, Coursera.
  19. 19. Thank you!