NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutional Neural Networks for Android Devices - ITCE 2019

1. NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutional Neural Networks for Android Devices Menoufia University Faculty of Computers and Information Information Technology Department By Ahmed Fawzy Gad International Conference on Innovative Trends in Computer Engineering (ITCE’2019)

2. Index 2 • Introduction • Related Work • Proposed Library (NumPyCNNAndroid) • Advantages • Development Process • Supported Layers • Example • Validation • Find NumPyCNNAndroid at Google Play • References

4. Introduction 4 Machine Learning Algorithm Data Trained Model Feature Engineering Data Scientist Data Analysis Features

5. Introduction ▰ Working with huge amounts of data makes it challenging to decide what are the suitable features for use. ▰ A dataset such as ImageNet contains millions of images distributed across thousands of classes. 5 Convolutional Neural Network (CNN) ▰ The problem with the deep learning architectures is being time consuming and require high processing power.

6. Introduction ▰ There are some already existing libraries for building deep learning models such as TensorFlow, Keras, Theano, Caffe, and others. ▰ These libraries are normally available for desktop computers due to the availability of both memory and processing power. ▰ Working with mobile devices, there are 2 important issues. 6 Availability Efficiency ▰ This paper solves the availability issue by proposing a library called NumPyCNNAndroid (NPCA) for building CNNs for Android devices.

8. Related Work ▰ A network architecture called MobileNets proposes the use of depthwise separable convolution. ▰ In regular convolution, the 2 steps (filtering and combining) are done in a single step. These steps are split in the dephwise separable convolution. The first step is applying a pointwise convolution with filter size equal to 1x1. Then another pointwise convolution for combining the results. 8Howard, Andrew G., et al. "Mobilenets: Efficient convolutional neural networks for mobile vision applications." arXiv preprint arXiv:1704.04861 (2017). ▰ MobileNets requires the creation of a new network architecture for every application in order to be supported by the mobile devices.

9. Related Work ▰ A GPU-accelerated library called CNNdroid. 9Latifi Oskouei, Seyyed Salar, et al. "CNNdroid: GPU-accelerated execution of trained deep convolutional neural networks on android." Proceedings of the 2016 ACM Conference on Multimedia. ACM, 2016.

10. Related Work ▰ Some of the exisiting libraries have some releases for supporting the mobile devices. For example, there is a library called TensorFlow Lite from TensorFlow. Also there is Caffe mobile from Caffe. ▰ Similar to CNNdroid, there must be an in-between step for converting the original model into a format supported by the mobile devices. 10

11. Related Work ▰ Here is the summary of the model conversion in TensorFlow. 11 Prepare the TF Model. Convert the TF Model to TF Lite Model Create an Android Project Import the TF Lite Model within the Project Call the Model within Java Code ▰ In TF Lite, the developer uses multiple tools (Python, TOCO, Android Studio, and Java)

13. Proposed Work Advantages ▰ NPCA uses Python for its development which is one of the best tools in data science. ▰ The same code, unchanged, is executed in all platforms due to the use of a cross-platform library called Kivy. ▰ Faster deployment for the models of the mobile devices as there is no in-between step to customize the model. 13

15. Proposed Work Development Process ▰ The CNN can be trained on the mobile device but it is time-consuming at least in the current version. ▰ The preferred way is to train the CNN in a desktop computer and then use it on the mobile device. 15

17. Proposed Work Supported Layers ▰ Convolution (conv) ▰ Rectified Linear Unit (relu) ▰ Max Pooling (maxpool) ▰ Average Pooling (avgpool) ▰ Fully Connected (fc) 17 GitHub – NumPyCNN https://github.com/ahmedfgad/NumPyCNN GitHub – NumPyCNNAndroid https://github.com/ahmedfgad/NumPyCNNAndroid

19. Proposed Work Example 19 Read Image - PIL

21. Validation 21

23. Find NumPyCNNAndroid at Google Play ▰ To facilitate testing the library, a free application named NumPyCNNAndroid is available at Google Play for download. ▰ https://play.google.com/store/ apps/details?id=org.numpycnn. numpycnn 23

24. Find NumPyCNNAndroid at Google Play 24

26. References ▰ Nanni, Loris, Stefano Ghidoni, and Sheryl Brahnam. "Handcrafted vs. non-handcrafted features for computer vision classification." Pattern Recognition 71 (2017): 158-172. ▰ Russakovsky, Olga, et al. "Imagenet large scale visual recognition challenge." International Journal of Computer Vision 115.3 (2015): 211-252. ▰ Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic-Sign Detection and Classification. Springer, 2017. ▰ Schmidhuber, Jürgen. "Deep learning in neural networks: An overview." Neural networks 61 (2015): 85-117. ▰ Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. ▰ Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014). ▰ Szegedy, Christian, et al. "Going deeper with convolutions." Cvpr, 2015. ▰ Cai, Ermao, et al. "Neuralpower: Predict and deploy energy-efficient convolutional neural networks." arXiv preprint arXiv:1710.05420 (2017). ▰ Kim, Yong-Deok, et al. "Compression of deep convolutional neural networks for fast and low power mobile applications." arXiv preprint arXiv:1511.06530 (2015). ▰ Latifi Oskouei, Seyyed Salar, et al. "CNNdroid: GPU-accelerated execution of trained deep convolutional neural networks on android." Proceedings of the 2016 ACM Conference on Multimedia. ACM, 2016. https://github.com/ENCP/CNNdroid 26

27. References ▰ Abadi, Martín, et al. "TensorFlow: A System for Large-Scale Machine Learning." OSDI. Vol. 16. 2016. ▰ Jia, Yangqing, et al. "Caffe: Convolutional architecture for fast feature embedding." Proceedings of the 22nd ACM international conference on Multimedia. ACM, 2014. ▰ Al-Rfou, Rami, et al. "Theano: A Python framework for fast computation of mathematical expressions." arXiv preprint arXiv:1605.02688 472 (2016): 473. ▰ Nicholson, A. Chris, and Adam Gibson. "Deeplearning4j: Open-source, Distributed Deep Learning for the JVM." https://Deeplearning4j.org (2017). ▰ Chen, Tianqi, et al. "MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems." arXiv preprint arXiv:1512.01274 (2015). ▰ Chollet, François. "Keras: Deep learning library for Theano and TensorFlow." URL: https://keras.io/ (2015): 8. ▰ Torch: http://torch.ch/ ▰ Wu, Jiaxiang, et al. "Quantized convolutional neural networks for mobile devices." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. ▰ Han, Song, Huizi Mao, and William J. Dally. "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding." arXiv preprint arXiv:1510.00149 (2015). ▰ Chen, Wenlin, et al. "Compressing neural networks with the hashing trick." International Conference on Machine Learning. 2015. ▰ Howard, Andrew G., et al. "Mobilenets: Efficient convolutional neural networks for mobile vision applications." arXiv preprint arXiv:1704.04861 (2017). ▰ Zhang, Xiangyu, et al. "Shufflenet: An extremely efficient convolutional neural network for mobile devices." arXiv preprint arXiv:1707.01083 (2017). 27

28. References 28 ▰ TensorFlow Lite: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite. ▰ Caffe Android: https://github.com/sh1r0/caffe-android-lib. ▰ Torch Android: https://github.com/soumith/torch-android. ▰ python-for-android: https://github.com/kivy/python-for-android ▰ Virbel, Mathieu, Thomas E. Hansen, and Oleksandr Lobunets. "Kivy-A Framework for Rapid Creation of Innovative User Interfaces." Mensch & Computer Workshopband. 2011. ▰ Walt, Stéfan van der, S. Chris Colbert, and Gael Varoquaux. "The NumPy array: a structure for efficient numerical computation." Computing in Science & Engineering 13.2 (2011): 22-30. ▰ Python Imaging Library (PI) http://www.pythonware.com/products/pil ▰ Buildozer: https://github.com/kivy/buildozer ▰ NumPyCNN (NPC): https://github.com/ahmedfgad/NumPyCNN ▰ NumPyCNNAndroid (NPCA): https://github.com/ahmedfgad/NumPyCNNAndroid ▰ NumPyCNNAndroid (NPCA): https://play.google.com/store/apps/details?id=org.numpycnn.numpycnn

29. 29 THANKS