instruction of install Caffe step by step on ubuntu 14.04
برای دریافت دی وی دی های این کارگاه (شامل فیلم کارگاه و فایل های مورد نیاز) به آدرس زیر ایمیل بزنید:
pouya.ahmadvand@gmail.com
2. Deep Learning in the Real World!
#Deep Learning has become the most popular approach for developing
Artificial Intelligence (AI) – machines that perceive and understand
the world.
#The focus is currently on specific perceptual tasks, and there are
many successes.
#Today, some of the world`s largest internet companies, as well as
foremost research institutions (e.g. Google, Facebook, Microsoft ,
etc.) are using deep learning in research and production.
2/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
3. Practical Deep Learning Examples
3/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
4. Deep Learning Advantages
# Robust :
• No need to design the features ahead of time – features are
automatically learned to be optimal for the task at hand
• Robustness to natural variations in the data is automatically learned
# Generalizable :
• The same neural net approach can be used for many different
applications and data types
# Scalable :
• Performance improves with more data, method is massively
parallelizable
4/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
6. Training CNNs (CPUs VS GPUs)
GPU Speed
Up
Training
Time GPU
Training
Time CPU
Batch Size
8.5X7.5 s64 s64 Images
8.5X14.5 s128 s128 Images
9.0X28.5 s257 s256 Images
AlexNet (5 Coevolution Layers, 2 Fully-connected)
Implemented with Caffe
Training time is for 20 iterations
CPU: Dual 10-core Ivy Bridge CPUs
GPU: 1 Tesla K40 GPU
6/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
7. How GPU Acceleration works
7/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
8. Deep Learning Frameworks
MatconvnetKaldiTorchCaffe
Deep Learning
Framework
Speech
Recognition
Toolkit
Scientific
Computing
Framework
Deep Learning
Framework
Domain
-CuDNN
-Multi-GPU
--Multi-CPU
MatlabPythonPython, Matlab
, Lua
Command line,
Python, Matlab
Interface(s)
8/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
9. Caffe Framework
#Caffe :
• Caffe is a deep learning framework made with expression, speed,
and modularity in mind. It is developed by the Berkeley Vision and
Learning Center (BVLC) and by community contributors. Yangqing
Jia created the project during his PhD at UC Berkeley. Caffe is
released under the BSD 2-Clause license.
http://caffe.berkeleyvision.org
9/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
10. Caffe Framework
# Why Caffe ?
# Expressive architecture: encourages application and innovation. Models and optimization are
defined by configuration without hard-coding. Switch between CPU and GPU by setting a single
flag to train on a GPU machine then deploy to commodity clusters or mobile devices.
# Extensible code: fosters active development. In Caffe’s first year, it has been forked by over 1,000
developers and had many significant changes contributed back. Thanks to these contributors the
framework tracks the state-of-the-art in both code and models.
# Speed: makes Caffe perfect for research experiments and industry deployment. Caffe can process
over 60M images per day with a single NVIDIA K40 GPU. That’s 1 ms/image for inference and 4
ms/image for learning. We believe that Caffe is the fastest convnet implementation available.
# Community: Caffe already powers academic research projects, startup prototypes, and even
large-scale industrial applications in vision, speech, and multimedia.
10/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http://caffe.berkeleyvision.org
11. Ubuntu Installation
#What you need :
• A flash with at least 8GB space
• At least 40GB unallocated space at the end of your hard drive
#What you must to do :
• Make the flash bootable by using Rufus software
• Reboot your system and boot from the bootabled flash
11/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
14. Update Ubuntu
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install build-essential
sudo apt-get install linux-source
sudo apt-get install linux-headers-generic
sudo apt-get dist-upgrade
sudo apt-get upgrade
sudo reboot
# Open a Terminal and type these commands :
14/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
15. CUDA
# CUDA is a parallel computing platform and application programming
interface (API) model created by NVIDIA. It allows software developers to use a
CUDA-enabled graphics processing unit (GPU) for general purpose processing –
an approach known as GPGPU. The CUDA platform is a software layer that gives
direct access to the GPU's virtual instruction set and parallel computational
elements.
# The CUDA platform is designed to work with programming languages such
as C, C++ and Fortran. When it was first introduced by NVIDIA, the name CUDA
was an acronym for Compute Unified Device Architecture, but NVIDIA
subsequently dropped the use of the acronym.
15/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.wikipedia.com
16. CUDA Installation
# You Need A NVIDIA GPU with CUDA Capability
# You can check your GPU CUDA Capability from this link :
• https://en.wikipedia.org/wiki/CUDA
# Open a Terminal and type these commands :
sudo add-apt-repository -r ppa:bumblebee/stable
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
chmod +x cuda_7.5.18_linux.run
16/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
17. CUDA Installation
# Press Ctrl + Alt + F2 for enter to tty2
# Stop lightdm service by this command
# Run cuda_7.5.18_linux.run by this command :
# Follow install instructions
# Update graphic driver
sudo service lightdm stop
./cuda_7.5.18_linux.run
sudo apt-get install nvidia-361 nvidia-prime -y
sudo apt-get install freeglut3 freeglut3-devsudo nvidia-modprobe -y
sudo reboot
17/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
18. CUDA Installation
# Add this lines to .bashrc file (~/.bashrc)
# Create “cuda.conf” file in /etc/ld.so.conf.d/
# Copy this lines to the file that you just created
# And finally run this command
export CUDA_HOME=/usr/local/cuda-7.5
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${CUDA_HOME}/lib
PATH=${CUDA_HOME}/bin:${PATH}
export PATH
export CUDA_ROOT=${CUDA_HOME}/bin
/usr/local/cuda-7.5/lib
/usr/local/cuda-7.5/lib64
sudo ldconfig
18/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
20. OpenCV
# Open-source Computer Vison Library
# It’s an open source library written in C++ for computer vision.
# It was originally designed by Intel (1991).
# 2,500+ algorithms and functions
# Corss-platform, portable API
# Real-time performance
# BSD Licensed (free and open source)
# Some of users
20/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.opencv.org
25. OpenCV Installation
OpenCVver="opencv-3.1.0“
cd ~/Develop/$OpenCVver/release
make -j $(nproc)
sudo make install -j $(nproc)
sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf‘
sudo sh -c 'echo "/usr/local/lib">/etc/ld.so.conf.d/opencv.conf‘
sudo ldconfig
sudo ln -s ~/Develop//$OpenCVver/release/lib/cv2.so /usr/lib/python2.7/dist-
packages/cv2.so
# Finally, run these commands :
25/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
26. Matlab Installation
mkdir matlab
unzip matlab_2014a.zip -d ./matlab/
sudo chmod 777 -R matlab
cd matlab
sudo ./install -javadir /usr/lib/jvm/java-7-openjdk-amd64/jre
cd ..
sudo cp ./libmwservices.so /usr/local/MATLAB/R2014a/bin/glnxa64/
sudo rm -dr matlab
sudo apt-get install matlab-support
# Go to the Matlab folder and type these commands :
26/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
27. CuDNN
# The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of
primitives for deep neural networks.
# Deep learning developers and researchers worldwide rely on the highly optimized
routines in cuDNN which allow them to focus on designing and training neural network
models rather than spending time on low-level performance tuning.
CPU is 16 core Haswell E5-2698 at 2.3 GHz, with 3.6 GHz Turbo
GPU is NVIDIA Titan X
27/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
http//:www.nvidia.com
28. CuDNN Instalation
tar -zxf cudnn-7.0-linux-x64-v4.0-rc.tgz -C ~/Developcd ~/Develop/cuda
sudo cp ~/Develop/cuda/lib64/* /usr/local/cuda/lib64/
sudo cp ~/Develop/cuda/include/cudnn.h /usr/local/cuda/include/
# Go to CuDNN folder and type these commands :
28/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
29. OpenBLAS
# OpenBLAS is an open source implementation of the BLAS (Basic Linear Algebra
Subprograms) API with many hand-crafted optimizations for specific processor types. It is
developed at the Lab of Parallel Software and Computational Science, ISCAS.
# OpenBLAS adds optimized implementations of linear algebra kernels for several
processor architectures. It claims to achieve performance comparable to the Intel MKL.
29/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
https://github.com/xianyi/OpenBLAS
30. OpenBLAS Installation
cd ~/Develop
git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS
make -j $(nproc)
make PREFIX=~/Develop/OpenBLAS/ install
sudo ln -s ~/Develop/OpenBLAS/libopenblas.so /usr/lib/libopenblas.so
sudo ln -s ~/Develop/OpenBLAS/libopenblas.so.0 /usr/lib/libopenblas.so.0
# Go to OpenBLAS folder and type these commands :
30/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
31. Caffe Dependencies
sudo apt-get install build-essential autoconf libtool pkg-config python-opengl python-imaging
python-pyrex python-pyside.qtopengl idle-python2.7 qt4-dev-tools qt4-designer libqtgui4
libqtcore4 libqt4-xml libqt4-test libqt4-script libqt4-network libqt4-dbus python-qt4 python-
qt4-gl libgle3 python-dev libatlas-base-dev libprotobuf-dev libleveldb-dev libsnappy-dev
libopencv-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-
compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install python-pip
sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook
python-pandas python-sympy python-nose python-flask
sudo pip install setuptools –upgrade
sudo easy_install green let
sudo easy_install gevent
# To install Caffe dependencies open an Terminal and type these commands :
31/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
32. Caffe Installation
cd ~/Develop
sudo apt-get install git
git clone https://github.com/BVLC/caffe.git
cd caffe
cd python
for req in $(cat requirements.txt); do sudo pip install $req; done
export PYTHONPATH=~/Develop/caffe/python:$PYTHONPATH
export caffe_root=~/Develop/caffe/$caffe_root
echo "export PYTHONPATH=~/Develop/caffe/python:$PYTHONPATHexport
caffe_root=~/Develop/caffe/$caffe_root" >> ~/.bashrc
# To download Caffe framework open a Terminal and type these commands :
32/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
33. Caffe Installation
5) # USE_CUDNN := 1 -> USE_CUDNN := 1
21) # OPENCV_VERSION := 3 -> OPENCV_VERSION := 3
46) BLAS := atlas -> BLAS := open
50) # BLAS_INCLUDE := /path/to/your/blas -> BLAS_INCLUDE := ~/Develop/OpenBLAS/include
51) # BLAS_LIB := /path/to/your/blas -> BLAS_LIB := ~/Develop/OpenBLAS/lib
59) # MATLAB_DIR := /usr/local -> MATLAB_DIR := /usr/local/MATLAB/R2014a
# Rename Makefile.config.example to Makefile.config and change this lines :
# Complie Caffe and download AlexNet :
cd ~/Develop/caffe
make everything -j $(nproc)
./scripts/download_model_binary.py models/bvlc_reference_caffenet
./data/ilsvrc12/get_ilsvrc_aux.sh
33/43
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
36. Step :
Collect your Data
36/43
Iranshahr Dataset# Collect your Dataset and create training and
validation files :
• train.txt and val.txt
# Labels file :
Filepath Classlable
Class_1_sample_1.jpg 0
Class_1_sample_2.jpg 0
Class_2_sample_1.jpg 1
Class_2_sample_2.jpg 1
LableName
Rasht
Zahedan
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
1
37. Step :
Create LMDB Database
2
# Lightning Memory-Mapped Database (LMDB) is a software library that provides a high-
performance embedded transactional database in the form of a key-value store.
# A script for creating LMDB database is located at : CAFFE_ROOT/examples/imagenet
# Set these variables and run the script (create_imagenet.sh) :
# After running this script, two folder will be created in the EXAMPLE path
# You can make your data mean by running make_imagenet_mean.sh
37/43
EXAMPLE= export path
DATA= where the train.txt and val.txt located
TOOLS= Caffe tools folder path
TRAIN_DATA_ROOT= where the train data located
VAL_DATA_ROOT= where the val data located
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
38. Step :
Define/Modify Network
# Define/Modify your model by using
Protobuf model format
• Strongly typed format
• Human readable
• Auto-generates and checks Caffe code
• Developed by Google
• Used to define network architecture
and training parameters
• No coding required!
38/43
name: "CaffeNet“
.
.
layer {
name: "pool1“
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
.
.
Pouya Ahmadvand, Deep Learning Workshop ,Shahid Rajaei Teacher Training University, 5 March 2016
3