SlideShare a Scribd company logo
1 of 35
Download to read offline
Caffe Framework Tutorial2
Layer, Net, Test
Index
• Layer
– Data
– ImageData
– Convolution
– Pooling
– ReLU
– InnerProduct
– LRN
• Net
– Mnist
– CIFAR-10
– ImageNet (Ilsvrc12)
• Net change Test
– Mnist
– CIFAR-10
• 64x64x3 Image Folder
• 64x64x3 Image Resize To 32x32x3
Data Layer
layer {
name: "data"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mean_file:
"examples/cifar10/mean.binaryproto"
}
data_param {
source:
"examples/cifar10/ilsvrc12_train_lmdb"
batch_size: 100
backend: LMDB
}
}
• Required
– source: the name of the directory
containing the database
– batch_size: the number of inputs to
process at one time
• Optional
– backend [default LEVELDB]: choose
whether to use a LEVELDB or LMDB
LevelDB ,LMDB : efficient databases
• Example
– Data.mdb(41MB)
– Lock.mdb(8.2kB
ImageData Layer
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
transform_param {
mirror: false
crop_size: 227
mean_file:
"data/ilsvrc12/imagenet_mean.binaryproto"
}
image_data_param {
source: "examples/_temp/file_list.txt"
batch_size: 50
new_height: 256
new_width: 256
}
}
• Required
– source: name of a text file, with each
line giving an image filename and label
– batch_size: number of images to batch
together
Convolution Layer
layer {
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1“
param { lr_mult: 1 } # learning rate for the filters
param { lr_mult: 2 } # learning rate for the biases
convolution_param {
num_output: 32 # learn 32 filters
kernel_size: 5 # each filter is 5x5
pad: 2
stride: 1 # step 1 pixels between each filter application
weight_filler {
type: "gaussian“# initialize the filters from a Gaussian
std: 0.0001 # default mean: 0
}
# initialize the biases to zero (0)
bias_filler { type: "constant" }
}
}
• Required
– num_output (c_o): the number of
filters
– kernel_size (or kernel_h and kern
el_w): specifies height and width
of each filter
• Strongly Recommended
– weight_filler [default type:
'constant' value: 0]
• Optional
– pad [default 0]: specifies the
number of pixels to (implicitly)
add to each side of the input
– stride [default 1]: specifies the
intervals at which to apply the
filters to the input
Pooling Layer
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
• Required
– kernel_size : specifies height and width of
each filter
• Optional
– pool [default MAX]: the pooling method.
Currently MAX, AVE, or STOCHASTIC
– pad [default 0]: specifies the number of
pixels to (implicitly) add to each side of
the input
– stride [default 1]: specifies the intervals at
which to apply the filters to the input
• 예) stride 2 : step two pixels (in the
bottom blob) between pooling regions
ReLU Layer (Rectified-Linear and Leaky-ReLU)
Rectified 정류된, Leaky-ReLU 새는, 구멍이 난
layer {
name: "relu1"
type: "ReLU"
bottom: "conv1"
top: "conv1" }
• Parameters optional
• negative_slope [default 0]:
– specifies whether to leak the
negative part by multiplying it
with the slope value rather than
setting it to 0.
• Input x, Compute Output y
• y = x (if x > 0)
• y = x * negative_slope (if x <= 0)
Why ReLU, Drop-Out!
• Drop-Out
– 2014. Toronto. paper title
• Dropout : A Simple Way to Prevent Neural Networks from Overfitting
• The key idea is to randomly drop units (along with their connections) from the
neural network during training
– Regularizer 의 일종
– Hidden Node를 모두 훈련시키지 않고, Random하게 Drop Out시킨다
• 관련된 Weight들은 훈련되지 않는다.
Drop-Out
• … using ReLUs trained with dropout during frame level training provide an 4.2% relative
improvement over a DNN trained with sigmoid units…
Rectified-Linear unit(ReLU)
• Drop-Out 은 학습이 느리다.
– Drop Out된 weight들은 학습이 일어나지 않는다.
• Non-Linear Activation Function의 교체
– 일반적으로 사용되는 Logistic Sigmoid, tanh 대신 ReLu 사용
• ReLU의 장점
– reduced likelihood of the gradient to vanish
– Sparsity
CVPR 2014 Tutorial
InnerProduct Layer
layer {
name: "ip1"
type: "InnerProduct"
bottom: "pool3"
top: "ip1"
param { lr_mult: 1 } # learning rate for the filters
param { lr_mult: 2 } # learning rate for the biases
inner_product_param {
num_output: 64
weight_filler {
type: "gaussian"
std: 0.1
}
bias_filler {type: "constant“ }
# initialize the biases to zero (0)
}
}
• Required
– num_output (c_o): the number
of filters
• Input
– n * 컬러채널 * height * width
• Output
– n * c_o
LRN Layer (Local Response Normalization)
layer {
name: "norm1"
type: "LRN"
bottom: "pool1"
top: "norm1"
lrn_param {
local_size: 5
alpha: 0.0001
beta: 0.75
}
}
• performs a kind of “lateral
inhibition - 측면억제(側面抑制)” by
normalizing over local input
regions
• Each input value is divided
by (1+(α/n)∑ix2i)β,
– where n is the size of each local region,
and the sum is taken over the region
centered at that value
• Optional
– local_size [default 5]: the number of
channels to sum over (for cross channel
LRN) or the side length of the square
region to sum over (for within channel
LRN)
– alpha [default 1]: the scaling parameter
– beta [default 5]: the exponent
CIFAR-10 (2010, Hinton/Krizhevsky)
• 10 classes
• 32x32 color image
• 6,000 images per class
• 5,000 training / 1,000
test images per class
• 전체 60,000장 =
(5,000+1,000) 개 * 10개
클래스
Ilsvrc12
(ImageNet large Scale Visual Recognition Challenge 2012)
• AlexNet
• 1.3 million high-resolution images
– Resize to 224x224x3
• Class 1,000
• 150,000 per class
• Net
– five convolutional layers
– two globally connected layers
– final 1000-way softmax.
• Caffe/examples/cifar10 모델 사용
• 응 교수의 matlab homework
– 64x64x3 image
– 4 클래스(비행기, 자동차,고양이, 개)
– Train(클래스당 500개), Test(클래스당 500개)
• 준비
– Resize 64x64x3 -> 32x32x3
– Mean data
Mnist Model net
Convolution1
Pooling1
Convolution2
Pooling2
ReLU1
InneProduct1
Accuracy
Data
SoftmaxWithLoss
ReLU2
CNN Exercise net in MATLAB
Convolution1
Pooling1
Convolution2
Pooling2
Softmax
1000 iterations
Data
Feature Extracted
Data
cifar10 net
Convolution1 Pooling1
Convolution2 Pooling2
ReLU1
ReLU2
Convolution3 Pooling3ReLU3
InnerProduct1 InnerProduct2
Accuracy
Data
SoftmaxWithLoss
Caffe framework tutorial2
Ilsvrc12(ImageNet large Scale Visual Recognition Challenge 2012)
https://github.com/dnouri/skynet/blob/master/cuda-convnet/imagenet-layers/layers-imagenet.cfg
Conv1 F48 ReLU1
Conv2 F32 Pool2 S2
Pool1 S2
ReLU2
Conv3 F96
Conv4 F32
ReLU3
InnerProduct8
Accuracy
Data
SoftmaxWithLoss
LRN1
LRN2
ReLU4
Conv5 F32 ReLU5 Pool5 S2
InnerProduct6 ReLU6 Dropout6 0.5
InnerProduct7 ReLU7 Dropout7 0.5
Network Visualization
http://cs.stanford.edu/people/karpathy/convnetjs/
• Conv (32x32x16)
– filter (5x5x3)
– stride 1
• relu (32x32x16)
• pool (16x16x16)
– Filter (2x2,)
• conv (16x16x20)
– Filter (5x5x16)
• relu (16x16x20)
• pool (8x8x20)
– Filter (2x2)
• conv (8x8x20)
– Filter (5x5x20)
• relu (8x8x20)
• pool (4x4x20)
– Filter (2x2)
'airplane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck
Network Visualization
http://cs.stanford.edu/people/karpathy/convnetjs/
• Conv (32x32x16)
– filter (5x5x3)
– stride 1
• relu (32x32x16)
• pool (16x16x16)
– Filter (2x2,)
• conv (16x16x20)
– Filter (5x5x16)
• relu (16x16x20)
• pool (8x8x20)
– Filter (2x2)
• conv (8x8x20)
– Filter (5x5x20)
• relu (8x8x20)
• pool (4x4x20)
– Filter (2x2)
• InnerProduct
– (1x1x10)
• softmax
– (1x1x10)
• Input (32x32x3)
'airplane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck
ImageNet 사용법
• 보조 데이터 다운로드
– $ ./data/ilsvrc12/get_ilsvrc_aux.sh
• backup folder : caffe/examples/imagenet
• 수정 Create_imagenet.sh
– RESIZE=false -> RESIZE=true
• Create the leveldbs with
– $ ./examples/imagenet/create_imagenet.sh
• Create mean data
– $ ./examples/imagenet/make_imagenet_mean.sh
– 생성 : data/ilsvrc12/imagenet_mean.binaryproto
• Train Start
– $ ./build/tools/caffe train --solver=models/bvlc_reference_caffenet/solver.prototxt
ImageNet Data Prepare
• Requirement
– Save Image Files
• /path/to/imagenet/train/n01440764/n01440764_10026.JPEG
• /path/to/imagenet/val/ILSVRC2012_val_00000001.JPEG
– Describe Folder
• train.txt
– n01440764/n01440764_10026.JPEG 0
– n01440764/n01440764_10027.JPEG 1
• val.txt
– ILSVRC2012_val_00004614.JPEG 954
– ILSVRC2012_val_00004615.JPEG 211
• Let’s Make lmdb
– $ ./examples/imagenet/create_imagenet.sh
• ilsvrc12_train_lmdb for Train
• ilsvrc12_val_lmdb for validate
Mnist Model net 수정 실험
• Iteration 100
• Basic model loss:0.34
• Remove convoution1, pooling1,
– loss:0.411
• Remove ReLU1
– loss:0.426
• Remove InnerProduct1
– loss:0.450
• Remove pooling2
– loss:0.522
• Remove convolution2
– loss:0.753
Convolution1
Pooling1
Convolution2
Pooling2
ReLU1
InnerProduct1
InnerProduct2
Accuracy
Data
SoftmaxWithLoss
Test
• Classify (Airplane, car, cat, dog)
– 이미지 사이즈 64x64x3
• Model
– Matlab cnn net
• softMax Layer에서만 학습 Iteratoin = 1,000
• 이미지 사이즈 조절 없음
– cifar10 net
• Iteration 5000
• 이미지 사이즈 64x64x3 -> 32x32x3 으로 조절해서 DB 생성
– Cifar10 net
• Iteration 5000
• 사이즈 조절 없이 DB 생성
– Labellio
• deep learning web service
Prepare 1/2. Make .mdb
• create_imagenet.sh
RESIZE=true
if $RESIZE; then
RESIZE_HEIGHT=32
RESIZE_WIDTH=32
else
RESIZE_HEIGHT=0
RESIZE_WIDTH=0
Fi
GLOG_logtostderr=1
$TOOLS/convert_imageset 
--resize_height=$RESIZE_HEIGHT 
--resize_width=$RESIZE_WIDTH 
--shuffle 
$TRAIN_DATA_ROOT 
$DATA/train.txt 
$EXAMPLE/ilsvrc12_train_lmdb
• 1. 64x64x3 이미지들을 폴더에 저장하고
• 2. train.txt 파일에 이미지 경로와 라벨을
모두 적는다.
• 3. lmdb 데이터 베이스를 만든다.
• lmdb 데이터 베이스를 만들 때 사이즈를
조절 할 수 있다.
– 사이즈를 조절 시 RESIZE=true
– 사이즈를 조절 NO RESIZE=false
– 사이즈를 조절 시 Data.mdb(32.9MB)
– 사이즈를 조절 NO Data.mdb(8.3MB)
– Lock.mbd 크기는 항상 고정 이다 8.2 kB
Prepare 2/2. Make mean data
• make_imagenet_mean.sh
EXAMPLE=examples/imagenet
DATA=data/ilsvrc12
TOOLS=build/tools
$TOOLS/compute_image_mean
$EXAMPLE/ilsvrc12_train_lmdb 
$DATA/mean.binaryproto
Result
• Matlab cnn net
– softMax Layer에서만 학습 Iteratoin = 1,000
– 이미지 사이즈 조절 없음
– Testing accuracy = 0.8
• cifar10 net
– 이미지 사이즈 64x64x3 -> 32x32x3 으로 조절 해서 db생성
– Iteration 5,000, loss = 0.00059
– Testing accuracy = 0.755, loss=1.58
– Overfitting!
• Cifar10 net
– Iteration 5000, traning loss = 0.00026
– Test accuracy = 0.724, loss=1.94
– worse overfitting!
• Labellio
– Training accuracy = 0.66
– Test accuracy = ?
Labellio
Labellio
Labellio
Caffe framework tutorial2
참고
• Face Feature Recognition System with
Deep Belief Networks, for Korean/KIISE
Thesis
– http://www.slideshare.net/sogo1127/face-
feature-recognition-system-with-deep-belief-
networks-for-korean
• Labellio
– http://devblogs.nvidia.com/parallelforall/label
lio-scalable-cloud-architecture-efficient-
multi-gpu-deep-learning/

More Related Content

What's hot

Deep Learning in Computer Vision
Deep Learning in Computer VisionDeep Learning in Computer Vision
Deep Learning in Computer VisionSungjoon Choi
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowS N
 
TensorFlow Tutorial Part2
TensorFlow Tutorial Part2TensorFlow Tutorial Part2
TensorFlow Tutorial Part2Sungjoon Choi
 
Introduction to deep learning in python and Matlab
Introduction to deep learning in python and MatlabIntroduction to deep learning in python and Matlab
Introduction to deep learning in python and MatlabImry Kissos
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntel Nervana
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016MLconf
 
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...Altoros
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural NetworksDatabricks
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Oswald Campesato
 
Case Study of Convolutional Neural Network
Case Study of Convolutional Neural NetworkCase Study of Convolutional Neural Network
Case Study of Convolutional Neural NetworkNamHyuk Ahn
 
Keras on tensorflow in R & Python
Keras on tensorflow in R & PythonKeras on tensorflow in R & Python
Keras on tensorflow in R & PythonLonghow Lam
 
Urs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural NetworksUrs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural NetworksIntel Nervana
 
IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」Preferred Networks
 
Intro to Scalable Deep Learning on AWS with Apache MXNet
Intro to Scalable Deep Learning on AWS with Apache MXNetIntro to Scalable Deep Learning on AWS with Apache MXNet
Intro to Scalable Deep Learning on AWS with Apache MXNetAmazon Web Services
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksDatabricks
 
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflowDistributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflowEmanuel Di Nardo
 
Introduction to Chainer Chemistry
Introduction to Chainer ChemistryIntroduction to Chainer Chemistry
Introduction to Chainer ChemistryPreferred Networks
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlowRajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlowAI Frontiers
 

What's hot (20)

Deep Learning in Computer Vision
Deep Learning in Computer VisionDeep Learning in Computer Vision
Deep Learning in Computer Vision
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
 
TensorFlow Tutorial Part2
TensorFlow Tutorial Part2TensorFlow Tutorial Part2
TensorFlow Tutorial Part2
 
Introduction to deep learning in python and Matlab
Introduction to deep learning in python and MatlabIntroduction to deep learning in python and Matlab
Introduction to deep learning in python and Matlab
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
 
nn network
nn networknn network
nn network
 
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
 
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)
 
Case Study of Convolutional Neural Network
Case Study of Convolutional Neural NetworkCase Study of Convolutional Neural Network
Case Study of Convolutional Neural Network
 
Keras on tensorflow in R & Python
Keras on tensorflow in R & PythonKeras on tensorflow in R & Python
Keras on tensorflow in R & Python
 
Urs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural NetworksUrs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural Networks
 
IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」
 
Intro to Scalable Deep Learning on AWS with Apache MXNet
Intro to Scalable Deep Learning on AWS with Apache MXNetIntro to Scalable Deep Learning on AWS with Apache MXNet
Intro to Scalable Deep Learning on AWS with Apache MXNet
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural Networks
 
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflowDistributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
 
Introduction to Chainer Chemistry
Introduction to Chainer ChemistryIntroduction to Chainer Chemistry
Introduction to Chainer Chemistry
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlow
 
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlowRajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
 

Viewers also liked

Caffe - A deep learning framework (Ramin Fahimi)
Caffe - A deep learning framework (Ramin Fahimi)Caffe - A deep learning framework (Ramin Fahimi)
Caffe - A deep learning framework (Ramin Fahimi)irpycon
 
Processor, Compiler and Python Programming Language
Processor, Compiler and Python Programming LanguageProcessor, Compiler and Python Programming Language
Processor, Compiler and Python Programming Languagearumdapta98
 
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...Petroleum Training Institute
 
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream)
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream) Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream)
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream) IT Arena
 
Using Gradient Descent for Optimization and Learning
Using Gradient Descent for Optimization and LearningUsing Gradient Descent for Optimization and Learning
Using Gradient Descent for Optimization and LearningDr. Volkan OBAN
 
Pattern Recognition and Machine Learning : Graphical Models
Pattern Recognition and Machine Learning : Graphical ModelsPattern Recognition and Machine Learning : Graphical Models
Pattern Recognition and Machine Learning : Graphical Modelsbutest
 
Semi fragile watermarking
Semi fragile watermarkingSemi fragile watermarking
Semi fragile watermarkingYash Diwakar
 
Computer vision, machine, and deep learning
Computer vision, machine, and deep learningComputer vision, machine, and deep learning
Computer vision, machine, and deep learningIgi Ardiyanto
 
[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkit[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkitde:code 2017
 
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...Joe Suzuki
 
Optimization in deep learning
Optimization in deep learningOptimization in deep learning
Optimization in deep learningJeremy Nixon
 
Center loss for Face Recognition
Center loss for Face RecognitionCenter loss for Face Recognition
Center loss for Face RecognitionJisung Kim
 
Face recognition and deep learning โดย ดร. สรรพฤทธิ์ มฤคทัต NECTEC
Face recognition and deep learning  โดย ดร. สรรพฤทธิ์ มฤคทัต NECTECFace recognition and deep learning  โดย ดร. สรรพฤทธิ์ มฤคทัต NECTEC
Face recognition and deep learning โดย ดร. สรรพฤทธิ์ มฤคทัต NECTECBAINIDA
 
Rattani - Ph.D. Defense Slides
Rattani - Ph.D. Defense SlidesRattani - Ph.D. Defense Slides
Rattani - Ph.D. Defense SlidesPluribus One
 
怖くない誤差逆伝播法 Chainerを添えて
怖くない誤差逆伝播法 Chainerを添えて怖くない誤差逆伝播法 Chainerを添えて
怖くない誤差逆伝播法 Chainerを添えてmarujirou
 
Pattern Recognition and Machine Learning: Section 3.3
Pattern Recognition and Machine Learning: Section 3.3Pattern Recognition and Machine Learning: Section 3.3
Pattern Recognition and Machine Learning: Section 3.3Yusuke Oda
 
Face Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural NetworksFace Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural NetworksElaheh Rashedi
 

Viewers also liked (20)

Caffe - A deep learning framework (Ramin Fahimi)
Caffe - A deep learning framework (Ramin Fahimi)Caffe - A deep learning framework (Ramin Fahimi)
Caffe - A deep learning framework (Ramin Fahimi)
 
Processor, Compiler and Python Programming Language
Processor, Compiler and Python Programming LanguageProcessor, Compiler and Python Programming Language
Processor, Compiler and Python Programming Language
 
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...
Muzammil Abdulrahman PPT On Gabor Wavelet Transform (GWT) Based Facial Expres...
 
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream)
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream) Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream)
Face Recognition Based on Deep Learning (Yurii Pashchenko Technology Stream)
 
Using Gradient Descent for Optimization and Learning
Using Gradient Descent for Optimization and LearningUsing Gradient Descent for Optimization and Learning
Using Gradient Descent for Optimization and Learning
 
Pattern Recognition and Machine Learning : Graphical Models
Pattern Recognition and Machine Learning : Graphical ModelsPattern Recognition and Machine Learning : Graphical Models
Pattern Recognition and Machine Learning : Graphical Models
 
Facebook Deep face
Facebook Deep faceFacebook Deep face
Facebook Deep face
 
Semi fragile watermarking
Semi fragile watermarkingSemi fragile watermarking
Semi fragile watermarking
 
портфоліо Бабич О.А.
портфоліо Бабич О.А.портфоліо Бабич О.А.
портфоліо Бабич О.А.
 
Computer vision, machine, and deep learning
Computer vision, machine, and deep learningComputer vision, machine, and deep learning
Computer vision, machine, and deep learning
 
[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkit[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkit
 
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...
Structure Learning of Bayesian Networks with p Nodes from n Samples when n&lt...
 
Optimization in deep learning
Optimization in deep learningOptimization in deep learning
Optimization in deep learning
 
Center loss for Face Recognition
Center loss for Face RecognitionCenter loss for Face Recognition
Center loss for Face Recognition
 
Face recognition and deep learning โดย ดร. สรรพฤทธิ์ มฤคทัต NECTEC
Face recognition and deep learning  โดย ดร. สรรพฤทธิ์ มฤคทัต NECTECFace recognition and deep learning  โดย ดร. สรรพฤทธิ์ มฤคทัต NECTEC
Face recognition and deep learning โดย ดร. สรรพฤทธิ์ มฤคทัต NECTEC
 
Rattani - Ph.D. Defense Slides
Rattani - Ph.D. Defense SlidesRattani - Ph.D. Defense Slides
Rattani - Ph.D. Defense Slides
 
怖くない誤差逆伝播法 Chainerを添えて
怖くない誤差逆伝播法 Chainerを添えて怖くない誤差逆伝播法 Chainerを添えて
怖くない誤差逆伝播法 Chainerを添えて
 
Pattern Recognition and Machine Learning: Section 3.3
Pattern Recognition and Machine Learning: Section 3.3Pattern Recognition and Machine Learning: Section 3.3
Pattern Recognition and Machine Learning: Section 3.3
 
Face Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural NetworksFace Recognition Methods based on Convolutional Neural Networks
Face Recognition Methods based on Convolutional Neural Networks
 
Deep Learning for Computer Vision: Face Recognition (UPC 2016)
Deep Learning for Computer Vision: Face Recognition (UPC 2016)Deep Learning for Computer Vision: Face Recognition (UPC 2016)
Deep Learning for Computer Vision: Face Recognition (UPC 2016)
 

Similar to Caffe framework tutorial2

CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...
CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...
CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...Thom Lane
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learningmilad abbasi
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningMehrnaz Faraz
 
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory Access
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory AccessAccelerating Key Bioinformatics Tasks 100-fold by Improving Memory Access
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory AccessIgor Sfiligoi
 
Mathemetics module
Mathemetics moduleMathemetics module
Mathemetics modulemanikanta361
 
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflow
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflowNVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflow
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflowNVIDIA Taiwan
 
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...Daosheng Mu
 
Jvm Performance Tunning
Jvm Performance TunningJvm Performance Tunning
Jvm Performance TunningTerry Cho
 
Jvm Performance Tunning
Jvm Performance TunningJvm Performance Tunning
Jvm Performance Tunningguest1f2740
 
running stable diffusion on android
running stable diffusion on androidrunning stable diffusion on android
running stable diffusion on androidKoan-Sin Tan
 
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 Separating Hype from Reality in Deep Learning with Sameer Farooqui Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer FarooquiDatabricks
 
Sebastian Schelter – Distributed Machine Learing with the Samsara DSL
Sebastian Schelter – Distributed Machine Learing with the Samsara DSLSebastian Schelter – Distributed Machine Learing with the Samsara DSL
Sebastian Schelter – Distributed Machine Learing with the Samsara DSLFlink Forward
 
FreeBSD 2014 Flame Graphs
FreeBSD 2014 Flame GraphsFreeBSD 2014 Flame Graphs
FreeBSD 2014 Flame GraphsBrendan Gregg
 
StackNet Meta-Modelling framework
StackNet Meta-Modelling frameworkStackNet Meta-Modelling framework
StackNet Meta-Modelling frameworkSri Ambati
 
Beirut Java User Group JVM presentation
Beirut Java User Group JVM presentationBeirut Java User Group JVM presentation
Beirut Java User Group JVM presentationMahmoud Anouti
 
Python高级编程(二)
Python高级编程(二)Python高级编程(二)
Python高级编程(二)Qiangning Hong
 
On-the-fly Visual Category Search in Web-scale Image Collections
On-the-fly Visual Category Search in Web-scale Image CollectionsOn-the-fly Visual Category Search in Web-scale Image Collections
On-the-fly Visual Category Search in Web-scale Image CollectionsKen Chatfield
 

Similar to Caffe framework tutorial2 (20)

CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...
CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...
CIFAR-10 for DAWNBench: Wide ResNets, Mixup Augmentation and "Super Convergen...
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory Access
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory AccessAccelerating Key Bioinformatics Tasks 100-fold by Improving Memory Access
Accelerating Key Bioinformatics Tasks 100-fold by Improving Memory Access
 
Mathemetics module
Mathemetics moduleMathemetics module
Mathemetics module
 
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
 
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflow
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflowNVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflow
NVIDIA 深度學習教育機構 (DLI): Image segmentation with tensorflow
 
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on SparkInferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
 
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...
Using The New Flash Stage3D Web Technology To Build Your Own Next 3D Browser ...
 
Jvm Performance Tunning
Jvm Performance TunningJvm Performance Tunning
Jvm Performance Tunning
 
Jvm Performance Tunning
Jvm Performance TunningJvm Performance Tunning
Jvm Performance Tunning
 
running stable diffusion on android
running stable diffusion on androidrunning stable diffusion on android
running stable diffusion on android
 
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 Separating Hype from Reality in Deep Learning with Sameer Farooqui Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 
Sebastian Schelter – Distributed Machine Learing with the Samsara DSL
Sebastian Schelter – Distributed Machine Learing with the Samsara DSLSebastian Schelter – Distributed Machine Learing with the Samsara DSL
Sebastian Schelter – Distributed Machine Learing with the Samsara DSL
 
Jvm memory model
Jvm memory modelJvm memory model
Jvm memory model
 
FreeBSD 2014 Flame Graphs
FreeBSD 2014 Flame GraphsFreeBSD 2014 Flame Graphs
FreeBSD 2014 Flame Graphs
 
StackNet Meta-Modelling framework
StackNet Meta-Modelling frameworkStackNet Meta-Modelling framework
StackNet Meta-Modelling framework
 
Beirut Java User Group JVM presentation
Beirut Java User Group JVM presentationBeirut Java User Group JVM presentation
Beirut Java User Group JVM presentation
 
Python高级编程(二)
Python高级编程(二)Python高级编程(二)
Python高级编程(二)
 
On-the-fly Visual Category Search in Web-scale Image Collections
On-the-fly Visual Category Search in Web-scale Image CollectionsOn-the-fly Visual Category Search in Web-scale Image Collections
On-the-fly Visual Category Search in Web-scale Image Collections
 

Recently uploaded

Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 

Recently uploaded (17)

Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 

Caffe framework tutorial2

  • 2. Index • Layer – Data – ImageData – Convolution – Pooling – ReLU – InnerProduct – LRN • Net – Mnist – CIFAR-10 – ImageNet (Ilsvrc12) • Net change Test – Mnist – CIFAR-10 • 64x64x3 Image Folder • 64x64x3 Image Resize To 32x32x3
  • 3. Data Layer layer { name: "data" type: "Data" top: "data" top: "label" include { phase: TRAIN } transform_param { mean_file: "examples/cifar10/mean.binaryproto" } data_param { source: "examples/cifar10/ilsvrc12_train_lmdb" batch_size: 100 backend: LMDB } } • Required – source: the name of the directory containing the database – batch_size: the number of inputs to process at one time • Optional – backend [default LEVELDB]: choose whether to use a LEVELDB or LMDB LevelDB ,LMDB : efficient databases • Example – Data.mdb(41MB) – Lock.mdb(8.2kB
  • 4. ImageData Layer layer { name: "data" type: "ImageData" top: "data" top: "label" transform_param { mirror: false crop_size: 227 mean_file: "data/ilsvrc12/imagenet_mean.binaryproto" } image_data_param { source: "examples/_temp/file_list.txt" batch_size: 50 new_height: 256 new_width: 256 } } • Required – source: name of a text file, with each line giving an image filename and label – batch_size: number of images to batch together
  • 5. Convolution Layer layer { name: "conv1" type: "Convolution" bottom: "data" top: "conv1“ param { lr_mult: 1 } # learning rate for the filters param { lr_mult: 2 } # learning rate for the biases convolution_param { num_output: 32 # learn 32 filters kernel_size: 5 # each filter is 5x5 pad: 2 stride: 1 # step 1 pixels between each filter application weight_filler { type: "gaussian“# initialize the filters from a Gaussian std: 0.0001 # default mean: 0 } # initialize the biases to zero (0) bias_filler { type: "constant" } } } • Required – num_output (c_o): the number of filters – kernel_size (or kernel_h and kern el_w): specifies height and width of each filter • Strongly Recommended – weight_filler [default type: 'constant' value: 0] • Optional – pad [default 0]: specifies the number of pixels to (implicitly) add to each side of the input – stride [default 1]: specifies the intervals at which to apply the filters to the input
  • 6. Pooling Layer layer { name: "pool1" type: "Pooling" bottom: "conv1" top: "pool1" pooling_param { pool: MAX kernel_size: 3 stride: 2 } } • Required – kernel_size : specifies height and width of each filter • Optional – pool [default MAX]: the pooling method. Currently MAX, AVE, or STOCHASTIC – pad [default 0]: specifies the number of pixels to (implicitly) add to each side of the input – stride [default 1]: specifies the intervals at which to apply the filters to the input • 예) stride 2 : step two pixels (in the bottom blob) between pooling regions
  • 7. ReLU Layer (Rectified-Linear and Leaky-ReLU) Rectified 정류된, Leaky-ReLU 새는, 구멍이 난 layer { name: "relu1" type: "ReLU" bottom: "conv1" top: "conv1" } • Parameters optional • negative_slope [default 0]: – specifies whether to leak the negative part by multiplying it with the slope value rather than setting it to 0. • Input x, Compute Output y • y = x (if x > 0) • y = x * negative_slope (if x <= 0)
  • 8. Why ReLU, Drop-Out! • Drop-Out – 2014. Toronto. paper title • Dropout : A Simple Way to Prevent Neural Networks from Overfitting • The key idea is to randomly drop units (along with their connections) from the neural network during training – Regularizer 의 일종 – Hidden Node를 모두 훈련시키지 않고, Random하게 Drop Out시킨다 • 관련된 Weight들은 훈련되지 않는다.
  • 9. Drop-Out • … using ReLUs trained with dropout during frame level training provide an 4.2% relative improvement over a DNN trained with sigmoid units…
  • 10. Rectified-Linear unit(ReLU) • Drop-Out 은 학습이 느리다. – Drop Out된 weight들은 학습이 일어나지 않는다. • Non-Linear Activation Function의 교체 – 일반적으로 사용되는 Logistic Sigmoid, tanh 대신 ReLu 사용 • ReLU의 장점 – reduced likelihood of the gradient to vanish – Sparsity
  • 12. InnerProduct Layer layer { name: "ip1" type: "InnerProduct" bottom: "pool3" top: "ip1" param { lr_mult: 1 } # learning rate for the filters param { lr_mult: 2 } # learning rate for the biases inner_product_param { num_output: 64 weight_filler { type: "gaussian" std: 0.1 } bias_filler {type: "constant“ } # initialize the biases to zero (0) } } • Required – num_output (c_o): the number of filters • Input – n * 컬러채널 * height * width • Output – n * c_o
  • 13. LRN Layer (Local Response Normalization) layer { name: "norm1" type: "LRN" bottom: "pool1" top: "norm1" lrn_param { local_size: 5 alpha: 0.0001 beta: 0.75 } } • performs a kind of “lateral inhibition - 측면억제(側面抑制)” by normalizing over local input regions • Each input value is divided by (1+(α/n)∑ix2i)β, – where n is the size of each local region, and the sum is taken over the region centered at that value • Optional – local_size [default 5]: the number of channels to sum over (for cross channel LRN) or the side length of the square region to sum over (for within channel LRN) – alpha [default 1]: the scaling parameter – beta [default 5]: the exponent
  • 14. CIFAR-10 (2010, Hinton/Krizhevsky) • 10 classes • 32x32 color image • 6,000 images per class • 5,000 training / 1,000 test images per class • 전체 60,000장 = (5,000+1,000) 개 * 10개 클래스
  • 15. Ilsvrc12 (ImageNet large Scale Visual Recognition Challenge 2012) • AlexNet • 1.3 million high-resolution images – Resize to 224x224x3 • Class 1,000 • 150,000 per class • Net – five convolutional layers – two globally connected layers – final 1000-way softmax.
  • 16. • Caffe/examples/cifar10 모델 사용 • 응 교수의 matlab homework – 64x64x3 image – 4 클래스(비행기, 자동차,고양이, 개) – Train(클래스당 500개), Test(클래스당 500개) • 준비 – Resize 64x64x3 -> 32x32x3 – Mean data
  • 18. CNN Exercise net in MATLAB Convolution1 Pooling1 Convolution2 Pooling2 Softmax 1000 iterations Data Feature Extracted Data
  • 19. cifar10 net Convolution1 Pooling1 Convolution2 Pooling2 ReLU1 ReLU2 Convolution3 Pooling3ReLU3 InnerProduct1 InnerProduct2 Accuracy Data SoftmaxWithLoss
  • 21. Ilsvrc12(ImageNet large Scale Visual Recognition Challenge 2012) https://github.com/dnouri/skynet/blob/master/cuda-convnet/imagenet-layers/layers-imagenet.cfg Conv1 F48 ReLU1 Conv2 F32 Pool2 S2 Pool1 S2 ReLU2 Conv3 F96 Conv4 F32 ReLU3 InnerProduct8 Accuracy Data SoftmaxWithLoss LRN1 LRN2 ReLU4 Conv5 F32 ReLU5 Pool5 S2 InnerProduct6 ReLU6 Dropout6 0.5 InnerProduct7 ReLU7 Dropout7 0.5
  • 22. Network Visualization http://cs.stanford.edu/people/karpathy/convnetjs/ • Conv (32x32x16) – filter (5x5x3) – stride 1 • relu (32x32x16) • pool (16x16x16) – Filter (2x2,) • conv (16x16x20) – Filter (5x5x16) • relu (16x16x20) • pool (8x8x20) – Filter (2x2) • conv (8x8x20) – Filter (5x5x20) • relu (8x8x20) • pool (4x4x20) – Filter (2x2) 'airplane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck
  • 23. Network Visualization http://cs.stanford.edu/people/karpathy/convnetjs/ • Conv (32x32x16) – filter (5x5x3) – stride 1 • relu (32x32x16) • pool (16x16x16) – Filter (2x2,) • conv (16x16x20) – Filter (5x5x16) • relu (16x16x20) • pool (8x8x20) – Filter (2x2) • conv (8x8x20) – Filter (5x5x20) • relu (8x8x20) • pool (4x4x20) – Filter (2x2) • InnerProduct – (1x1x10) • softmax – (1x1x10) • Input (32x32x3) 'airplane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck
  • 24. ImageNet 사용법 • 보조 데이터 다운로드 – $ ./data/ilsvrc12/get_ilsvrc_aux.sh • backup folder : caffe/examples/imagenet • 수정 Create_imagenet.sh – RESIZE=false -> RESIZE=true • Create the leveldbs with – $ ./examples/imagenet/create_imagenet.sh • Create mean data – $ ./examples/imagenet/make_imagenet_mean.sh – 생성 : data/ilsvrc12/imagenet_mean.binaryproto • Train Start – $ ./build/tools/caffe train --solver=models/bvlc_reference_caffenet/solver.prototxt
  • 25. ImageNet Data Prepare • Requirement – Save Image Files • /path/to/imagenet/train/n01440764/n01440764_10026.JPEG • /path/to/imagenet/val/ILSVRC2012_val_00000001.JPEG – Describe Folder • train.txt – n01440764/n01440764_10026.JPEG 0 – n01440764/n01440764_10027.JPEG 1 • val.txt – ILSVRC2012_val_00004614.JPEG 954 – ILSVRC2012_val_00004615.JPEG 211 • Let’s Make lmdb – $ ./examples/imagenet/create_imagenet.sh • ilsvrc12_train_lmdb for Train • ilsvrc12_val_lmdb for validate
  • 26. Mnist Model net 수정 실험 • Iteration 100 • Basic model loss:0.34 • Remove convoution1, pooling1, – loss:0.411 • Remove ReLU1 – loss:0.426 • Remove InnerProduct1 – loss:0.450 • Remove pooling2 – loss:0.522 • Remove convolution2 – loss:0.753 Convolution1 Pooling1 Convolution2 Pooling2 ReLU1 InnerProduct1 InnerProduct2 Accuracy Data SoftmaxWithLoss
  • 27. Test • Classify (Airplane, car, cat, dog) – 이미지 사이즈 64x64x3 • Model – Matlab cnn net • softMax Layer에서만 학습 Iteratoin = 1,000 • 이미지 사이즈 조절 없음 – cifar10 net • Iteration 5000 • 이미지 사이즈 64x64x3 -> 32x32x3 으로 조절해서 DB 생성 – Cifar10 net • Iteration 5000 • 사이즈 조절 없이 DB 생성 – Labellio • deep learning web service
  • 28. Prepare 1/2. Make .mdb • create_imagenet.sh RESIZE=true if $RESIZE; then RESIZE_HEIGHT=32 RESIZE_WIDTH=32 else RESIZE_HEIGHT=0 RESIZE_WIDTH=0 Fi GLOG_logtostderr=1 $TOOLS/convert_imageset --resize_height=$RESIZE_HEIGHT --resize_width=$RESIZE_WIDTH --shuffle $TRAIN_DATA_ROOT $DATA/train.txt $EXAMPLE/ilsvrc12_train_lmdb • 1. 64x64x3 이미지들을 폴더에 저장하고 • 2. train.txt 파일에 이미지 경로와 라벨을 모두 적는다. • 3. lmdb 데이터 베이스를 만든다. • lmdb 데이터 베이스를 만들 때 사이즈를 조절 할 수 있다. – 사이즈를 조절 시 RESIZE=true – 사이즈를 조절 NO RESIZE=false – 사이즈를 조절 시 Data.mdb(32.9MB) – 사이즈를 조절 NO Data.mdb(8.3MB) – Lock.mbd 크기는 항상 고정 이다 8.2 kB
  • 29. Prepare 2/2. Make mean data • make_imagenet_mean.sh EXAMPLE=examples/imagenet DATA=data/ilsvrc12 TOOLS=build/tools $TOOLS/compute_image_mean $EXAMPLE/ilsvrc12_train_lmdb $DATA/mean.binaryproto
  • 30. Result • Matlab cnn net – softMax Layer에서만 학습 Iteratoin = 1,000 – 이미지 사이즈 조절 없음 – Testing accuracy = 0.8 • cifar10 net – 이미지 사이즈 64x64x3 -> 32x32x3 으로 조절 해서 db생성 – Iteration 5,000, loss = 0.00059 – Testing accuracy = 0.755, loss=1.58 – Overfitting! • Cifar10 net – Iteration 5000, traning loss = 0.00026 – Test accuracy = 0.724, loss=1.94 – worse overfitting! • Labellio – Training accuracy = 0.66 – Test accuracy = ?
  • 35. 참고 • Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE Thesis – http://www.slideshare.net/sogo1127/face- feature-recognition-system-with-deep-belief- networks-for-korean • Labellio – http://devblogs.nvidia.com/parallelforall/label lio-scalable-cloud-architecture-efficient- multi-gpu-deep-learning/

Editor's Notes

  1. STOCHASTIC 추계학(推計學)의, 확률(론)적인