2017年5月19日 Stapy x 理研AIP オープンソース研究会 https://startpython.connpass.com/event/55579/

1. 深層学習フレームワーク概要と
Chainerの事例紹介
Stapy x 理研AIP オープンソース研究会
@理化学研究所 ⾰新知能統合研究センター
2017年5⽉19⽇
株式会社Preferred Networks ⼤野健太
oono@preferred.jp

2. 大野 健太
• twitter: @delta2323_
• 経歴
• 数学専攻（修⼠） → 2011.4 PFI → 2014.10 PFN
• 担当分野
• バイオプロジェクト
• Chainerコアチーム
• インターン・採⽤チーム
2

3. Chainer Meetup #05
2017年6⽉10⽇
@⽇本マイクロソフト株式会社
品川オフィス
3

4. • 2014年3⽉設⽴
• 本社：東京 アメリカ⼦会社：カリフォルニア州サンマテオ
• 社員数：約80名（8割以上はエンジニア・リサーチャー）
• 事業内容：深層学習の産業、特に産業⽤ロボット・交通・バイオ
ヘルスケアへの応⽤
4
YouTube Channel
preferredjp
Factory
Robot
Healthcare
Automotive

5. Warring states period of deep learning
frameworks
5

6. Technology stack of a DL framework
6
name functions example
Graphical visualization DIGITS, TensorBoard
Machine learning workflow
management
Dataset prep, Save/Load
Training loop
Keras, TF slim
Computational graph(CG)
management
Build/Optimize CGs
Forward/Back prop
Theano, TensorFlow
Torch.nn
Multi-dimensional
array processing
High-level array
manipulation
NumPy, CuPy
Eigen, Torch (core)
Numerical computation Matrix operation
Convolution
BLAS(OpenBLAS, MKL),
cuBLAS, cuDNN, MKL
DNN
Computational device CPU, GPU, TPU, FPGA

7. üconstructing NNs as a Python programming
üdynamic NN construction
üCPU/GPU agnostic code with CuPy
7
Mission
Speed up research and development of
deep learning and its applications.
Features
Flexible and intuitive description of complex NNs by
http://chainer.org
*NN = Neural Network

8. Software stack
8
CPU
CuPy
NVIDIA GPU
CUDA
cuDNN
BLAS
NumPy
Chainer
MKL

9. f g
x f g
Static
graph
construction
Data feed
x yf zg
Dynamic
graph
construction
Define-And-Run
(Most frameworks)
Define-By-Run
(Chainer)
9
x yf
x
z
Static Dynamic
Optimization ✓ △
Flexibility △ ✓

10. 10
(MinPy)
Era of dynamic graph frameworks

11. Construct NNs as
Python programming
class MLP(Link):
def __int__(self):
super(MLP, self).__init__(
l1=Linear(784, 1000),
l2=Linear(1000, 1000),
l3=Linear(1000, 10))
def __call__(x):
h1 = F.relu(self.l1(x))
h2 = F.relu(self.l2(l1))
return self.l3(h2)
Linear l1
ｘ
W bias
ReLU
Linear l2
h1
W bias
ReLU
Linear l3
h2
W bias
11

12. Release history
• 2015/06: v1.0.0
• 2015/09: v1.3.0 (CuPy)
• 2015/11: v1.5.0 (Link/Chain, CuPy in Cython)
• 2016/06: [MinPy]
• 2016/07: v1.11.0 (Trainer)
• 2017/01: [PyTorch, TensorFlow Fold]
• 2017/02: v2.0.0a
• 2017/04: v2.0.0b
• 2017/05: v1.24.0 (Last v1 release)

13. Chainer v2
• First major version up that breaks backward compatibility.
• Important features (almost fixed)
• CuPy separation
• Unified configuration (chainer.config, esp. train mode)
• train argument is removed from many functions
• Variable updated: Parameter class, uninitialized var, volatile removed
• Funcion.retain_inputs and retain_outputs to reduce memory usage
• New-style parameter/child link registration (just setting them as an attribute)
• UpdateRule customized for each parameter
• Extention.initialize added, invoke_before_training removed
• No duplicated memory between training graph and evaluation graph
• Input size is made optional in many links (L.Linear(100))
• wscale option is removed from many links
13
Will release on May 30th 2017

14. Libraries on top of Chainer
ChainerRL (beta): Reinforcement learning
ChainerMN (v0.1.0): Multi-node distributed learning
ChainerCV (v0.4.5): Computer vision

15. ChainerRL
15
ü Implement latest deep reinforcement learning algorithms
ü Work with OpenAI Gym

16. ChainerMN
16
ü Distributed deep learning with MPI, NCCL
ü approx. 100x speed with 128 GPUs

17. ChainerCV
17
ü Dataset Wrapper For well-known CV datasets (CUB, Pascal VOC)
ü Dataset transformer (random crop, random flop)
ü Implements typical workflow in CV

18. Application of Chainer
PaintsChainer : Line draw colorization tool
PonanzaChainer : Shogi AI with Deep Learning

19. PaintsChainer
19https://paintschainer.preferred.tech

20. Ponanza Chainer
20

21. NumPy-like API accelerated with CUDA
# CPU
x_cpu = numpy.array([1, 2, 3])
l2_cpu = numpy.linalg.norm(x_cpu)
# GPU
x_gpu = cupy.array([1, 2, 3])
l2_gpu = cupy.linalg.norm(x_gpu)
CuPy will be an independent project from Chainer from Chainer v2.
>150 NumPy functions are supported

22. Development team (as of May 2017)
• Chainer, CuPy
• Core development team: approx. 10 members
• Reviewer team: approx. 10 members
• Chainer user group: approx. 5 members
• Chainer RL, Chainer MN, Chainer CV: 2, 3 members for each
• Paints Chainer: approx. 10 members
22
* some members overlap

23. CI
23
Travis CI
Run all CPU tests of all PRs
Jenkins installation test

24. CI
24
Jenkins dairy test (internal): Run all tests with various configuration

25. Community activities
25
Chainer meetup (#0 - #4)
#5 will be held in June 10th 2017
Deep learning mini course @UCSF
application for biology
Google group (ja, en), Slack (ja, en), Twitter (ja, en)

26. Conclusion
• Chainer is a Python-based deep learning framework that
leverages flexible and intuitive description of NNs.
• Many libraries and services are being developed on top of
Chainer (ChainerRL/MN/CV, PaintsChainer, PonanzaChainer,
CuPy).
• Introduced the development and the user-group teams of
Chainer
26

27. Try Chainer ! http://chainer.org
27
Seiya
Tokui
Kenta
Oono
Yuya
Unno
Ryosuke
Okuta
Chainer core development team
Brian
Vogel
Gentaro
Watanabe
Shunta
Saito
Daisuke
Nishino
and many
contributors !
Contact: oono@preferred.jp
Google Group: Chainer User Group