SlideShare a Scribd company logo
1 of 55
Download to read offline
Julia Tokyo #2 
Julias»‚ 
Hamiltonian Monte Carlo 
(NUTSâ–) 
,‡ W+ (Kenta Sato) 
@bicycle1885 
1 / 55
²òÅòÃ 
ß…Ãu 
Ýê²Ô×7áòŪêì] (MCMC) 
Metropolis-Hastings 
Hamiltonian Monte Carlo (HMC) 
No-U-Turn Sampler (NUTS) 
JuliasžT—MCMCа–· 
2 / 55
_ŸvÑÇzkzP¼se 
MCMCw”—´òÖéò¯žÕè®Û®¸wck[vP 
Stanvuz´òÖè–zqvž£fkP 
JuliasžT—´òÖè–ž—–kP 
3 / 55
¶ 
ÔzZÖ|MCMCß9{
zw÷˜— _se 
D»0wrÃvø{?cvPsAaP 
Æ?w{Qz¶žorPŠeX|š“’T«PXN— 
WŽc˜ŠgŸ 
izÔ{izÝs®,crPkl]—tˆW–Še 
4 / 55
ß…Ãu / а–·Ãu 
5 / 55
ß…Ãu 
,‡ W+ 
Twitter/GitHub: @bicycle1885 
}: rRRȂȗ 
š: Bioinformatics 
JuliaÉ: ñz|›xW 
¾Yv{§: Julia / Haskell 
”[žR{§: Python / R 
6 / 55
HaskellŽ”™c[V,PcŠe 
²Haskellw”— Ê€ íÖì¯èÞò¯³z™ëÒä–žcŠc 
k} 
7 / 55
а–·Ãu 
DocOpt.jl - https://github.com/docopt/DocOpt.jl 
×êÖຖ·žÐ–¸c|²ÝòÈè£òÂDzЖ¸že— 
RandomForests.jl - 
https://github.com/bicycle1885/RandomForests.jl 
A†»?¡ê³é¹ßRandom ForestzJuliaö 
GeneOntology.jl - https://github.com/bicycle1885/GeneOntology.jl 
f»zGene OntologyzÖê¬ÂÇžö®cr— 
8 / 55
Ýê²Ô×7áòŪêì] 
(MCMC) 
9 / 55
MCMC]t{ 
1	Y
 
{›ft—˜k|±
Ú W•z´òÖêžÝê²Ô×7žJ 
Pr¤—´òÖéò¯]z~tp} 
¤•˜k´òÖêß9{
Úz?–
Œvuö“v–ž4‘e— 
zwž›˜—} 
Ýê²Ô×7t{|xWzæAz‹sÕzæAz±
ÚXEŠ—± 
f z_tžPR} 
1	9U
Y  9  Y
2
 9U  YU
  1	9U
Y  9U  YU
 
Y Y 5	Y Y
 
_ztY|æA W• „·¤e—±ž t„Y|·¤± 
t‚} 
10 / 55
Ýê²Ô×7z. 
·¤± 5	Y Y
  1	9U
Y  9U  Y
 
11 / 55
ccPMCMC 
	Y
 
ö0z±
Ú W•´òÖéò¯e—w{|·¤±Xºka 
v]˜|v•vP[õXN—} 

ÚzÓ/[: 
§ê³–È[: 
	Y
  D 5	Y Y
	Y
EY 
1	U
	Y
 ³ 	Y
 BT U ³ 7
 GPSBOZ 1	
	Y
 
©´òÖè–X_˜•žºketPR_tzÚvu{à{cvP 
(sYvP)} 
12 / 55
ûÕpósz´òÖéò¯{ÕcP 
Q	Y
 Q	Y
 
±Ã_D iz”cLBÃ_D XN—te—} 
ûÕpósz´òÖéò¯zÕca 
ýP¥°{|óÖz`[´ówK¯crP— 
Q	Y
 
cWci˜Xu_W{´òÖéò¯¼w{
W•vP 
2pzƒ 
1. izÝW•ýP[„ýP[„t‹ 
2. ýPt_™žfp]k•i_W•9Pt_™„{NŠ–íWvP 
➠ MCMC{Šawiz”Rvƒžt— 
13 / 55
Metropolis­Hastings 
Night View with Tokyo Tower Special Lightup (Shibakouen, Tokyo, Japan) by t-mizo is licensed under CC BY 2.0 
14 / 55
Metropolis­Hastings 
MCMC´òÖéò¯z~tps|Ž
ÚtPR´òÖéò¯ckP 

Út{*z
ÚW•°BFž–cc| P–v•izFž*v 
c|iRsv]˜|izÝwtuŠ—} 
R	ú 
 
°BFžfe—Ž
Ú {özvPcL
Úvu|´ò 
Öéò¯ceP
Úwe—} 
ú 	ú 	N

 
	ú 	N

  NJO
 
 Q	ú
R		N
  ú
 
°BF {’Az± s*va˜—: 
Q		N

R	ú 	N

 
Q	
 Q	
 
__s| {´òÖéò¯ckP
Ú z”cLBÃ_D 
15 / 55
Metropolis­Hastings 
z¡ê³é¹ß 
”cLB±
ÚD W•´òÖéò¯e— 
1. |æA žE| we— 
2. Ž
Ú W•kvF žt— 
3. ± s ž´òÖêtcr*vc|iRsv]˜| 
@qe— 
4. *va˜kÝ{ tc|@qa˜kÝ{ 
te— 
5. tcr|2~4ž z´òÖêX¤•˜—Šs– 
4e 
Q	
 
	
 N ±  
R	ú 	N

 ú 
	ú 	N

 ú 
	N
± ú 
	N
± 	N
 
N ± N
. 
16 / 55
Ž
Ú{cL
Ú( randn) 
# p: (unnormalized) probability density function 
# θ₀: initial state 
# M: number of samples 
# ϵ: step size 
function metropolis(p::Function, θ₀::Vector{Float64}, M::Int, ϵ::Float64) 
d = length(θ₀) 
# allocate samples' holder 
samples = Array(typeof(θ₀), M) 
# set the current state to the initial state 
θ = θ₀ 
for m in 1:M 
# generate a candidate sample from 
# the proposal distribution (normal distribution) 
θ̃ = randn(d) * ϵ + θ 
if rand()  min(1.0, p(θ̃) / p(θ)) 
# accept the proposal 
θ = θ̃ 
end 
samples[m] = θ 
print_sample(θ) 
end 
samples 
end 
metropolis.jl 
17 / 55
cc 
2/Dz/DwözN—cL
Ú 
# mean 
μ = [0.0, 0.0] 
# covariance matrix 
Σ = [1.0 0.8; 
0.8 1.0] 
# precision matrix 
Λ = inv(Σ) 
# unnormalized multivariate normal distribution 
normal = x - exp(-0.5 * ((x - μ)' * Λ * (x - μ))[1]) 
|–x₀|´òÖêDM|¸ÅÂÖüϵž®cr´òÖéò¯ 
samples = metropolis(normal, x₀, M, ϵ) 
18 / 55
HQ ­created 
with Gadfly.jl 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
4 
2 
0 
-2 
-4 
y 
Metropolis (ϵ = 1.0) 
19 / 55
HQ ­created 
with Gadfly.jl 
-3 -2 -1 0 1 
x 
Iteration 
500 
400 
300 
200 
100 
1 
0.5 
0.0 
-0.5 
-1.0 
-1.5 
-2.0 
y 
Metropolis (ϵ = 0.1) 
-3 -2 -1 0 1 2 
x 
Iteration 
500 
400 
300 
200 
100 
1 
3 
2 
1 
0 
-1 
-2 
-3 
y 
Metropolis (ϵ = 0.5) 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
4 
2 
0 
-2 
-4 
y 
Metropolis (ϵ = 1.0) 
-3 -2 -1 0 1 2 3 
x 
Iteration 
500 
400 
300 
200 
100 
1 
3 
2 
1 
0 
-1 
-2 
-3 
y 
Metropolis (ϵ = 2.0) 
20 / 55
Metropolis­Hastings 
zþUF 
1. @qzÇë–È©Ô 
1 
¸ÅÂÖ´£¹ z–s|@qt[ÿzÇë–È©ÔXN— 
2. èò¿ß¥¨–® 
´òÖêzÊXèò¿ß¥¨–®že— 
21 / 55
þU1: @qzÇë–È©Ô 
±
Úz–XK¯cr—z{`[´ól]} 
1 
1 
¸ÅÂÖ´£¹ R ➠ RY[”]—X|@qXÖX— 
¸ÅÂÖ´£¹ Ÿ ➠ @q{WT•˜—X|NŠ–”]vP 
MCMCW•v—…[¬€v´òÖꞤ—w{¸ÅÂÖ´£¹žRY 
[ckPX|@qXÖX—k´òÖéò¯zµXu[v—Çë 
–È©ÔXN—} 
1 
Õp(´òÖéò¯e—/D)w”orÐèà–¾ z P–X£v 
–|ۏXÕcP} 
22 / 55
þU2: èò¿ß¥¨–® 
Metropolis-HastingsW•¤•˜k´òÖêÊ{|èò¿ß¥¨–®ž 
crP— 
ú 	N
 
Ž
ÚXÜe—°BF {|xWz– W•‹r[0 
X¤”ck–W•ewûorYrcŠR_tXN— 
	N
 
èò¿ß¥¨–®s{(VVŠWw{or)`ûxDz[$w‡º 
ck—|cWvP 
óž…W•…ŠsU—zwWv–`ûxDX«Pwv— 
23 / 55
Hamiltonian Mote Carlo (HMC) 
Hamiltonian circuit on a small rhombicosidodecahedron by fdecomite is licensed under CC BY 2.0 
24 / 55
Hamiltonian Monte Carlo 
Hamiltonian Monte Carlo](HMC){|ÎÞêÇò¦»(Hamiltonian 
dynamics)žpwæŽa˜kMCMC]z~tp} 
±Ã_Dz¶žqJe— (|Œ0v±
Ú{sYvP) 
ószŠ¨z唞ðor´òÖꞤ— 
)zMCMCz¡ê³é¹ßt‡¶cr|öz vP P´òÖ 
êX¤•˜— 
_z]žPagkNo-U-Turn Sampler (NUTS){StantPR 
Ø£¹4zkzÖì¯èÞò¯{§wöa˜rP— 
25 / 55
Boltzmann
Ú 
Y 	Y
 1	Y
 
æA z§Ìê­– t±
Ú {Õz”Rw×Ô]• 
˜—} 
 
; 
1	Y
  FYQ 	+	Y

 
; 
__s| {±
ÚzcLBDsN—} 
_˜žuwžT||±
Úz§Ìê­–X4‘sY—} 
	Y
  + MPH 1	Y
 + MPH ; 
26 / 55
ÎÞêÇò¦» 
 S 
Š¨z唞æT—} žŠ¨z”ŸØ®Çê| žå”£Ø®Çê 
tckÔzŠ¨z唞E—ÎÞêÇò[ ä: 
EJ 
EU 
ESJ 
EU 
 ) 
SJ 
 + 
) 
J 
)	
 S
 6	
 
__s|ÎÞêÇÊ¡ò {ÜÅò¶âê§Ìê­– t 
唧Ìê­– z÷tcraa˜—} 
,	S
 
)	
 S
  6
,	S
 
27 / 55
´òÖéò¯„z%J 
/D 
”ŸØ®Çê : ´òÖéò¯ckP±/D 
唣خÇê : å”zBˆ0v/D 
§Ìê­– 
ÜÅò¶âê§Ìê­– : Boltzmann
Úž2w 
唧Ìê­– : 2|v唧Ìê­–w 
”ŸØ®Çêt唣خÇêz•Ô
Ú { 
”–’Az”Rw
wsY—} 
 
S 
6	
 
,	S
 
Q	
 S
 
)	
 S
  6
,	S
 
Q	
 S
  FYQ +)	
 S
  FYQ +6	
 FYQ 	+,	S

 
 
; 
 
; 
28 / 55
HMCz*?± 
Üa˜k°BFwe—*?±{’Az”Rwv—} 
	  
  NJO ú 
 FYQ 	)	
 S
 + )	ú
 S
ú
 
) )	
 S
 + )	ú
 S
ú   
   
võ0w{| z–{Ó/vzs ’T«f 
*va˜— ( ) {flX|²òÓä–¾sD–0wÎÞêÇò[ 
 äž|ŒBcrw[t«f¨1XPfe—kxö0w{@q{ 
ȓsvP} 
29 / 55
Leapfrog|ŒB 
ÎÞêÇò[ ä{w}0ww[z{ÕcPzs|D–
žíR} 
i_s{|Leapfrog|ŒBtPR’AzÒäžpWR} 
SJ 	U
1
 
J 	U
1
 
SJ 	U
1
 
 SJ	U
 + 
1 
 
6		U

 
J 
 J	U
1SJ	U
1
 
 SJ	U
1
 + 
1 
 
6		U
1

 
J 
30 / 55
vhLeapfrog|ŒBvzW 
Q	
 S
 )	
 S
 
•Ô
Ú žÓ/we—kw{| z9žÓ/w 
cv]˜|v•vP 
cWc|Euler]vus{(h_zuažÏÁcrŽ)9X/Bc 
rcŠRzs| Q	
 S
 
XÓ/wv•vP 
Leapfrog|ŒBs{|3pzÒä{i˜j˜¹	(shear 
mapping)vzs|i˜j˜2JcrŽ9X/BcvP 
VerticalShear m=1.25 by RobHar - Own work using Inkscape. Licensed under Public domain via Wikimedia Commons - 
http://commons.wikimedia.org/wiki/File:VerticalShear_m%3D1.25.svg#mediaviewer/File:VerticalShear_m%3D1.25.svg 
31 / 55
HMCw”—´òÖéò¯¡ê³é¹ß 
1. |æA žE| we— 
2. 唣žcL
ÚvuW•´òÖéò¯e— 
3. W•¸ÅÂÖ´£¹ sLeapfrog|ŒBw”—Òž x 
–4c| ž¤— 
4. ± s*vc|iRsv] 
˜|@qe— 
5. *va˜kÝ{ tc|@qa˜kÝ{ 
te— 
6. tcr|2~5ž z´òÖêX¤•˜—Šs– 
4e 
	
 N ±  
	N
 1 - 
ú 
NJO 
 FYQ 	)	
 S
 + )	ú
 S
ú
 
	N
± ú 
	N
± 	N
 
N ± N
. 
32 / 55
# U : potential energy function 
# ∇U : gradient of the potential energy function 
# θ₀ : initial state 
# M : number of samples 
# ϵ : step size 
# L : number of steps 
function hmc(U::Function, ∇U::Function, θ₀::Vector{Float64}, M::Int, ϵ::Float64, L::Int) 
d = length(θ₀) 
# allocate sampels' holder 
samples = Array(typeof(θ₀), M) 
# set the current sate to the initail state 
θ = θ₀ 
for m in 1:M 
# sample momentum variable 
p = randn(d) 
H = U(θ) + p ⋅ p / 2 
θ̃ = θ 
for l in 1:L 
p -= ϵ / 2 * ∇U(θ̃) # half step in momentum variable 
θ̃ += ϵ * p # full step in location variable 
p -= ϵ / 2 * ∇U(θ̃) # half step in momentum variable again 
end 
H̃ = U(θ̃) + p ⋅ p / 2 
if randn()  min(1.0, exp(H - H̃)) 
# accept the proposal 
θ = θ̃ 
end 
samples[m] = θ 
print_sample(θ) 
end 
samples 
end 
hmc.jl 
33 / 55
HQ ­created 
with Gadfly.jl 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
4 
2 
0 
-2 
-4 
y 
HMC (ϵ = 0.1, L = 10) 
-   34 / 55
HQ ­created 
with Gadfly.jl 
-2 -1 0 1 
x 
Iteration 
500 
400 
300 
200 
100 
1 
1 
0 
-1 
-2 
y 
HMC (ϵ = 0.01, L = 10) 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
3 
2 
1 
0 
-1 
-2 
-3 
y 
HMC (ϵ = 0.05, L = 10) 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
4 
2 
0 
-2 
-4 
y 
HMC (ϵ = 0.1, L = 10) 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
4 
2 
0 
-2 
-4 
y 
HMC (ϵ = 0.5, L = 10) 
-   
35 / 55
HQ ­created 
with Gadfly.jl 
-0.3 -0.2 -0.1 0.0 0.1 
x 
Iteration 
500 
400 
300 
200 
100 
1 
0.1 
0.0 
-0.1 
-0.2 
-0.3 
y 
HMC (ϵ = 0.01, L = 1) 
-1.0 -0.5 0.0 0.5 
x 
Iteration 
500 
400 
300 
200 
100 
1 
0.5 
0.0 
-0.5 
-1.0 
-1.5 
y 
HMC (ϵ = 0.05, L = 1) 
-2 -1 0 1 
x 
Iteration 
500 
400 
300 
200 
100 
1 
1 
0 
-1 
-2 
y 
HMC (ϵ = 0.1, L = 1) 
-4 -2 0 2 4 
x 
Iteration 
500 
400 
300 
200 
100 
1 
3 
2 
1 
0 
-1 
-2 
-3 
y 
HMC (ϵ = 0.5, L = 1) 
-   
36 / 55

More Related Content

What's hot

PRML輪読#2
PRML輪読#2PRML輪読#2
PRML輪読#2matsuolab
 
階層ベイズと自由エネルギー
階層ベイズと自由エネルギー階層ベイズと自由エネルギー
階層ベイズと自由エネルギーHiroshi Shimizu
 
あなたの心にBridgeSampling
あなたの心にBridgeSamplingあなたの心にBridgeSampling
あなたの心にBridgeSamplingdaiki hojo
 
強化学習その1
強化学習その1強化学習その1
強化学習その1nishio
 
PRML第6章「カーネル法」
PRML第6章「カーネル法」PRML第6章「カーネル法」
PRML第6章「カーネル法」Keisuke Sugawara
 
ホモトピー型理論入門
ホモトピー型理論入門ホモトピー型理論入門
ホモトピー型理論入門k h
 
POMDP下での強化学習の基礎と応用
POMDP下での強化学習の基礎と応用POMDP下での強化学習の基礎と応用
POMDP下での強化学習の基礎と応用Yasunori Ozaki
 
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)RyuichiKanoh
 
Prophet入門【R編】Facebookの時系列予測ツール
Prophet入門【R編】Facebookの時系列予測ツールProphet入門【R編】Facebookの時系列予測ツール
Prophet入門【R編】Facebookの時系列予測ツールhoxo_m
 
パターン認識と機械学習 §6.2 カーネル関数の構成
パターン認識と機械学習 §6.2 カーネル関数の構成パターン認識と機械学習 §6.2 カーネル関数の構成
パターン認識と機械学習 §6.2 カーネル関数の構成Prunus 1350
 
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半#みどりぼん 11章「空間構造のある階層ベイズモデル」後半
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半Katsushi Yamashita
 
統計的学習の基礎 5章前半(~5.6)
統計的学習の基礎 5章前半(~5.6)統計的学習の基礎 5章前半(~5.6)
統計的学習の基礎 5章前半(~5.6)Kota Mori
 
大規模な組合せ最適化問題に対する発見的解法
大規模な組合せ最適化問題に対する発見的解法大規模な組合せ最適化問題に対する発見的解法
大規模な組合せ最適化問題に対する発見的解法Shunji Umetani
 
比例ハザードモデルはとってもtricky!
比例ハザードモデルはとってもtricky!比例ハザードモデルはとってもtricky!
比例ハザードモデルはとってもtricky!takehikoihayashi
 
グラフィカルモデル入門
グラフィカルモデル入門グラフィカルモデル入門
グラフィカルモデル入門Kawamoto_Kazuhiko
 
統計的因果推論 勉強用 isseing333
統計的因果推論 勉強用 isseing333統計的因果推論 勉強用 isseing333
統計的因果推論 勉強用 isseing333Issei Kurahashi
 

What's hot (20)

PRML輪読#2
PRML輪読#2PRML輪読#2
PRML輪読#2
 
階層ベイズと自由エネルギー
階層ベイズと自由エネルギー階層ベイズと自由エネルギー
階層ベイズと自由エネルギー
 
あなたの心にBridgeSampling
あなたの心にBridgeSamplingあなたの心にBridgeSampling
あなたの心にBridgeSampling
 
強化学習その1
強化学習その1強化学習その1
強化学習その1
 
PRML第6章「カーネル法」
PRML第6章「カーネル法」PRML第6章「カーネル法」
PRML第6章「カーネル法」
 
ホモトピー型理論入門
ホモトピー型理論入門ホモトピー型理論入門
ホモトピー型理論入門
 
Stan超初心者入門
Stan超初心者入門Stan超初心者入門
Stan超初心者入門
 
POMDP下での強化学習の基礎と応用
POMDP下での強化学習の基礎と応用POMDP下での強化学習の基礎と応用
POMDP下での強化学習の基礎と応用
 
Rの高速化
Rの高速化Rの高速化
Rの高速化
 
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)
勾配ブースティングの基礎と最新の動向 (MIRU2020 Tutorial)
 
Prophet入門【R編】Facebookの時系列予測ツール
Prophet入門【R編】Facebookの時系列予測ツールProphet入門【R編】Facebookの時系列予測ツール
Prophet入門【R編】Facebookの時系列予測ツール
 
パターン認識と機械学習 §6.2 カーネル関数の構成
パターン認識と機械学習 §6.2 カーネル関数の構成パターン認識と機械学習 §6.2 カーネル関数の構成
パターン認識と機械学習 §6.2 カーネル関数の構成
 
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半#みどりぼん 11章「空間構造のある階層ベイズモデル」後半
#みどりぼん 11章「空間構造のある階層ベイズモデル」後半
 
統計的学習の基礎 5章前半(~5.6)
統計的学習の基礎 5章前半(~5.6)統計的学習の基礎 5章前半(~5.6)
統計的学習の基礎 5章前半(~5.6)
 
大規模な組合せ最適化問題に対する発見的解法
大規模な組合せ最適化問題に対する発見的解法大規模な組合せ最適化問題に対する発見的解法
大規模な組合せ最適化問題に対する発見的解法
 
比例ハザードモデルはとってもtricky!
比例ハザードモデルはとってもtricky!比例ハザードモデルはとってもtricky!
比例ハザードモデルはとってもtricky!
 
時系列分析入門
時系列分析入門時系列分析入門
時系列分析入門
 
WAICとWBICのご紹介
WAICとWBICのご紹介WAICとWBICのご紹介
WAICとWBICのご紹介
 
グラフィカルモデル入門
グラフィカルモデル入門グラフィカルモデル入門
グラフィカルモデル入門
 
統計的因果推論 勉強用 isseing333
統計的因果推論 勉強用 isseing333統計的因果推論 勉強用 isseing333
統計的因果推論 勉強用 isseing333
 

Similar to Juliaで学ぶ Hamiltonian Monte Carlo (NUTS 入り)

5.5 to 5.7 Review.pdf
5.5 to 5.7 Review.pdf5.5 to 5.7 Review.pdf
5.5 to 5.7 Review.pdfLomasAlg1
 
Fabricでお手軽サーバ管理
Fabricでお手軽サーバ管理Fabricでお手軽サーバ管理
Fabricでお手軽サーバ管理niratama
 
67243- cooling and heating & calculation
67243- cooling and heating  & calculation67243- cooling and heating  & calculation
67243- cooling and heating & calculationA.M. ATIQULLAH
 
Module 3 example 15
Module 3 example 15Module 3 example 15
Module 3 example 15ccooking
 
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)SAJJAD KHUDHUR ABBAS
 
Solutions_Manual_to_accompany_Applied_Nu.pdf
Solutions_Manual_to_accompany_Applied_Nu.pdfSolutions_Manual_to_accompany_Applied_Nu.pdf
Solutions_Manual_to_accompany_Applied_Nu.pdfWaleedHussain30
 
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan Tanah
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan TanahPembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan Tanah
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan TanahChristopherAbhistaAr
 
TC74VCX244FT PSpice Model (Free SPICE Model)
TC74VCX244FT PSpice Model (Free SPICE Model)TC74VCX244FT PSpice Model (Free SPICE Model)
TC74VCX244FT PSpice Model (Free SPICE Model)Tsuyoshi Horigome
 
vibration of machines and structures
vibration of machines and structuresvibration of machines and structures
vibration of machines and structuresAniruddhsinh Barad
 
Mathematical Modelling of Electrical/Mechanical modellinng in MATLAB
Mathematical Modelling of Electrical/Mechanical modellinng in MATLABMathematical Modelling of Electrical/Mechanical modellinng in MATLAB
Mathematical Modelling of Electrical/Mechanical modellinng in MATLABCOMSATS Abbottabad
 
kintone university 03-1. AD+カスタマイズ入門第6版 サンプル
kintone university 03-1. AD+カスタマイズ入門第6版 サンプルkintone university 03-1. AD+カスタマイズ入門第6版 サンプル
kintone university 03-1. AD+カスタマイズ入門第6版 サンプルYudai Shibuya
 
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...Jhonatan Minchán
 
Sol mat haeussler_by_priale
Sol mat haeussler_by_prialeSol mat haeussler_by_priale
Sol mat haeussler_by_prialeJeff Chasi
 
31350052 introductory-mathematical-analysis-textbook-solution-manual
31350052 introductory-mathematical-analysis-textbook-solution-manual31350052 introductory-mathematical-analysis-textbook-solution-manual
31350052 introductory-mathematical-analysis-textbook-solution-manualMahrukh Khalid
 
Solucionario de matemáticas para administación y economia
Solucionario de matemáticas para administación y economiaSolucionario de matemáticas para administación y economia
Solucionario de matemáticas para administación y economiaLuis Perez Anampa
 
Aspect math [www.onlinebcs.com]
Aspect math [www.onlinebcs.com]Aspect math [www.onlinebcs.com]
Aspect math [www.onlinebcs.com]Itmona
 

Similar to Juliaで学ぶ Hamiltonian Monte Carlo (NUTS 入り) (20)

5.5 to 5.7 Review.pdf
5.5 to 5.7 Review.pdf5.5 to 5.7 Review.pdf
5.5 to 5.7 Review.pdf
 
Fabricでお手軽サーバ管理
Fabricでお手軽サーバ管理Fabricでお手軽サーバ管理
Fabricでお手軽サーバ管理
 
67243- cooling and heating & calculation
67243- cooling and heating  & calculation67243- cooling and heating  & calculation
67243- cooling and heating & calculation
 
Examen final
Examen finalExamen final
Examen final
 
HIDRAULICA DE CANALES
HIDRAULICA DE CANALESHIDRAULICA DE CANALES
HIDRAULICA DE CANALES
 
Module 3 example 15
Module 3 example 15Module 3 example 15
Module 3 example 15
 
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)
Episode 40 : DESIGN EXAMPLE – DILUTE PHASE PNEUMATIC CONVEYING (Part 2)
 
Cholesky sqrt venka
Cholesky sqrt venkaCholesky sqrt venka
Cholesky sqrt venka
 
Solutions_Manual_to_accompany_Applied_Nu.pdf
Solutions_Manual_to_accompany_Applied_Nu.pdfSolutions_Manual_to_accompany_Applied_Nu.pdf
Solutions_Manual_to_accompany_Applied_Nu.pdf
 
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan Tanah
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan TanahPembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan Tanah
Pembahasan Soal Teknik Fondasi Telapak dan Dinding Penahan Tanah
 
TC74VCX244FT PSpice Model (Free SPICE Model)
TC74VCX244FT PSpice Model (Free SPICE Model)TC74VCX244FT PSpice Model (Free SPICE Model)
TC74VCX244FT PSpice Model (Free SPICE Model)
 
vibration of machines and structures
vibration of machines and structuresvibration of machines and structures
vibration of machines and structures
 
project designa.docx
project designa.docxproject designa.docx
project designa.docx
 
Mathematical Modelling of Electrical/Mechanical modellinng in MATLAB
Mathematical Modelling of Electrical/Mechanical modellinng in MATLABMathematical Modelling of Electrical/Mechanical modellinng in MATLAB
Mathematical Modelling of Electrical/Mechanical modellinng in MATLAB
 
kintone university 03-1. AD+カスタマイズ入門第6版 サンプル
kintone university 03-1. AD+カスタマイズ入門第6版 サンプルkintone university 03-1. AD+カスタマイズ入門第6版 サンプル
kintone university 03-1. AD+カスタマイズ入門第6版 サンプル
 
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...
Ernest f. haeussler, richard s. paul y richard j. wood. matemáticas para admi...
 
Sol mat haeussler_by_priale
Sol mat haeussler_by_prialeSol mat haeussler_by_priale
Sol mat haeussler_by_priale
 
31350052 introductory-mathematical-analysis-textbook-solution-manual
31350052 introductory-mathematical-analysis-textbook-solution-manual31350052 introductory-mathematical-analysis-textbook-solution-manual
31350052 introductory-mathematical-analysis-textbook-solution-manual
 
Solucionario de matemáticas para administación y economia
Solucionario de matemáticas para administación y economiaSolucionario de matemáticas para administación y economia
Solucionario de matemáticas para administación y economia
 
Aspect math [www.onlinebcs.com]
Aspect math [www.onlinebcs.com]Aspect math [www.onlinebcs.com]
Aspect math [www.onlinebcs.com]
 

More from Kenta Sato

Julia最新情報 2015
Julia最新情報 2015Julia最新情報 2015
Julia最新情報 2015Kenta Sato
 
Introduction to Julia for bioinformacis
Introduction to Julia for bioinformacisIntroduction to Julia for bioinformacis
Introduction to Julia for bioinformacisKenta Sato
 
Juliaのパッケージをつくろう!
Juliaのパッケージをつくろう!Juliaのパッケージをつくろう!
Juliaのパッケージをつくろう!Kenta Sato
 
Julia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerJulia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerKenta Sato
 
実践! D3.jsで可視化入門
実践! D3.jsで可視化入門実践! D3.jsで可視化入門
実践! D3.jsで可視化入門Kenta Sato
 
モナドがいっぱい!
モナドがいっぱい!モナドがいっぱい!
モナドがいっぱい!Kenta Sato
 

More from Kenta Sato (6)

Julia最新情報 2015
Julia最新情報 2015Julia最新情報 2015
Julia最新情報 2015
 
Introduction to Julia for bioinformacis
Introduction to Julia for bioinformacisIntroduction to Julia for bioinformacis
Introduction to Julia for bioinformacis
 
Juliaのパッケージをつくろう!
Juliaのパッケージをつくろう!Juliaのパッケージをつくろう!
Juliaのパッケージをつくろう!
 
Julia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, StrongerJulia - Easier, Better, Faster, Stronger
Julia - Easier, Better, Faster, Stronger
 
実践! D3.jsで可視化入門
実践! D3.jsで可視化入門実践! D3.jsで可視化入門
実践! D3.jsで可視化入門
 
モナドがいっぱい!
モナドがいっぱい!モナドがいっぱい!
モナドがいっぱい!
 

Recently uploaded

HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKAR
HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKARHAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKAR
HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKARKOUSTAV SARKAR
 
data_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdfdata_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdfJiananWang21
 
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...Arindam Chakraborty, Ph.D., P.E. (CA, TX)
 
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptxOrlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptxMuhammadAsimMuhammad6
 
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...Amil baba
 
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLE
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLEGEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLE
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLEselvakumar948
 
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...drmkjayanthikannan
 
Moment Distribution Method For Btech Civil
Moment Distribution Method For Btech CivilMoment Distribution Method For Btech Civil
Moment Distribution Method For Btech CivilVinayVitekari
 
Double Revolving field theory-how the rotor develops torque
Double Revolving field theory-how the rotor develops torqueDouble Revolving field theory-how the rotor develops torque
Double Revolving field theory-how the rotor develops torqueBhangaleSonal
 
Thermal Engineering-R & A / C - unit - V
Thermal Engineering-R & A / C - unit - VThermal Engineering-R & A / C - unit - V
Thermal Engineering-R & A / C - unit - VDineshKumar4165
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayEpec Engineered Technologies
 
DC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equationDC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equationBhangaleSonal
 
Online food ordering system project report.pdf
Online food ordering system project report.pdfOnline food ordering system project report.pdf
Online food ordering system project report.pdfKamal Acharya
 
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdfAldoGarca30
 
Unleashing the Power of the SORA AI lastest leap
Unleashing the Power of the SORA AI lastest leapUnleashing the Power of the SORA AI lastest leap
Unleashing the Power of the SORA AI lastest leapRishantSharmaFr
 
Verification of thevenin's theorem for BEEE Lab (1).pptx
Verification of thevenin's theorem for BEEE Lab (1).pptxVerification of thevenin's theorem for BEEE Lab (1).pptx
Verification of thevenin's theorem for BEEE Lab (1).pptxchumtiyababu
 
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwait
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills KuwaitKuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwait
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwaitjaanualu31
 
School management system project Report.pdf
School management system project Report.pdfSchool management system project Report.pdf
School management system project Report.pdfKamal Acharya
 

Recently uploaded (20)

HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKAR
HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKARHAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKAR
HAND TOOLS USED AT ELECTRONICS WORK PRESENTED BY KOUSTAV SARKAR
 
data_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdfdata_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdf
 
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...
Navigating Complexity: The Role of Trusted Partners and VIAS3D in Dassault Sy...
 
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptxOrlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
Orlando’s Arnold Palmer Hospital Layout Strategy-1.pptx
 
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...
NO1 Top No1 Amil Baba In Azad Kashmir, Kashmir Black Magic Specialist Expert ...
 
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLE
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLEGEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLE
GEAR TRAIN- BASIC CONCEPTS AND WORKING PRINCIPLE
 
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...
Unit 4_Part 1 CSE2001 Exception Handling and Function Template and Class Temp...
 
Moment Distribution Method For Btech Civil
Moment Distribution Method For Btech CivilMoment Distribution Method For Btech Civil
Moment Distribution Method For Btech Civil
 
Double Revolving field theory-how the rotor develops torque
Double Revolving field theory-how the rotor develops torqueDouble Revolving field theory-how the rotor develops torque
Double Revolving field theory-how the rotor develops torque
 
Thermal Engineering-R & A / C - unit - V
Thermal Engineering-R & A / C - unit - VThermal Engineering-R & A / C - unit - V
Thermal Engineering-R & A / C - unit - V
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power Play
 
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced LoadsFEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads
 
Integrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - NeometrixIntegrated Test Rig For HTFE-25 - Neometrix
Integrated Test Rig For HTFE-25 - Neometrix
 
DC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equationDC MACHINE-Motoring and generation, Armature circuit equation
DC MACHINE-Motoring and generation, Armature circuit equation
 
Online food ordering system project report.pdf
Online food ordering system project report.pdfOnline food ordering system project report.pdf
Online food ordering system project report.pdf
 
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
1_Introduction + EAM Vocabulary + how to navigate in EAM.pdf
 
Unleashing the Power of the SORA AI lastest leap
Unleashing the Power of the SORA AI lastest leapUnleashing the Power of the SORA AI lastest leap
Unleashing the Power of the SORA AI lastest leap
 
Verification of thevenin's theorem for BEEE Lab (1).pptx
Verification of thevenin's theorem for BEEE Lab (1).pptxVerification of thevenin's theorem for BEEE Lab (1).pptx
Verification of thevenin's theorem for BEEE Lab (1).pptx
 
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwait
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills KuwaitKuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwait
Kuwait City MTP kit ((+919101817206)) Buy Abortion Pills Kuwait
 
School management system project Report.pdf
School management system project Report.pdfSchool management system project Report.pdf
School management system project Report.pdf
 

Juliaで学ぶ Hamiltonian Monte Carlo (NUTS 入り)

  • 1. Julia Tokyo #2 Julias»‚ Hamiltonian Monte Carlo (NUTSâ–) ,‡ W+ (Kenta Sato) @bicycle1885 1 / 55
  • 2. ²òÅòà ߅Ãu Ýê²Ô×7áòŪêì] (MCMC) Metropolis-Hastings Hamiltonian Monte Carlo (HMC) No-U-Turn Sampler (NUTS) JuliasžT—MCMCа–· 2 / 55
  • 5. zw÷˜— _se D»0wrÃvø{?cvPsAaP Æ?w{Qz¶žorPŠeX|š“’T«PXN— WŽc˜ŠgŸ izÔ{izÝs®,crPkl]—tˆW–Še 4 / 55
  • 7. ß…Ãu ,‡ W+ Twitter/GitHub: @bicycle1885 }: rRR»Ç»ó š: Bioinformatics JuliaÉ: ñz|›xW ¾Yv{§: Julia / Haskell ”[žR{§: Python / R 6 / 55
  • 8. HaskellŽ”™c[V,PcŠe ²Haskellw”— Ê€ íÖì¯èÞò¯³z™ëÒä–žcŠc k} 7 / 55
  • 9. а–·Ãu DocOpt.jl - https://github.com/docopt/DocOpt.jl ×êÖຖ·žÐ–¸c|²ÝòÈè£òÂDzЖ¸že— RandomForests.jl - https://github.com/bicycle1885/RandomForests.jl A†»?¡ê³é¹ßRandom ForestzJuliaö GeneOntology.jl - https://github.com/bicycle1885/GeneOntology.jl f»zGene OntologyzÖê¬ÂÇžö®cr— 8 / 55
  • 11. MCMC]t{ 1 Y {›ft—˜k|± Ú W•z´òÖêžÝê²Ô×7žJ Pr¤—´òÖéò¯]z~tp} ¤•˜k´òÖêß9{ Úz?– Œvuö“v–ž4‘e— zwž›˜—} Ýê²Ô×7t{|xWzæAz‹sÕzæAz± ÚXEŠ—± f z_tžPR} 1 9U
  • 12. Y 9 Y 2 9U YU 1 9U
  • 13. Y 9U YU Y Y 5 Y Y _ztY|æA W• „·¤e—±ž t„Y|·¤± t‚} 10 / 55
  • 15. Y 9U Y 11 / 55
  • 16. ccPMCMC Y ö0z± Ú W•´òÖéò¯e—w{|·¤±Xºka v]˜|v•vP[õXN—} ÚzÓ/[: §ê³–È[: Y D 5 Y Y Y EY 1 U Y ³ Y BT U ³ 7 GPSBOZ 1 Y ©´òÖè–X_˜•žºketPR_tzÚvu{à{cvP (sYvP)} 12 / 55
  • 17. ûÕpósz´òÖéò¯{ÕcP Q Y Q Y ±Ã_D iz”cLBÃ_D XN—te—} ûÕpósz´òÖéò¯zÕca ýP¥°{|óÖz`[´ówK¯crP— Q Y cWci˜Xu_W{´òÖéò¯¼w{ W•vP 2pzƒ 1. izÝW•ýP[„ýP[„t‹ 2. ýPt_™žfp]k•i_W•9Pt_™„{NŠ–íWvP ➠ MCMC{Šawiz”Rvƒžt— 13 / 55
  • 18. Metropolis­Hastings Night View with Tokyo Tower Special Lightup (Shibakouen, Tokyo, Japan) by t-mizo is licensed under CC BY 2.0 14 / 55
  • 19. Metropolis­Hastings MCMC´òÖéò¯z~tps|Ž ÚtPR´òÖéò¯ckP Út{*z ÚW•°BFž–cc| P–v•izFž*v c|iRsv]˜|izÝwtuŠ—} R ú °BFžfe—Ž Ú {özvPcL Úvu|´ò Öéò¯ceP Úwe—} ú ú N ú N NJO Q ú R N ú °BF {’Az± s*va˜—: Q N R ú N Q Q __s| {´òÖéò¯ckP Ú z”cLBÃ_D 15 / 55
  • 20. Metropolis­Hastings z¡ê³é¹ß ”cLB± ÚD W•´òÖéò¯e— 1. |æA žE| we— 2. Ž Ú W•kvF žt— 3. ± s ž´òÖêtcr*vc|iRsv]˜| @qe— 4. *va˜kÝ{ tc|@qa˜kÝ{ te— 5. tcr|2~4ž z´òÖêX¤•˜—Šs– 4e Q N ± R ú N ú ú N ú N
  • 22. ± N N ± N
  • 23. . 16 / 55
  • 24. Ž Ú{cL Ú( randn) # p: (unnormalized) probability density function # θ₀: initial state # M: number of samples # ϵ: step size function metropolis(p::Function, θ₀::Vector{Float64}, M::Int, ϵ::Float64) d = length(θ₀) # allocate samples' holder samples = Array(typeof(θ₀), M) # set the current state to the initial state θ = θ₀ for m in 1:M # generate a candidate sample from # the proposal distribution (normal distribution) θ̃ = randn(d) * ϵ + θ if rand() min(1.0, p(θ̃) / p(θ)) # accept the proposal θ = θ̃ end samples[m] = θ print_sample(θ) end samples end metropolis.jl 17 / 55
  • 25. cc 2/Dz/DwözN—cL Ú # mean μ = [0.0, 0.0] # covariance matrix Σ = [1.0 0.8; 0.8 1.0] # precision matrix Λ = inv(Σ) # unnormalized multivariate normal distribution normal = x - exp(-0.5 * ((x - μ)' * Λ * (x - μ))[1]) |–x₀|´òÖêDM|¸ÅÂÖüϵž®cr´òÖéò¯ samples = metropolis(normal, x₀, M, ϵ) 18 / 55
  • 26. HQ ­created with Gadfly.jl -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y Metropolis (ϵ = 1.0) 19 / 55
  • 27. HQ ­created with Gadfly.jl -3 -2 -1 0 1 x Iteration 500 400 300 200 100 1 0.5 0.0 -0.5 -1.0 -1.5 -2.0 y Metropolis (ϵ = 0.1) -3 -2 -1 0 1 2 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y Metropolis (ϵ = 0.5) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y Metropolis (ϵ = 1.0) -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y Metropolis (ϵ = 2.0) 20 / 55
  • 28. Metropolis­Hastings zþUF 1. @qzÇë–È©Ô 1 ¸ÅÂÖ´£¹ z–s|@qt[ÿzÇë–È©ÔXN— 2. èò¿ß¥¨–® ´òÖêzÊXèò¿ß¥¨–®že— 21 / 55
  • 29. þU1: @qzÇë–È©Ô ± Úz–XK¯cr—z{`[´ól]} 1 1 ¸ÅÂÖ´£¹ R ➠ RY[”]—X|@qXÖX— ¸ÅÂÖ´£¹ Ÿ ➠ @q{WT•˜—X|NŠ–”]vP MCMCW•v—…[¬€v´òÖꞤ—w{¸ÅÂÖ´£¹žRY [ckPX|@qXÖX—k´òÖéò¯zµXu[v—Çë –È©ÔXN—} 1 Õp(´òÖéò¯e—/D)w”orÐèà–¾ z P–X£v –|ۏXÕcP} 22 / 55
  • 30. þU2: èò¿ß¥¨–® Metropolis-HastingsW•¤•˜k´òÖêÊ{|èò¿ß¥¨–®ž crP— ú N Ž ÚXÜe—°BF {|xWz– W•‹r[0 X¤”ck–W•ewûorYrcŠR_tXN— N èò¿ß¥¨–®s{(VVŠWw{or)`ûxDz[$w‡º ck—|cWvP óž…W•…ŠsU—zwWv–`ûxDX«Pwv— 23 / 55
  • 31. Hamiltonian Mote Carlo (HMC) Hamiltonian circuit on a small rhombicosidodecahedron by fdecomite is licensed under CC BY 2.0 24 / 55
  • 32. Hamiltonian Monte Carlo Hamiltonian Monte Carlo](HMC){|ÎÞêÇò¦»(Hamiltonian dynamics)žpwæŽa˜kMCMC]z~tp} ±Ã_Dz¶žqJe— (|Œ0v± Ú{sYvP) ószŠ¨z唞ðor´òÖꞤ— )zMCMCz¡ê³é¹ßt‡¶cr|öz vP P´òÖ êX¤•˜— _z]žPagkNo-U-Turn Sampler (NUTS){StantPR Ø£¹4zkzÖì¯èÞò¯{§wöa˜rP— 25 / 55
  • 33. Boltzmann Ú Y Y 1 Y æA z§Ìê­– t± Ú {Õz”Rw×Ô]• ˜—} ; 1 Y FYQ + Y ; __s| {± ÚzcLBDsN—} _˜žuwžT||± Úz§Ìê­–X4‘sY—} Y + MPH 1 Y + MPH ; 26 / 55
  • 34. ÎÞêÇò¦» S Š¨z唞æT—} žŠ¨z”ŸØ®Çê| žå”£Ø®Çê tckÔzŠ¨z唞E—ÎÞêÇò[ ä: EJ EU ESJ EU ) SJ + ) J ) S 6 __s|ÎÞêÇÊ¡ò {ÜÅò¶âê§Ìê­– t 唧Ìê­– z÷tcraa˜—} , S ) S 6
  • 35. , S 27 / 55
  • 36. ´òÖéò¯„z%J /D ”ŸØ®Çê : ´òÖéò¯ckP±/D 唣خÇê : å”zBˆ0v/D §Ìê­– ÜÅò¶âê§Ìê­– : Boltzmann Úž2w 唧Ìê­– : 2|v唧Ìê­–w ”ŸØ®Çêt唣خÇêz•Ô Ú { ”–’Az”Rw wsY—} S 6 , S Q S ) S 6
  • 37. , S Q S FYQ +) S FYQ +6 FYQ +, S ; ; 28 / 55
  • 38. HMCz*?± Üa˜k°BFwe—*?±{’Az”Rwv—} NJO ú FYQ ) S + ) ú S ú ) ) S + ) ú S ú võ0w{| z–{Ó/vzs ’T«f *va˜— ( ) {flX|²òÓä–¾sD–0wÎÞêÇò[ äž|ŒBcrw[t«f¨1XPfe—kxö0w{@q{ »ìsvP} 29 / 55
  • 39. Leapfrog|ŒB ÎÞêÇò[ ä{w}0ww[z{ÕcPzs|D– žíR} i_s{|Leapfrog|ŒBtPR’AzÒäžpWR} SJ U
  • 40. 1 J U
  • 42. 1 SJ U + 1 6 U J J U
  • 43. 1SJ U
  • 45. 1 + 1 6 U
  • 46. 1 J 30 / 55
  • 47. vhLeapfrog|ŒBvzW Q S ) S •Ô Ú žÓ/we—kw{| z9žÓ/w cv]˜|v•vP cWc|Euler]vus{(h_zuažÏÁcrŽ)9X/Bc rcŠRzs| Q S XÓ/wv•vP Leapfrog|ŒBs{|3pzÒä{i˜j˜¹ (shear mapping)vzs|i˜j˜2JcrŽ9X/BcvP VerticalShear m=1.25 by RobHar - Own work using Inkscape. Licensed under Public domain via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:VerticalShear_m%3D1.25.svg#mediaviewer/File:VerticalShear_m%3D1.25.svg 31 / 55
  • 48. HMCw”—´òÖéò¯¡ê³é¹ß 1. |æA žE| we— 2. 唣žcL ÚvuW•´òÖéò¯e— 3. W•¸ÅÂÖ´£¹ sLeapfrog|ŒBw”—Òž x –4c| ž¤— 4. ± s*vc|iRsv] ˜|@qe— 5. *va˜kÝ{ tc|@qa˜kÝ{ te— 6. tcr|2~5ž z´òÖêX¤•˜—Šs– 4e N ± N 1 - ú NJO FYQ ) S + ) ú S ú N
  • 50. ± N N ± N
  • 51. . 32 / 55
  • 52. # U : potential energy function # ∇U : gradient of the potential energy function # θ₀ : initial state # M : number of samples # ϵ : step size # L : number of steps function hmc(U::Function, ∇U::Function, θ₀::Vector{Float64}, M::Int, ϵ::Float64, L::Int) d = length(θ₀) # allocate sampels' holder samples = Array(typeof(θ₀), M) # set the current sate to the initail state θ = θ₀ for m in 1:M # sample momentum variable p = randn(d) H = U(θ) + p ⋅ p / 2 θ̃ = θ for l in 1:L p -= ϵ / 2 * ∇U(θ̃) # half step in momentum variable θ̃ += ϵ * p # full step in location variable p -= ϵ / 2 * ∇U(θ̃) # half step in momentum variable again end H̃ = U(θ̃) + p ⋅ p / 2 if randn() min(1.0, exp(H - H̃)) # accept the proposal θ = θ̃ end samples[m] = θ print_sample(θ) end samples end hmc.jl 33 / 55
  • 53. HQ ­created with Gadfly.jl -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y HMC (ϵ = 0.1, L = 10) - 34 / 55
  • 54. HQ ­created with Gadfly.jl -2 -1 0 1 x Iteration 500 400 300 200 100 1 1 0 -1 -2 y HMC (ϵ = 0.01, L = 10) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y HMC (ϵ = 0.05, L = 10) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y HMC (ϵ = 0.1, L = 10) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y HMC (ϵ = 0.5, L = 10) - 35 / 55
  • 55. HQ ­created with Gadfly.jl -0.3 -0.2 -0.1 0.0 0.1 x Iteration 500 400 300 200 100 1 0.1 0.0 -0.1 -0.2 -0.3 y HMC (ϵ = 0.01, L = 1) -1.0 -0.5 0.0 0.5 x Iteration 500 400 300 200 100 1 0.5 0.0 -0.5 -1.0 -1.5 y HMC (ϵ = 0.05, L = 1) -2 -1 0 1 x Iteration 500 400 300 200 100 1 1 0 -1 -2 y HMC (ϵ = 0.1, L = 1) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y HMC (ϵ = 0.5, L = 1) - 36 / 55
  • 56. HQ ­created with Gadfly.jl -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y HMC (ϵ = 0.01, L = 50) -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y HMC (ϵ = 0.05, L = 50) -4 -2 0 2 4 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y HMC (ϵ = 0.1, L = 50) -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 y HMC (ϵ = 0.5, L = 50) - 37 / 55
  • 57. HMCXwEck_t §Ìê­–z¶áQžžR_ts|Iÿwvok_t: 1 ¸ÅÂÖ´£¹ žP Ÿa[t˜|Leapfrogz ¨1XŸa [v–|@qž[WT•˜— Š¨X - ¸ÅÂÖ×crÍ•Ww¤”e—k|èò¿ß¥¨ –®t‡…r[Šs”]— @qžWTpp¼z”Ÿ”–[Šs”[_tXsY—”Rwv –|¤•˜—´òÖêX”–¬€v´òÖêwßqPk} 38 / 55
  • 58. HMCzÕca HMCzqF{|唞ۏe—2pzÐèà–¾ ¸ÅÂÖ´£¹ ¸ÅÂÖD 1 - z–Xm“Ru P–wa˜rP—tPR_tw•#crP—} Úzæw”orm“Ru P–X/›—k|_˜•z–žuŸ v ÚwŽRŠ[P[”R6e—z{ÓIÿ} 39 / 55
  • 59. 1 - t zÛ^ž«T—tuRv—W 1 ¸ÅÂÖ´£¹ : XŸaeZ— ➠ Š¨XNŠ–”WvP XRYeZ— ➠ leapfrog|ŒBXìeZr@qXÖX— 1 1 1 - žŸa[e—t žRY[cvPtP]vPk|4‘²¸ÇŽW W—} - ¸ÅÂÖD : XŸaeZ— ➠ èò¿ß¥¨–®žcrcŠR XRYeZ— ➠ Š¨XÂY4e (U¾–ò) - - ➠ ß”0wÐèà–¾žÛckP 40 / 55
  • 60. No­U­Turn Sampler (NUTS) No Turning Back by Pak Gwei is licensed under CC BY-NC-SA 2.0 41 / 55
  • 61. No­U­Turn Sampler HMC{¸ÅÂÖ´£¹ 1 t¸ÅÂÖD - z2pzÐèà–¾wÑ÷l okX| No-U-Turn Sampler (NUTS)s{_˜•zÐèà–¾(§w )žRŠP_tۏcr[˜—} zۏ ➠ ´òÖéò¯¼zdual averagingw”–?2B zۏ ➠ ´òÖéò¯¯zŠ¨z}cP唞ccrµŠ— - 1 - ÑÇXsÐèà–¾zÀä–Êò¯e—_tv[?2vHMC´òÖ è–t•d[•õz P´òÖêX¤•˜—”RwvorP—} 42 / 55
  • 62. ±¿seXNUTSz¡ê³é¹ßse (Hoffman Gelman, 2014) 43 / 55
  • 63. NUTSzPF aeXwÀóžÃue—z{rcPzsPFžÃue—t| ´òÖêzP…{|0|vp—sàP[XPP vzs6 - žgf|P…žuŸuŸü|crP[ ü|ceZrå”XU¾–òž¥k•|P…žü|ezžµ— izP…wN—Æ–¾FW•|cP´òÖꞤ— P…W•z´òÖéò¯{|ÍQ–PžVavP”Rwe— ÍQ–P(detailed balance)❏ t{| ÚXÓ/wv—kzP ãL 44 / 55
  • 64. No! U­Turn!! ú P…zàazÔ/B{|¥F W•xWzF ŠszØ®Çêt 唣خÇê Sú zw‡ºe—: ú 5 E ú ú 5Sú + + + E EU ú+ 5 ú+ EU _z–X ’Awvok•|P…XU¾–òžc¥k_twv—} 45 / 55
  • 65. àPzsDwv—[{´òÖê²–Èznuts.jlžˆ¹crAaP} # L: logarithm of the joint density θ # ∇L: gradient of L # θ₀: initial state # M: number of samples # ϵ: step size function nuts(L::Function, ∇L::Function, θ₀::Vector{Float64}, M::Int, ϵ::Float64) d = length(θ₀) samples = Array(typeof(θ₀), M) θ = θ₀ for m in 1:M r₀ = randn(d) u = rand() * exp(L(θ) - r₀ ⋅ r₀ / 2) θ⁻ = θ⁺ = θ r⁻ = r⁺ = r₀ C = Set([(θ, r₀)]) j = 0 s = 1 while s == 1 v = randbool() ? -1 : 1 if v == -1 θ⁻, r⁻, _, _, C′, s′ = build_tree(L, ∇L, θ⁻, r⁻, u, v, j, ϵ) else _, _, θ⁺, r⁺, C′, s′ = build_tree(L, ∇L, θ⁺, r⁺, u, v, j, ϵ) end if s′ == 1 C = C ∪ C′ end s = s′ * ((θ⁺ - θ⁻) ⋅ r⁻ ≥ 0) * ((θ⁺ - θ⁻) ⋅ r⁺ ≥ 0) j += 1 end θ, _ = rand(C) samples[m] = θ print_sample(θ) end samples end nuts.jl 46 / 55
  • 66. HQ ­created with Gadfly.jl -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y NUTS (ϵ = 0.1) 47 / 55
  • 67. HQ ­created with Gadfly.jl -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 4 2 0 -2 -4 y NUTS (ϵ = 0.01) -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y NUTS (ϵ = 0.05) -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y NUTS (ϵ = 0.1) -3 -2 -1 0 1 2 3 x Iteration 500 400 300 200 100 1 3 2 1 0 -1 -2 -3 y NUTS (ϵ = 0.5) 48 / 55
  • 69. JuliasžT—MCMCа–· MCMC.jl - https://github.com/JuliaStats/MCMC.jl ´òÖè–zÆÐ–Ç (12$·!) ȬäàòÇXvP Stan.jl - https://github.com/goedman/Stan.jl StanÏ£òÆ¢ò¯ (via CmdStan) izRmMCMC.jlw–Š˜—o‰P? Mamba.jl - https://github.com/brian-j-smith/Mamba.jl Wv–˜Do‰PJuliakzöJ0vMCMCÔë–ßî–® OözȬäàòÇ 50 / 55
  • 70. Št HMC{Š¨z唞ð…cr´òÖéò¯e—_tw”–|@q žA^•˜— NUTS{HMCzÕcPÐèà–¾Ûž|ß”Bcr[˜— Mamba.jlXJuliazöJ0v´òÖè–z¶ö×W 51 / 55
  • 71. ˆæ Bishop, C. M. (2006). Pattern recognition and machine learning. New York: springer. (pJÙ (2012) ´òÖéò¯] о–òéå tA†»? A, pp.237-273. ½ch) Hoffman, M. D., Gelman, A. (2014). The No-U-Turn Sampler : Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15, 1351–1381. MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge University Press. Neal, R. M. (2011). MCMC Using Hamiltonian Dynamics. In Handbook of Markov Chain Monte Carlo, pp.113-162. Chapman Hall/CRC. iJ;. (2008). Ýê²Ô×7áòŪêì] ÒÞ„? 52 / 55
  • 72. VŠ] 53 / 55
  • 73. Juliaz¼–¸²–ÈtͲ–Èz£àv·Í function build_tree(L::Function, ∇L::Function, θ::Vector{Float64}, r::Vector{Float64}, u::Float64, v::Int if j == 0 θ′, r′ = leapfrog(∇L, θ, r, v * ϵ) C′ = u ≤ exp(L(θ′) - r′ ⋅ r′ / 2) ? Set([(θ′, r′)]) : Set([]) s′ = int(L(θ′) - r′ ⋅ r′ / 2 log(u) - Δmax) return θ′, r′, θ′, r′, C′, s′ else θ⁻, r⁻, θ⁺, r⁺, C′, s′ = build_tree(L, ∇L, θ, r, u, v, j - 1, ϵ) if v == -1 θ⁻, r⁻, _, _, C″, s″ = build_tree(L, ∇L, θ⁻, r⁻, u, v, j - 1, ϵ) else _, _, θ⁺, r⁺, C″, s″ = build_tree(L, ∇L, θ⁺, r⁺, u, v, j - 1, ϵ) end s′ = s′ * s″ * ((θ⁺ - θ⁻) ⋅ r⁻ ≥ 0) * ((θ⁺ - θ⁻) ⋅ r⁺ ≥ 0) C′ = C′ ∪ C″ return θ⁻, r⁻, θ⁺, r⁺, C′, s′ end end nuts.jl 54 / 55
  • 75. YJ EU QJ EU #J ) ) ) ) #J + QJ YJ QJ YJ 55 / 55