ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines
•
1 like•1,212 views
The presentation material for the reading club of Element of Statistical Learning by Hastie et al. The contents of the sections cover - Graphical Lasso (fitting for Gaussian graphical model) - Derivation and Learning of Boltzmann Machines - Restricted Boltzmann Machines and Contrastive Divergence Methods ------------------------------------------------------------------------- 研究室での『統計学習の基礎』（Hastieら著）の輪講用発表資料（ぜんぶ英語）です。 担当範囲は ・グラフィカルLASSO（ガウスグラフィカルモデルへの当てはめ） ・ボルツマンマシンの導出と学習について ・制限ボルツマンマシンとコントラスティブダイバージェンス法
Recommended
More Related Content
Viewers also liked
Recently uploaded
Recently uploaded (20)
ESL 17.3.2-17.4: Graphical Lasso and Boltzmann Machines
- 1. ESL17.3.2-17.4 GraphicalLasso &BoltzmannMachines June 8, 2015 Talk by Shinichi TAMURA Mathematical Informatics Lab @ NAIST
- 5. [Review]Propertiesofgraphicalmodels FittingGaussianGraphicalModels 1 2 34 5 ・ ・ 0 0 ・ ・ ・ ・ 0 0 0 ・ ・ ・ 0 0 0 ・ ・ ・ ・ 0 0 ・ ・
- 14. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 WΘ Γ 0 0 0 0 0 0 0 0 0 0 ? 1 2 34 5 ?
- 15. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 WΘ Γ 0 0 0 0 0 0 0 0 0 0 ? 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 34 5
- 16. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk WΘ Γ 0 0 0 0 0 0 0 0 0 0 s11 s12 s15 s21 s22 s23 s32 s33 s34 s43 s44 s45 s51 s54 s55 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 34 5
- 17. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk WΘ Γ ? ? 0 0 ? ? ? ? 0 0 0 ? ? ? 0 0 0 ? ? ? ? 0 0 ? ? s11 s12 ? ? s15 s21 s22 s23 ? ? ? s32 s33 s34 ? ? ? s43 s44 s45 s51 ? ? s54 s55 0 0 ? ? 0 0 0 0 ? ? ? 0 0 0 ? ? ? 0 0 0 0 ? ? 0 0 1 2 34 5
- 18. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk θjk=0 wjk=0 WΘ Γ ? ? 0 0 ? ? ? ? 0 0 0 ? ? ? 0 0 0 ? ? ? ? 0 0 ? ? s11 s12 ? ? s15 s21 s22 s23 ? ? ? s32 s33 s34 ? ? ? s43 s44 s45 s51 ? ? s54 s55 0 0 ? ? 0 0 0 0 ? ? ? 0 0 0 ? ? ? 0 0 0 0 ? ? 0 0 1 2 34 5
- 19. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk θjk=0 wjk=0 WΘ Γ ? ? 0 0 ? ? ? ? 0 0 0 ? ? ? 0 0 0 ? ? ? ? 0 0 ? ? s11 s12 ? ? s15 s21 s22 s23 ? ? ? s32 s33 s34 ? ? ? s43 s44 s45 s51 ? ? s54 s55 0 0 ? ? 0 0 0 0 ? ? ? 0 0 0 ? ? ? 0 0 0 0 ? ? 0 0 1 2 34 5
- 20. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk θjk=0 wjk=0 WΘ Γ ? ? 0 0 ? ? ? ? 0 0 0 ? ? ? 0 0 0 ? ? ? ? 0 0 ? ? s11 s12 ? ? s15 s21 s22 s23 ? ? ? s32 s33 s34 ? ? ? s43 s44 s45 s51 ? ? s54 s55 0 0 ? ? 0 0 0 0 ? ? ? 0 0 0 ? ? ? 0 0 0 0 ? ? 0 0 1 2 34 5
- 21. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W−S−Γ=0. θjk=0 γjk=0 wjk=sjk θjk=0 wjk=0 WΘ Γ ? ? 0 0 ? ? ? ? 0 0 0 ? ? ? 0 0 0 ? ? ? ? 0 0 ? ? s11 s12 ? ? s15 s21 s22 s23 ? ? ? s32 s33 s34 ? ? ? s43 s44 s45 s51 ? ? s54 s55 0 0 ? ? 0 0 0 0 ? ? ? 0 0 0 ? ? ? 0 0 0 0 ? ? 0 0 1 2 34 5
- 23. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM w12−s12−γ12=0 WΘ Γ Θ11 θ12 θT 12 θ22 W11 w12 wT 12 w22 Γ11 γ12 γT 12 γ22 1 2 34 5
- 24. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM w12−s12−γ12=0 w12−s12−γ12=0 W11β−s12−γ12=0 (β=θ12/θ22) W* 11β*−s* 12=0 WΘ Γ Θ11 θ12 θT 12 θ22 W11 w12 wT 12 w22 Γ11 γ12 γT 12 γ22 1 2 34 5
- 32. [Review]Propertiesofgraphicalmodels Block-wiseAlgorithmforGGM W=S+λI W* 11β*−s* 12=0 θ12=−βθ22 Θ=W-1
- 52. GraphicalLasso Sub-derivative f(x)=|x| {-1} x<0 [-1,1] x=0 {1} x>0 “Sign” “sign” = -1 (θ<0) Sign(θ) ∈ [-1,1] (θ=0) = 1 (θ>0){
- 53. GraphicalLasso Sub-derivative f(x)=|x| {-1} x<0 [-1,1] x=0 {1} x>0 “Sign” “sign” = -1 (θ<0) Sign(θ) ∈ [-1,1] (θ=0) = 1 (θ>0) Sign(0) sign(0)=0 {
- 62. GraphicalLasso Block-wisealgorithm w12–s12–λ∙Sign(θ12)=0 θ12=–θ22W-1 11w12 and θ22>0 w12–s12+λ∙Sign(W-1 11w12)=0 W11 w12 wT 12 w22
- 117. GraphicalLasso Settingofλ λ y 0 λ=0 λ=7 λ=14 λ=27 λ=36
- 122. Today'stopics
- 123. Today'stopics
- 176. Today'stopics
- 177. Today'stopics
- 178. Today'stopics