Introduction to Generative Adversarial Networks

1. Introduction to GAN
서울대학교 방사선의학물리연구실
이 지 민 ( ljm861@gmail.com )

2. 참고 자료 출처 (본 슬라이드 인용 순)
2
좋은 자료를 만들어주신 많은 분들께 다시 한 번 감사의 인사를 전하고 싶고,
슬라이드 좌측 하단에 출처를 명시하였으니, 꼭 찾아보시길 바랍니다. 
0.
•
•
•
•
•
•
•

3. Contents
3
1. Generative Model
2. Auto-Regressive Models
3. Variational Auto-Encoder
4. Generative Adversarial Networks
5. Significant Variants & Applications

4. Generative Model
4

5. Generative Model
5
1.

6. Generative Model
6
1.

7. Generative Model
7
1.

8. Generative Model
8
1.

9. Generative Model
9
1.

10. Generative Model
10
Ideal Generative Model
1.
CAT
Short
Hair
Big
Ear
Model

11. Generative Model
11
Ideal Generative Model
1.
CAT
Short
Hair
Big
Ear
Model

12. Generative Model
12
Ideal Generative Model
1.
CAT
Short
Hair
Big
Ear
Tabby
Model

13. Generative Model
13
Ideal Generative Model
1.
CAT
Short
Hair
Big
Ear
Tabby
Savannah Cat
Model

14. Generative Model
14
1.

15. Generative Model
15
1.
https://www.slideshare.net/carpedm20/pycon-korea-2016 (지적 대화를 위한 깊고 넓은 딥러닝, 김태훈)

16. Generative Model
16
1.
https://www.slideshare.net/carpedm20/pycon-korea-2016 (지적 대화를 위한 깊고 넓은 딥러닝, 김태훈)

17. Generative Model
17
1.
https://www.slideshare.net/carpedm20/pycon-korea-2016 (지적 대화를 위한 깊고 넓은 딥러닝, 김태훈)

18. Generative Model
18
1.
https://www.slideshare.net/carpedm20/pycon-korea-2016 (지적 대화를 위한 깊고 넓은 딥러닝, 김태훈)

19. Generative Model
19
1.
https://www.slideshare.net/carpedm20/pycon-korea-2016 (지적 대화를 위한 깊고 넓은 딥러닝, 김태훈)

20. Generative Model
20
1.
https://blog.openai.com/generative-models/

21. Generative Model
21
Why Generative Model
1.
•
•
•
•
•
•
Li, Yijun, et al., Generative face completion, 2017

22. Generative Model
22
Deep Generative Models
1.
 Auto-Regressive Models
 Variational Auto-Encoder
 Generative Adversarial Networks

23. Auto-Regressive Models
23

24. Auto-Regressive Models
24
Pixel-by-pixel generation
2.
http://slazebni.cs.illinois.edu/spring17/lec13_advanced.pdf

25. Auto-Regressive Models
25
Multi-Dimensional RNNs (2013)
2.
Graves et al, Multi-Dimensional Recurrent Neural Networks, 2013

26. Auto-Regressive Models
26
Spatial LSTM (2015)
2.
Theis et al., Generative Image Modeling Using Spatial LSTMs, 2015

27. Auto-Regressive Models
27
Pixel RNN (2016)
2.
Aaron et al, Pixel Recurrent Neural Networks, 2016

28. Auto-Regressive Models
28
Sampling
2.
•
➔
•
➔
•
➔
•
•

29. Auto-Regressive Models
29
Sampling
2.

30. Auto-Regressive Models
30
Sampling
2.

31. Auto-Regressive Models
31
Sampling
2.

32. Auto-Regressive Models
32
Sampling
2.

33. Auto-Regressive Models
33
Pixel RNN (2016)
2.
Aaron et al, Pixel Recurrent Neural Networks, 2016

34. Auto-Regressive Models
34
Features
2.
•
•
•
•

35. Variational Auto-Encoder
35

36. Variational Auto-Encoder
36
Auto-Encoder
3.
http://kvfrans.com/variational-autoencoders-explained/

37. Variational Auto-Encoder
37
Auto-Encoder
3.
http://kvfrans.com/variational-autoencoders-explained/
?

38. Variational Auto-Encoder
38
Variational Auto-Encoder
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf

39. Variational Auto-Encoder
39
Variational Auto-Encoder
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

40. Variational Auto-Encoder
40
Variational Auto-Encoder
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

41. Variational Auto-Encoder
41
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

42. Variational Auto-Encoder
42
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

43. Variational Auto-Encoder
43
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

44. Variational Auto-Encoder
44
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

45. Variational Auto-Encoder
45
3.
http://kvfrans.com/variational-autoencoders-explained/, https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

46. Variational Auto-Encoder
46
Kullback-Leibler Divergence
3.
https://en.wikipedia.org/wiki/Kullback–Leibler_divergence

47. Variational Auto-Encoder
47
Loss function
3.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/08_Autoencoder/%5B2%EA%B8%B0%5DAutoEncoder.pdf (최건호)

48. Variational Auto-Encoder
48
Reparameterization Trick
3.
Carl Doersch, Tutorial on Variational Autoencoders, 2016

49. Variational Auto-Encoder
49
Results
3.
Kingma et al., Auto-Encoding Variational Bayes, 2014

50. Variational Auto-Encoder
50
Results
3.
https://www.slideshare.net/HyungjooCho2/deep-generative-modelpdf (Deep Generative Models, 조형주)

51. Variational Auto-Encoder
51
Features
3.
•
•
•
•

52. Generative Adversarial Networks
52

53. Generative Adversarial Networks
53
4.

54. Generative Adversarial Networks
54
4.
생성 모델

55. Generative Adversarial Networks
55
4.
적대적 학습

56. Generative Adversarial Networks
56
4.
적대적 학습
Generator vs Discriminator

57. Generative Adversarial Networks
57
4.
https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f

58. Generative Adversarial Networks
58
4.
https://www.slideshare.net/ssuser77ee21/generative-adversarial-networks-70896091 (Generative Adversarial Networks, 김남주)

59. Generative Adversarial Networks
59
Value Function
4.

60. Generative Adversarial Networks
60
Value Function
4.
https://www.slideshare.net/NaverEngineering/1-gangenerative-adversarial-network (1시간 만에 GAN 완전 정복하기, 최윤제)

61. Generative Adversarial Networks
61
Value Function
4.
https://www.slideshare.net/NaverEngineering/1-gangenerative-adversarial-network (1시간 만에 GAN 완전 정복하기, 최윤제)

62. Generative Adversarial Networks
62
4.
Goodfellow et al, Generative Adversarial Networks, 2014

63. Generative Adversarial Networks
63
Results
4.
Goodfellow et al, Generative Adversarial Networks, 2014

64. Generative Adversarial Networks
64
Features
4.
•
•
•
•

65. Generative Adversarial Networks
65
Comparison with Auto-regressive models and VAE
4.
http://slazebni.cs.illinois.edu/spring17/lec13_advanced.pdf, https://openai.com/blog/generative-models/

66. Generative Adversarial Networks
66
DCGAN (Deep Convolutional GAN)
4.
Radford et al, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 2015

67. Generative Adversarial Networks
67
DCGAN (Deep Convolutional GAN)
4.
Radford et al, Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 2015 / https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6

68. Generative Adversarial Networks
68
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

69. Generative Adversarial Networks
69
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

70. Generative Adversarial Networks
70
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

71. Generative Adversarial Networks
71
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

72. Generative Adversarial Networks
72
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

73. Generative Adversarial Networks
73
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

74. Generative Adversarial Networks
74
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

75. Generative Adversarial Networks
75
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

76. Generative Adversarial Networks
76
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

77. Generative Adversarial Networks
77
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

78. Generative Adversarial Networks
78
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

79. Generative Adversarial Networks
79
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

80. Generative Adversarial Networks
80
Pytorch Implementation
4.
https://github.com/GunhoChoi/PyTorch-FastCampus/blob/master/09_GAN/1_DCGAN/DCGAN.ipynb (최건호)

81. Generative Adversarial Networks
81
Issues during Training
4.
•
•
•
•
•

82. Generative Adversarial Networks
82
Mode collapsing / Oscillating
Metz et al, Unrolled Generative Adversarial Networks, 2016
4.

83. Generative Adversarial Networks
83
Mode collapsing / Oscillating
4.
https://www.slideshare.net/HyungjooCho2/deep-generative-modelpdf (Deep Generative Models, 조형주)

84. Generative Adversarial Networks
84
Mode collapsing / Oscillating
4.
https://www.slideshare.net/HyungjooCho2/deep-generative-modelpdf (Deep Generative Models, 조형주)

85. Generative Adversarial Networks
85
Mode collapsing / Oscillating
4.
https://www.slideshare.net/HyungjooCho2/deep-generative-modelpdf (Deep Generative Models, 조형주)

86. Generative Adversarial Networks
86
Mode collapsing / Oscillating
4.
Target MLE (=KL) JS Reverse KL

87. Generative Adversarial Networks
87
Mode collapsing / Oscillating
4.
Metz et al, Unrolled Generative Adversarial Networks, 2016

88. Generative Adversarial Networks
88
Intractable loss
4.
https://www.slideshare.net/ssuser7e10e4/wasserstein-gan-i (Wasserstein GAN 수학 이해하기, 임성빈)

89. Generative Adversarial Networks
89
Intractable loss
4.
https://www.slideshare.net/ssuser7e10e4/wasserstein-gan-i (Wasserstein GAN 수학 이해하기, 임성빈)

90. Generative Adversarial Networks
90
Intractable loss
4.
https://www.slideshare.net/ssuser7e10e4/wasserstein-gan-i (Wasserstein GAN 수학 이해하기, 임성빈)

91. Generative Adversarial Networks
91
Intractable loss
4.
Vanilla GAN
LSGAN WGAN

92. Generative Adversarial Networks
92
Intractable loss
4.
Arjovsky et al, Wasserstein Generative Adversarial Networks, 2017

93. Generative Adversarial Networks
93
Intractable loss
4.
Arjovsky et al, Wasserstein Generative Adversarial Networks, 2017

94. Generative Adversarial Networks
94
Balance between Generator & Discriminator
4.
Berthelot et al, BEGAN, 2017

95. Generative Adversarial Networks
95
Manipulation
4.
Mirza et al, Conditional Generative Adversarial Networks, 2014

96. Generative Adversarial Networks
96
Quality
4.
Karras et al, Progressive Growing of GANs For Improved Quality, Stability, and Variation, 2017

97. Generative Adversarial Networks
97
Quality
4.
https://www.youtube.com/watch?v=XOxxPcy5Gr4

98. Generative Adversarial Networks
98
Quality
4.
Karras et al, Progressive Growing of GANs For Improved Quality, Stability, and Variation, 2017

99. Significant Variants & Applications
99

100. Significant Variants
100
Info GAN
5.
Chen et al, InfoGAN, 2017

101. Significant Variants
101
Pix2Pix
5.
Isola et al, Image-to-image translation with conditional GAN, 2016

102. Significant Variants
102
Domain Cross GAN
5.
Taigman et al, Unsupervised Cross-Domain Image Generation, 2016

103. Significant Variants
103
CycleGAN
5.
Zhu et al, Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, 2017

104. Significant Variants
104
CycleGAN
5.
Zhu et al, Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, 2017

105. Significant Variants
105
DiscoGAN
5.
Kim et al, Learning to Discover Cross Domain Relations with Generative Adversarial Networks, 2017

106. Applications
106
Time Series Generation (InfoGAN)
5.
https://github.com/buriburisuri/timeseries_gan (김남주)

107. Applications
107
ehrGAN
5.
Che et al., Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records, 2017

108. Applications
108
RCGAN
5.
Che et al., Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records, 2017

109. Applications
109
GAN for Low-dose CT
5.
Jelmer M. et al, Generative Adversarial Networks for Noise Reduction in Low-Dose CT, 2017, http://medicine.utah.edu/radiology/news/2016/low-dose-ct-zeng-award.php

110. Applications
110
GAN for Low-dose CT
5.
Jelmer M. et al, Generative Adversarial Networks for Noise Reduction in Low-Dose CT, 2017

111. Applications
111
Stain Style Transfer
5.
H Cho et al, Neural Stain-Style Transfer Learning using GAN for Histopathological Images, 2017

112. Applications
112
Simulated & Unsupervised Learning
5.
Shrivastava et al, Learning from Simulated and Unsupervised Images through Adversarial Training, 2016

113. Applications
113
AnoGAN
5.
Schlegl et al, Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker discovery, 2017

114. Q & A
114

115. 감사합니다.
115