Recommended
Recommended
More Related Content
Similar to ICCV 2019 REVIEW [CDM]
Similar to ICCV 2019 REVIEW [CDM] (20)
More from Dongmin Choi
More from Dongmin Choi (20)
Recently uploaded
Recently uploaded (20)
ICCV 2019 REVIEW [CDM]
- 1. Yonsei University Severance Hospital CCIDS ICCV 2019 Review Dongmin Choi
- 2. ICCV - ICCV : International Conference on Computer Vision - 2019년 국내에서 첫 개최 : 10.27 ~ 11.2, COEX - 2019 ICCV : 약 7,500명 참가 총 1,077편의 논문 Yonsei University Severance Hospital CCIDS
- 3. ICCV ICCV 2019 Summary 1. Visual Recognition for Medical Images 2. Image and Video Synthesis: How, Why and What if? 3. Interpretating and Explaining Visual AI Models 4. SinGAN: Learning a Generative Model from a Single Natural Images (Best Paper) Yonsei University Severance Hospital CCIDS
- 4. Yonsei University Severance Hospital CCIDS Visual Recognition for Medical Images https://sites.google.com/view/iccv19-vrmi
- 5. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Deep Learning in Neuroimaging and Radiotherapy Dinggang Shen University of North Carolina at Chapel Hill
- 6. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Registration Template T Subject S Deep-learning based Registration J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
- 7. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Dissimilarity Loss LD(T, S') S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
- 8. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric (inspired by GAN) Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Discriminator S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration” Generator
- 9. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS On Unseen Testing Dataset J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
- 10. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Registration Result J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
- 11. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Synthesis MRI CT Can we synthesize CT image from a single MRI Image? CT images are highly desired for - Specific diagnosis - Dose planning - PET attenuation correction D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks”
- 12. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Use context features to iteratively refine the training results
- 13. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Refinements Difference maps
- 14. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Q&A Q. Whole Image가 아니라 Patch를 사용하는 이유? A. 데이터 수가 딥러닝을 학습하기에는 부족하기 때문이다. Patch를 사용하면 훨씬 많은 학습 데이터를 확보할 수 있기 때문에 patch를 사용한다. 물론 데이터 수가 충분히 많다면 patch가 아니라 whole image를 사용하는 것이 더욱 좋다.
- 15. Yonsei University Severance Hospital CCIDS Image and Video Synthesis: How, Why and What if https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn
- 16. Image and Video Synthesis: How, Why and What if? Yonsei University Severance Hospital CCIDS Seeing What a GAN Cannot Generate David Bau MIT http://ganseeing.csail.mit.edu//
- 17. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS https://software.intel.com/en-us/blogs/2017/08/21/mode-collapse-in-gans Model Collapse in GAN is serious Problem ※ Model Collapse : A problem when all the generator outputs are identical (all of them or most of the samples are equal)
- 18. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu//
- 19. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images Inception Inception Fake Inception Feature Space Real Inception Feature Space http://ganseeing.csail.mit.edu// FID (Frechet Inception Distance) Measuring GAN Quality
- 20. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu// 1. What is actually missing in the distribution? 2. What is actually missing in each image?
- 21. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
- 22. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
- 23. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z)
- 24. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z) Pairs (x, G(z*)) reveals omissions Objective : z* = argminz Loss(x, G(z))
- 25. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// Two steps to invert a large generator
- 26. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// x = G(z) x = G(z*) x = real x = G(z*) Generated Reconstruction Real Photo Reconstruction When G generates x, reconstruction is precise When reconstruction is imperfect we know G cannot generate x
- 27. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// GANs don’t like people
- 28. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image
- 29. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Optimized z
- 30. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Adapted Cheese
- 31. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Real Image x
- 32. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*) z* = argminz Loss(x, G(z))
- 33. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*, θ*) θ* z*, θ* = argminz,θ Loss(x, G(z)) + R(θ) Regularizer Inspired by Deep Image Prior [Ulyanove et al, 2018]
- 34. Yonsei University Severance Hospital CCIDS Interpretating and Explaining Visual AI Models http://xai.unist.ac.kr/workshop/2019/
- 35. Interpretating and Explaining Visual AI Models Yonsei University Severance Hospital CCIDS Recent progress towards XAI at UC Berkeley Trevor Darrel UC Berkeley
- 36. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS
- 37. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Explainable NN
- 38. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Salience for Introspection Image LIME GradCAM RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
- 39. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Overview of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
- 40. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Pros of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models Cons of RISE A more general framework as the importance map is obtained with access to only the input and output of the base model Time and resource complexity
- 41. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
- 42. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
- 43. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
- 44. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Unmasking Clever Hans Predictors S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Images with a copyright watermark (a, c) are classified to “horse” class, but images without an added copyright watermark (b, d) aren’t.
- 45. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Understanding Learning Behaviour S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Model learns 1. Track the ball 2. Focus on paddle 3. Focus in the tunnel
- 46. Yonsei University Severance Hospital CCIDS Learning a Generative Model from a Single Natural Image https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn SinGAN Tamar Rott Shaham Google AI ICCV 2019 Best Paper
- 47. SinGAN Yonsei University Severance Hospital CCIDS
- 48. SinGAN Yonsei University Severance Hospital CCIDS
- 49. SinGAN Yonsei University Severance Hospital CCIDS
- 50. SinGAN Yonsei University Severance Hospital CCIDS ▶ Same model ▶ No extra training SinGAN for single image animation https://www.youtube.com/watch?v=xk8bWLZk4DU&feature=youtu.be
- 51. SinGAN Yonsei University Severance Hospital CCIDS Model
- 52. SinGAN Yonsei University Severance Hospital CCIDS Summary - SinGAN : Single Image GAN - Controllable Generation - Image Manipulation Applications
- 53. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image
- 54. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image Generated High-Resolution Image
- 55. Thank You