What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. A team from TNG Technology Consulting managed to build an AI showcase using OpenCV and Tensorflow to realize such goggles.

1. Stereoscopic Style Transfer AIStereoscopic Style Transfer AI
Art is not what you see?Art is not what you see?

2. Edgar DegasEdgar Degas
@Mar nFoertsch @originalone1984

3. What is this all about?What is this all about?
@Mar nFoertsch @originalone1984

5. Hardware Hacking TeamHardware Hacking Team
TNG Technology Consulting GmbHTNG Technology Consulting GmbH
Jonas Mayer Thomas Endres Martin Förtsch Thomas Reifenberger Florian Gather Markus Spanier
Christoph Bergemann Holger Frydrich Mathias Arens Andreas Würl Mirco Michel Leopold Kellers

6. AgendaAgenda
Introduc on
Deep Learning in a Nutshell
Style Transfer
Code & Technology

7. RetrospectionRetrospection
Augmented Rift (2015)Augmented Rift (2015)
@Mar nFoertsch @originalone1984

8. RetrospectionRetrospection
Augmented Rift (2015)Augmented Rift (2015)
@Mar nFoertsch @originalone1984

9. IntroductionIntroduction
Style TransferStyle Transfer
@Mar nFoertsch @originalone1984
1. Von Vincent van Gogh - bgEuwDxel93-Pg at Google Cultural Institute, zoom level maximum, Gemeinfrei, https://commons.wikimedia.org/w/index.php?curid=25498286
2. Photo by Martin Förtsch, "Window to the World", Mesa Arch, Canyonlands, 2015

10. IntroductionIntroduction
Style TransferStyle Transfer
@Mar nFoertsch @originalone1984

11. IntroductionIntroduction
Style TransferStyle Transfer
@Mar nFoertsch @originalone1984
1. Edvard Munch - WebMuseum at ibiblioPage URL: http://www.ibiblio.org/wm/paint/auth/munch/munch.scream.jpg, Gemeinfrei, https://commons.wikimedia.org/w/index.php?curid=37610298

12. IntroductionIntroduction
Style TransferStyle Transfer
@Mar nFoertsch @originalone1984
1. Pablo Picasso, Frau mit grünem Hut, https://www.albertina.at/site/assets/files/1456/9_pablo_picasso_-_frau_mit_gruenem_hut.720x0.jpg

13. IntroductionIntroduction
DeepArt.ioDeepArt.io
@Mar nFoertsch @originalone1984

14. IntroductionIntroduction
How Paintings workHow Paintings work
@Mar nFoertsch @originalone1984

15. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984
1.
2.
By BruceBlaus - Own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=28761830
By Original by en:User:Chris 73, updated by en:User:Diberri, converted to SVG by tiZom - Own work, CC BY-SA 3.0

16. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

17. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

18. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

19. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

20. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

21. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

22. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

23. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

24. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

25. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984
1. By JacoTen [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], from Wikimedia Commons

26. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984
1. By JacoTen [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], from Wikimedia Commons

27. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984
1. https://medium.com/@arvind_70185/gradient-descent-for-machine-learning-52ce08c96296

28. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

29. IntroductionIntroduction
Deep Learning in a NutshellDeep Learning in a Nutshell
@Mar nFoertsch @originalone1984

30. Style TransferStyle Transfer
Offline Role Play Game (Content Image)Offline Role Play Game (Content Image)
@Mar nFoertsch @originalone1984
1. Von A.Savin (Wikimedia Commons · WikiPhotoSpace) - Eigenes Werk, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=21174383

31. Style TransferStyle Transfer
Offline Role Play Game (Style Image)Offline Role Play Game (Style Image)
@Mar nFoertsch @originalone1984

32. Technical ImplementationTechnical Implementation
@Mar nFoertsch @originalone1984

33. Technical ImplementationTechnical Implementation
Style Transfer WorkflowStyle Transfer Workflow
@Mar nFoertsch @originalone1984

34. Technical ImplementationTechnical Implementation
Style Transfer Workflow by Gatys et al. (2015)Style Transfer Workflow by Gatys et al. (2015)
@Mar nFoertsch @originalone1984

35. Technical ImplementationTechnical Implementation
VGG19 WorkflowVGG19 Workflow
@Mar nFoertsch @originalone1984

36. Technical ImplementationTechnical Implementation
VGG19 - Live Demo!VGG19 - Live Demo!
@Mar nFoertsch @originalone1984

37. Technical ImplementationTechnical Implementation
Calculating Content Loss with VGG19Calculating Content Loss with VGG19
@Mar nFoertsch @originalone1984

38. Technical ImplementationTechnical Implementation
Calculating Content Loss with VGG19Calculating Content Loss with VGG19
@Mar nFoertsch @originalone1984

39. Technical ImplementationTechnical Implementation
Calculating Style Loss with VGG19Calculating Style Loss with VGG19
@Mar nFoertsch @originalone1984

40. Technical ImplementationTechnical Implementation
Problems are Time Consumption and NondeterminismProblems are Time Consumption and Nondeterminism
@Mar nFoertsch @originalone1984

41. Technical ImplementationTechnical Implementation
Fast Style Transfer Workflow by Ulyanov et al. (2016)Fast Style Transfer Workflow by Ulyanov et al. (2016)
@Mar nFoertsch @originalone1984

42. Code & TechnologyCode & Technology
Architecture of the CNNArchitecture of the CNN
@Mar nFoertsch @originalone1984

43. Code & TechnologyCode & Technology
Temporal Style Transfer for VideosTemporal Style Transfer for Videos
@Mar nFoertsch @originalone1984
1. https://www.youtube.com/watch?v=pW6nZXeWlGM

44. Training DatasetsTraining Datasets
COCO Common Objects in ContextCOCO Common Objects in Context
@Mar nFoertsch @originalone1984

45. Demonstration TimeDemonstration Time
Realtime Style TransferRealtime Style Transfer
@Mar nFoertsch @originalone1984
1. https://medium.com/data-science-group-iitr/artistic-style-transfer-with-convolutional-neural-network-7ce2476039fd

46. Code & TechnologyCode & Technology
Dell VisorDell Visor
@Mar nFoertsch @originalone1984

47. Code & TechnologyCode & Technology
Architecture OverviewArchitecture Overview
@Mar nFoertsch @originalone1984

48. Code & TechnologyCode & Technology
TrainingTraining
@Mar nFoertsch @originalone1984

49. Experiment TimeExperiment Time
Let's do something crazy!Let's do something crazy!
@Mar nFoertsch @originalone1984

50. Experiment TimeExperiment Time
Let's do something crazy!Let's do something crazy!
@Mar nFoertsch @originalone1984

51. Experiment TimeExperiment Time
Let's do something crazy!Let's do something crazy!
@Mar nFoertsch @originalone1984