+ Updated version for NEXT Conference Hamburg + https://nextconf.eu/event/how-deep-learning-is-changing-the-design-process/ Deep learning is a new and exciting subfield of machine learning which attempts to sidestep the whole feature design process. This session explains how it derives from AI, why it quietly became a part of user experience and how it also changes the actual design workflow. The talk highlights a range of use cases and doesn’t forget to illustrate why user experience design for artificial intelligence matters the other way around.

1. ALEXANDER MEINHARDT
@KUTTIN3DGE
KRUNCHTIME.ORG

2. © Lawrence Lek
HOW DEEP LEARNING
IS CHANGING THE DESIGN
PROCESS

3. WHAT IS
DEEP
LEARNING?
© Lawrence Lek© Lawrence Lek

4. Artificial
Intelligence
Machine
Learning
Deep
Learning

5. Artificial
Intelligence
Machine
Learning
Deep
Learning
Early artificial intelligence
stirs exitement
Machine learning begins
to flourish.
Deep learning breaktroughs
drive AI boom.
1950’s 1950’s 1970’s 1980’s 1990’s 2000’s 2010’s

6. THE TERMS ARE
OFTEN USED
INTERCHANGEABLY –
YET THEY AREN’T
THE SAME

7. Machine Learning
Machine learning is the ability
to learn without being explicitly
programmed.
It helps programs to learn from their own
mistakes and improve their performance over
time.

8. Machine Learning
Data mining
Image recognition
Facebook newsfeed
Netflix suggestions
Big Data analysis
Amazon suggestions
Predictive analytics

9. Machine Learning
Notifications
Predictive Keyboards
Spam/Quality Control Systems
Copywriting
Design Patterns
Gesture Recognition
Content Actions/
Feedback
Magic Numbers
Search
Related Content
Character Recognition

10. Machine Learning
Notifications
Predictive Keyboards
Spam/Quality Control Systems
Copywriting
Design Patterns
Gesture Recognition
Content Actions/
Feedback
Magic Numbers
Search
Related Content
Character Recognition

11. Artificial IntelligenceAI landscape
© Narrative Science

12. © Narrative Science

13. Artificial IntelligenceAI landscape
© cloud-nqb.com

14. Artificial IntelligenceAI landscape
© Lux Research

15. © Matthew Mayo
Machine Learning
Artifical Intelligence
Data Mining
Data Science
Deep
Learning
Big Data

16. Deep Learning
Deep Learning is based on
Artificial Neural Networks.
These are systems that are designed to
process information in ways that are similar to
the ways biological brains work. It uses a
certain set of Machine Learning algorithms
that run in multiple layers.

17. Artificial IntelligenceMachine Learning vs Deep Learning
© Rasmus Rothe

18. AI HAS BY NOW SUCCEEDED
IN DOING ESSENTIALLY
EVERYTHING THAT REQUIRES
‘THINKING’
Donald Knuth, Stanford University

19. BUT HAS FAILED TO DO
MOST OF WHAT PEOPLE DO
‘WITHOUT THINKING’
Donald Knuth, Stanford University

20. Possible
Desirable
Profitable
A user experience
powered by
a system that
learns
© Fabien Girardin

23. TensorFlow / Lite Caffe2Go Machine Learning Kit

24. © Lawrence Lek
WHAT
CAN BE
DONE
TODAY?
© Lawrence Lek

25. © Frank Chen

26. © Frank Chen

27. © Frank Chen

28. © Frank Chen

29. © Frank Chen

30. Image recognition error rate

31. COMPUTER VISION
USE CASES

32. Google Lens
Can an algorithm see what you see
and help you take action based on
this information?

33. Google Lens

34. Google Lens

35. Google Search

39. Blippr

40. Photoeditor SDK
Can an algorithm extract foreground
content from its background?

41. Photoeditor SDK

42. Photoeditor SDK

43. Adobe - Deep Image Matting

44. Adobe - Deep Image Matting

45. Adobe - Deep Image Matting

46. Adobe - Deep Image Matting

47. Google Research
Can an algorithm remove
watermarks from stock photos?

49. Google Research

50. Google Research

51. Google Research

52. Google Research
Can an algorithm understand what
makes good photography?

53. Google Research

54. Google Research

55. Google Research

56. Google Research

57. EyeEm - Vision
Can an algorithm learn and suggest
aesthetics in a way an art director
would?

58. EyeEm - VisionEyeEm - Vision

59. Neural Storyteller
Can an algorithm make up poetry
from a given image?

60. Neural Storyteller

61. Neural Storyteller

62. Neural Storyteller

63. Rapto
Can an algorithm make up lyrics
from any given object?

64. HOW DEEP LEARNING CHANGES THE DESIGN
PROCESS
Rapto

65. FACE RECOGNITION
USE CASES

66. Google Allo
Can an algorithm create an avatar
for you on the fly?

67. Google Allo

68. Google Allo

69. Adobe Sensei
Can an algorithm optimize a selfie
on the fly?

70. Adobe Selfie App

71. Adobe Sensei
Content intelligence
Face Aware Editing
Visual Stock Search
Auto Lip Sync
Semantic Segmentation
Face-Aware Liquify

72. Apple - Machine Learning Kit

73. Apple - Live Photo Loops

74. Apple - Portrait Lightting

75. DRAWING RECOGNITION
USE CASES

76. Apple Notes
Can an algorithm understand what
you were writing?

77. Apple Notes

78. Apple - Natural Language API

79. Google Quick Draw
Can an algorithm understand what
you were drawing?

80. Google Quick Draw

81. Google Quick Draw

82. Google Quick Draw

83. Google AutoDraw
Can an algorithm even help to teach
you drawing?

84. Google AutoDraw

85. Google AutoDraw

86. pix2pix
Can an algorithm turn your drawings
into compositions?

87. pix2pix

88. pix2pix

89. pix2pix

90. pix2pix
Philli
p
Leoni
e
Tobi

91. Drawing music
Can an algorithm turn your drawings
into music?

92. Drawing music

93. IT DOESN’T STOP
HERE

94. Lyrebird
Can an algorithm create realistic
voice overs from a given person?

95. Lyrebird

96. University of Washington
Can an algorithm create realistic
video from audio files alone?

97. University of Washington

98. pix2code
Can an algorithm program a user
interface from any given layout?

99. pix2code

100. pix2code

101. Style Transfer
Can an algorithm re-create complex
aesthetics and even fine art?

102. Style Transfer

103. Style Transfer

104. Style Transfer

105. Style Transfer

106. Style Transfer

107. Style Transfer

108. Style Transfer

109. Adobe Research

110. ING / Microsoft

111. ING / Microsoft

112. ING / Microsoft

113. Creative Adversarial Networks
Can an algorithm invent new styles
on its own?

114. Creative Adversarial Networks

115. Fontjoy
Can an algorithm generate aesthetic
combinations?

116. Fontjoy

117. Fontjoy
Can an algorithm generate aesthetic
combinations?

118. Logo Experiment
Can an algorithm generate logos?

119. Logo Experiment
Rob Peart tried to
generate Death Metal
logos from Neural
Networks ;)

120. Darkthrone
Logo Experiment

121. Waking The Cadaver
Logo Experiment

122. From the sunset, forest and grief
Logo Experiment

123. Logo Experiment

124. Logo Experiment

125. Logo Experiment

126. Logo Experiment

127. Can an algorithm generate logos?
Logo Experiment

128. NVIDIA - Iray
Can an algorithm squash production
times?

129. NVIDIA - Iray

130. NVIDIA - Iray

131. Autodesk - DreamCatcher
Can an algorithm generate products
that solve complex problems?

132. Autodesk - DreamCatcher

133. Autodesk - DreamCatcher
The three structural elements shown are all designed to
carry the same structural loads and forces.

134. Autodesk - DreamCatcher
A Lightweight bike stem generated by an algorithm

135. Autodesk - DreamCatcher
This is a car frame that is designed by a generative
algorithm

136. Autodesk - DreamCatcher
A a lightweight load-bearing engine block

137. Objectifier
Can an algorithm let objects
respond to human gestures?

138. Objectifier

139. Objectifier

140. Objectifier

141. WHAT IS
NEXT?
© Lawrence Lek© Lawrence Lek

142. Artificial Narrow Intelligence
Artificial General Intelligence
Artificial Superintelligence

143. Open AI - UNIVERSE

144. WHAT ARE
CHALLENGES
FOR UX?
© Lawrence Lek

145. MACHINE LEARNING
WON’T REACH ITS POTENTIAL
IF IT DOESN’T DEVELOP
IN TANDEM WITH
USER EXPERIENCE DESIGN
Caroline Sinders, Fast Company

146. Develop relevant use cases

147. Spotify
© Fabien Girardin

148. Spotify
© Fabien Girardin

149. MACHINE LEARNING
WILL CHANGE
CUSTOMER PERSONAS
FOREVER
Andre Smith, Digitalist Magazine / SAP

150. Re-think customer personas

151. Create intuitive AI interfaces

152. Make tons of data manageable

153. Use data to be super-relevant
or be silent

154. Let users tell about poor
information

155. • Filter Bubble - Tweak
algorithms to be less accurate
• Profile Detox - Let an open
door to reshape profiles
• Human Computation - Enlist
humans to give more diversity
© Fabien Girardin
Design for discovery

156. EVERY TIME
I CHECK MY PHONE,
I’M PLAYING
THE SLOT MACHINE
Tristan Harris, a former Google product manager

157. Design for engagement responsibly

158. Empathy is not (yet) available

159. Questions are the new answers
© Jan Korsanke

160. Illustrate for transparency

161. Seamful design

162. Machine bias: AI can lead to
discrimination

163. ULTIMATELY,
DESIGNERS MUST ACT
AS A BULWARK
AGAINST IRRESPONSIBLE,
UNETHICAL USE OF AI
Katharine Schwab, Fast Company

164. Satya Nadella, Microsoft CEO
• AI must be designed to assist humanity
• AI must be transparent
• AI must maximize efficiencies without
destroying the dignity of people
• AI must be designed for intelligent privacy
• AI must have algorithmic accountability
• AI must guard against bias

166. LET’S MAKE
DIGITAL SUCK LESS

167. @kuttin3dge
hotline@krunchtime.org
krunchtime.org
THANK YOU