Fabio Galasso Sapienza Università di Roma

1. Intelligenza Artificiale nella PA: esperienze e
prospettive
Roma, 5 Novembre 2019 h. 9.00-13.30
organizzato e promosso da
www.prsmonitor.euwww.di.uniroma1.itwww.miur.gov.it
Computer Vision e Machine Learning
Applicazioni e Sviluppi
Fabio Galasso
Sapienza Università di Roma

2. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
2
Successes of computer vision and machine learning
Courtesy of A. Torralba, @ICVSS’18

3. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
3
Visual Recognition Tasks
Courtesy of G. Gkioxari, @ICVSS’18

4. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
4
Progress in Visual Recognition
• Mask R-CNN, best paper award @ICCV’17

5. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
5
Progress in Visual Recognition
• Mask R-CNN, best paper award @ICCV’17
Applied to pose estimation

6. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
6
Challenges in Visual Recognition
• Detection performance
• Visual recognition is far from solved
Courtesy of G. Gkioxari, @ICVSS’18

7. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
7
Promising Research Pathway
• Deep Neural Networks match the intuition.. and a part of our brain
Learning Hierarchical Representations of Object
Categories for Robot Vision. Leonardis, Fidler. 2007
Backpropagation Applied to Handwritten Zip Code
Recognition. LeCun et al. 1989

8. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
8
Outline
• Research background
• Innovation transfer
• Research highlights

9. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
9
Outline
• Research background
• Innovation transfer
• Research highlights

10. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
10
• 2005-2009
PhD in Engineering
• 2009-2011
PDoc Researcher
Image sources: https://www.telegraph.co.uk/news/uknews/12045089/Cambridge-University-students-glorify-rape-and-play-stand-up-if-you-hate-the-poor-during-society-meetings.html

11. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
11
Textures and 3D reconstruction
BMVC’07, ISVC’07, ISVC’08, CVPR’09
• Leverage texture to recover the shape
In controlled environments
In the “wild”

12. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
12
Label propagation and video analysis
CVPR’10, ICCV’11
• Model videos with a graphical model
• First label propagation

13. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
13
• 2011-2014
PDoc Research Associate
Image source: https://www2.cs.uni-saarland.de/forschung-research/forschungsinstitutionen/index.html

14. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
14
Video segmentation
ACCV’12, ICCV’13, ACCV’16
• Learn from videos to mimic the human perception
Spectral graph reductionVideo
Ground truth Ours
Segm. propagation
Grundmann et al. CVPR’10

15. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
15
Clustering with graphs
CVPR’14, GCPR’14, CVPR’15
• Learn graph representations for videos
• Learn grouping constraints
• Prove equivalence of graph reductions
Pixel/voxel
Graph for a video
Superpixel
Frame

16. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
16
Outline
• Research background
• Innovation transfer
• Research highlights

17. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
17
• 2014-2019
R&D Manager
Head of Computer Vision Dept
Image source: https://www.handelsblatt.com/unternehmen/industrie/lichtkonzern-osram-tief-in-den-roten-zahlen/24855322.html

18. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
18
Business potential
• Light is everywhere people are
• Cameras are small and mass produced
• Cameras enable several use cases
Presence detection for automatic switch on/off
People localization for counting
Gesture recognition for human-light interaction
Activity recognition for automatic lighting setup
Light estimation for calibration
People tracking for pattern recognition in shops
People identification for personalized lighting
…
Image sources: https://www.amazon.it/Victorinox-Swisschamp-Coltello-colore-Rosso/dp/B0001GS19U/
https://www.wallpaperflare.com/city-skyline-during-nighttime-wallpaper-82253
https://www.calonassociates.com/faq-occupancy-detection---what-is-the
https://www.amazon.es/Port%C3%A1til-Interior-Nocturna-Vigilancia-Movimiento/dp/B07T6DVX2S

19. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
19
Business potential
• Light is everywhere people are
• Cameras are small and mass produced
Image sources: https://www.amazon.it/Victorinox-Swisschamp-Coltello-colore-Rosso/dp/B0001GS19U/
https://www.wallpaperflare.com/city-skyline-during-nighttime-wallpaper-82253
https://www.calonassociates.com/faq-occupancy-detection---what-is-the
https://www.amazon.es/Port%C3%A1til-Interior-Nocturna-Vigilancia-Movimiento/dp/B07T6DVX2S

20. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
20
Business potential
• Light is everywhere people are
• Cameras are small and mass produced
• Cameras enable several use cases
Presence detection for automatic switch on/off
People localization for counting
Gesture recognition for human-light interaction
Activity recognition for automatic lighting setup
Light estimation for calibration
People tracking for pattern recognition in shops
People identification for personalized lighting
…
Image sources: https://www.amazon.it/Victorinox-Swisschamp-Coltello-colore-Rosso/dp/B0001GS19U/
https://www.wallpaperflare.com/city-skyline-during-nighttime-wallpaper-82253
https://www.calonassociates.com/faq-occupancy-detection---what-is-the
https://www.amazon.es/Port%C3%A1til-Interior-Nocturna-Vigilancia-Movimiento/dp/B07T6DVX2S

21. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
21
Occupancy detection for lighting control
• Top-view people detection is challenging
Image source: screenshot from Google image search

22. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
22
Occupancy detection for lighting control
• Top-view people detection is challenging
• It requires
Specific modelling
Specific data
Data annotations
Image source: screenshot from Google image search

23. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
23
From Conception to Market
• All that glitters is not gold
Is it worth it?
May that really be solved?
Courtesy of G. Gkioxari, @ICVSS’18
Image sources: https://www.sdmmag.com/articles/93743-wireless-cameras-rising-in-popularity
https://conrad.ru/catalog/datchiki_dvigeniya/1485335/, https://ruwimgt.pw/google-vision-kit-face-recognition.html
https://www.theverge.com/circuitbreaker/2016/5/16/11682468/raspberry-pi-zero-camera-connector

24. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
24
From Conception to Market
• All that glitters is not gold
Is it worth it?
May that really be solved?
Would people accept it?
Image sources: https://www.sdmmag.com/articles/93743-wireless-cameras-rising-in-popularity
https://conrad.ru/catalog/datchiki_dvigeniya/1485335/, https://ruwimgt.pw/google-vision-kit-face-recognition.html
https://www.theverge.com/circuitbreaker/2016/5/16/11682468/raspberry-pi-zero-camera-connector

25. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
25
Marketing AI may require a change in perspective
Image source: https://panpica.pw/learn-to-embrace-the-storms-of-your-life-meaning.html

26. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
26
Pick the “Right” Market
• Typical spendings in a building
29 €/m2
290 €/m2
2900 €/m2
Courtesy of www.visn.io

27. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
27
VISN
• Optimize the rented space
Courtesy of www.visn.io

28. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
28
VISN
• Optimize the rented space
Courtesy of www.visn.io

29. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
29
VISN
• Optimize the rented space
• Create a more efficient environment for employees
Find the perfect workspace Find your co-workers
Courtesy of www.visn.io

30. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
30
A success story
• Recognition by field experts
• Recognition by customers
• Now on sale, also with thermal cameras
Courtesy of www.visn.io

31. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
31
Outline
• Research background
• Innovation transfer
• Research highlights
People forecasting (WACV’18, CVPR’18, TPAMI’19)
Person re-identification and search

32. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
32
Forecasting
• Human-robot interaction
Image sources: https://www.grasp.upenn.edu/projects/haptics-human-robot-interaction
http://www.woostercollective.com/post/3d-optical-illusion-painted-on-street-to-make-drivers-slow-down

33. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
33
Forecasting
• Human-robot interaction
• Long-term tracking across occlusions
Image sources: https://www.grasp.upenn.edu/projects/haptics-human-robot-interaction
http://www.woostercollective.com/post/3d-optical-illusion-painted-on-street-to-make-drivers-slow-down

34. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
34
Forecasting
• Human-robot interaction
• Long-term tracking across occlusions
• Safety of autonomous vehicles
Image sources: https://www.grasp.upenn.edu/projects/haptics-human-robot-interaction
http://www.woostercollective.com/post/3d-optical-illusion-painted-on-street-to-make-drivers-slow-down

35. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
35
Also with Forecasting
• Weather and natural disasters, e.g. earthquakes
• Market trends
• Critical pressure of a power plant
• Gas supply and demand
• …
Image sources: http://www.vestinet.rs/pogledi/ozbiljno-upozorenje-naucnika-u-2018-ocekujte-razorne-zemljotrese-evo-gde-ce-biti-najgore-video
https://allstarcharts.com/monthlysjune2018/, https://www.thetimes.co.uk/article/national-gas-alert-as-prices-soar-08xjfqc8q

36. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
36
More on the Gaze

37. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
37
Head Pose and Motion
People watch their steps

38. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
38
Head-pose
• Key to people attention

39. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
39
When the Head Pose Diverges

40. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
40
MX-LSTM
• Use people velocity and
head pose jointly
• To forecast people motion
• In line with sociological
findings and data
statistics

41. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
41
MX-LSTM
Social Pooling
• Leverage head pose for person-person interaction

42. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
42
Social LSTM Vs MX-LSTM
Prior work (Social-LSTM)
Our MX-LSTM

43. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
43
Outline
• Research background
• Innovation transfer
• Research highlights
People forecasting
Person re-identification and search (CVPR’19, BMVC’19)

44. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
44
Re-identification
• Find the queried person in a gallery of cropped samples
Query
Gallery

45. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
45
Person Search
• Find a queried person within a gallery of images (detection + re-id)
GalleryQuery

46. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
46
Motivation for Person Search
• Surveillance
• Long-term tracking across-views
• Novel algorithms for granting access
Image source: https://research.qut.edu.au/saivt/research-projects/person-re-identification-using-soft-biometrics/

47. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
47
New State-of-the-art at CVPR’19 and BMVC’19
• Strong quantitative and qualitative results
Query Retrieved Query Retrieved

48. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
48
Outline
• Research background
• Innovation transfer
• Research highlights

49. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
49
Conclusions
• Computer vision and machine learning undergo great progress,
but need more research
• Innovation based on AI may be the new oil,
but it requires deep technical, financial and social awareness
• With progress, more potential unlock
Autonomous cars
Collaborative robots
Safer cities and factories

50. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
50
Conclusions
• Computer vision and machine learning undergo great progress,
but need more research
• Innovation based on AI may be the new oil,
but it requires deep technical, financial and social awareness
Image sources: https://towardsdatascience.com/data-is-the-new-oil-f11440e80dd0
https://capital.com/dividend-investing-for-outsized-returns, https://www.webgyoronline.com/

51. Fabio Galasso
Computer Vision e Machine Learning; Applicazioni e Sviluppi | 5 November 2019
51
Conclusions
• Computer vision and machine learning undergo great progress,
but need more research
• Innovation based on AI may be the new oil,
but it requires deep technical, financial and social awareness
• With progress, more potential unlock
Autonomous cars
Collaborative robots
Safer cities and factories
Image sources: https://towardsdatascience.com/data-is-the-new-oil-f11440e80dd0
https://capital.com/dividend-investing-for-outsized-returns, https://www.webgyoronline.com/

52. Intelligenza Artificiale nella PA: esperienze e
prospettive
Roma, 5 Novembre 2019 h. 9.00-13.30
organizzato e promosso da
www.prsmonitor.euwww.di.uniroma1.itwww.miur.gov.it
Thank you
Fabio Galasso
Sapienza Università di Roma
Disclaimer: the opinions expressed in this presentation and in these slides are solely those of the presenter.