Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to 雲端影音與物聯網平台的軟體工程挑戰：以 Skywatch 為例-陳維超
Similar to 雲端影音與物聯網平台的軟體工程挑戰：以 Skywatch 為例-陳維超 (20)
More from 台灣資料科學年會
More from 台灣資料科學年會 (20)
Recently uploaded
Recently uploaded (20)
雲端影音與物聯網平台的軟體工程挑戰：以 Skywatch 為例-陳維超
- 1. http://www.skywatch24.com 雲端影⾳音與物聯網平台之 軟體⼯工程挑戰 陳維超 Wei-Chao Chen | Skywatch Inc. | NationalTaiwan University
- 2. WARNING May contain shameless self-promotion YOU HAVE BEEN WARNED
- 3. 1998 2002 2006 2010 2014 ABOUT MYSELF
- 4. Internet Camera x Cloud Power www.skywatch24.com
- 5. extend
- 6. your
- 7. perception
- 9. giving
- 10. meaning
- 11. to
- 12. the
- 14. are
- 15. 3
- 16. people
- 17. in
- 18. your
- 19. yard!
- 23. and
- 24. reliable
- 25. connectivity
- 26. in
- 27. over
- 28. 10
- 29. countries
- 30. complete
- 31. hardware
- 33. software
- 34. solution
- 35. cloud
- 36. video
- 37. analytics
- 38. make
- 39. regular
- 40. network
- 41. cameras
- 42. smart
- 43. ! ? …
- 44. 318 BROADCASTING
- 46. THE RESEARCH
- 47. VIDEO ANALYTICS A methodology to reason and understand video I saw three person last hour
- 48. People Count https://www.youtube.com/watch?v=TPjmsha6gWY Loitering https://www.youtube.com/watch?v=f5DxUe5I8as Trip wire https://www.youtube.com/watch?v=MH2HZi6-4jA Speed test https://www.youtube.com/watch?v=KV6CXdzmhNs
- 49. SMART FAST FORWARD Video activity level Single button interface
- 50. SMART FAST FORWARD •20x average speedup •Compressed domain processing •30x faster processing compared to competition •3 seconds for every 10 minutes
- 52. SMART DAILY •Intelligent video summary •Daily email on your past day’s activity - Compressed domain analysis - Very fast - totally I/O bound - Detect events based on activity / sensible motion
- 53. Click to see video Yes, it is secure (How?)
- 54. SMART CAPTION •Image to text / deep learning
- 55. SMART CAPTION •Accuracy not critical for application (?!) •Deep learning classification •Berkley Caffe •ImageNet / Microsoft COCO • Stanford NeuralTalk •GPU / CUDA powered •1 second per image
- 58. FEATURE MATRIX TrainingCost Runtime Cost Face Detect Loitering Trip wire Area Alert Left object Motion License Plate People Detection Face Recognition Object Classification Age Gender People Count Heat Map Smart Caption Smart Fast Forward Smart Daily Skywatch Regular IVA
- 59. EDGE COMPUTATION Compute The Internet Video Meta-data Je suis un Edge
- 60. CLOUD COMPUTATION (SOMEWHAT POINTLESS) The Internet Video Meta-data Compute
- 61. CLOUD COMPUTATION (SENSIBLE) Video Meta-data The Internet Compute Data model Video storage
- 62. HYBRID COMPUTATION (OURVISION) Early Vision Partial Meta-data The Internet Late Vision Data model Video storage Various meta-data results
- 63. CLOUDVS LOCAL Cloud Local How? Cheap camera Cloud computer Special camera Local computer Best for Search classify Big data Real-time alerts Advantages Easy to deploy manage Cheap camera Rental model Lower bandwidth Real-time applications Disadvantages Higher bandwidth Maybe not for alerts Difficult to deploy manage Expensive camera Higher upfront cost
- 64. THE ENGINEERING Ivan Sutherland, the Sketchpad
- 65. WE HAVE BUILT STUFFS •Chief shader architect for Playstation 3 (RSX) •75% of the GPU chip •Expensive bugs ( $100K USD each) •Targeted test, random test, monkey test •From architecture to driver to hardware •Shipped over 100M systems
- 66. WE HAVE INVENTED STUFFS •OpenCV •MPEG4 Source / OpenLF •Fancy publications •US Patents
- 67. HOW HARD CAN IT BE? Get the plumbing done, then add smartness
- 68. Prototype Scalable Stable Initial Production Prototype
- 69. SOFTWARE ENGINEERING Aspects What do we do? Development Scrum / continuous integration Testing Unit / regression testing Review Design review / Design patterns Code review Operation Push-button code deployment Server monitors / camera diagnostic tools What about the cameras?
- 71. Sandbox (Truman’s world) TESTABILITY Test Subject Test Data Fake / Mock Interface
- 72. Sandbox (Truman’s world) TESTABILITY Test Subject Test Data Fake Camera Interface Really hard to make a good fake device
- 73. COMPLEXITY Firmware version Camera models Server config Software delivery Browser type App version
- 74. CONTROLTHE COMPLEXITY OTA upgrade Fewer models VM / container Push-button release Versioned API Limit OS revision Frameworks are your friends(?!)
- 75. High-bandwidth video traffic Low-bandwidth API traffic System Architecture Overview (Dual Region) Storage SAN Compute Servers Streaming Servers Region A P2P Video Storage SAN Compute Servers Streaming Servers Region B User DB Application Servers Application Servers
- 76. Embedded Linux Camera Monitors VPN Interface P2P Interface RTSP Streaming RTMP Streaming LAN Interface Camera API VPN Interface RESTful Cloud API User DB Stream Broadcaster Video Files Camera Software Stack Skywatch Service API Internet Skywatch Firmware / Software Camera Firmware
- 78. #1: BOB SAW ALICE (AND MAYBE CHARLES) Alice Bob Bob Alice Charles Very early mistake — this won’t happen again
- 79. •How did this happen? •Not thread-safe / process-safe •Unique IDs are not unique (after a certain scale) •Loose DB network synchronisation •Old tunnels hijacked by new cameras •Camera bug — HTTP authentication off in a firmware version #1: BOB SAW ALICE (AND MAYBE CHARLES)
- 80. Alice •Why it won’t happen again? #1: BOB SAW ALICE (AND MAYBE CHARLES) Alice Bob HTTP Password authenticated Charles Individually signed CRC’ed files Scanner to make sure camera authentication is on Channel scanner to clean up dangling connections RESTful API check for signature
- 81. #2: FIRMWARE UPGRADE EQUALS ANGRY CUSTOMERS •Auto upgrade? •Bad idea — people unplug and brick the camera •User-triggered OTA upgrade •Camera restore to default WiFi settings gone •Continuous upgrade testing is needed •Monitor customer upgrade events / failures
- 82. #3: JAVA - WRITE ONCE, DEBUG EVERYWHERE •Browser H.264 decoder / multi-channel player •Originally JAVA-based. Why? •Because a big company is using it •Because Flash was killed by Jobs •Because we can save money (P2P) •Tons of support issue (!) Browser Player Flash / HTML5 player
- 83. #3: JAVA - WRITE ONCE, DEBUG EVERYWHERE •Browser H.264 decoder / multi-channel player •Originally JAVA-based. Why? •Because a big company is using it •Because Flash was killed by Jobs •Because we can save money (P2P) Browser Player Flash / HTML5 player Believe in yourself, not the competitor Choose a popular weapon Worry about money saving when it becomes a real issue
- 84. THE OUTLOOK Preikestolen (cc) Anna Bialkowska https://www.flickr.com/photos/annabialkowska/117380607
- 85. Converse Control Connect Data and services need to talk to each other
- 86. Soon sensors
- 87. that
- 88. talk
- 89. to
- 91. each
- 92. other
- 93. intelligently
- 94. record
- 95. kid’s
- 96. study
- 97. time
- 98. and
- 99. compare
- 100. with
- 101. her
- 103. stranger
- 104. sneaking
- 105. around
- 106. in
- 107. your
- 110. Power transmission lines (cc) Oran Viriyincy https://www.flickr.com/photos/viriyincy/7004881064 Home Automation v.s. Grid Management