Web-Enabled Collaborative Semantic Video Annotation

1. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-1
TeLLNet
Learning
Layers
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
SeViAnno 2.0: Web-Enabled Collaborative
Semantic Video Annotation Beyond the
Obvious
Petru Nicolaescu & Ralf Klamma
RWTH Aachen University
Advanced Community Information Systems (ACIS)
{lastname}@dbis.rwth-aachen.de

2. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-2
TeLLNet
Learning
Layers
Responsive
Open
Community
Information
Systems
Community
Visualization
and
Simulation
Community
Analytics
Community
Support
WebAnalytics
WebEngineering
Advanced Community
Information Systems (ACIS)
Requirements
Engineering

3. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-3
TeLLNet
Learning
Layers
Agenda
 Motivation
 Background
 A Web information system for video annotation
– Collaborative
– Near real-time
– Cloud-enabled
 SeViAnno 2.0: design and implementation
 Conclusions and outlook

4. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-4
TeLLNet
Learning
Layers
Motivation or
„We kill people based on metadata“ (David Cole)
 Good tools for automatic multimedia annotation
available „for special domains“
 Good single user tools for multimedia annotation
 Beyond the obvious: (Web) tagging the non-visible
 Collaborative multimedia annotation
 Decoupled infrastructure
+
+
-
-
-

5. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-5
TeLLNet
Learning
Layers
Related Work
SVCAT
VideoAnnEx
videoANT
M-OntoMat 2.0
SeViAnno 2.0
Tool Platform Usage Tagging Method
Desktop Web-based Single user Collaborative Manual Automatic
++ ++ + ++
++ ++ + ++
++ ++ ++
++ ++ ++
++ ++ ++
M-OntoMat 2.0
+ Via-tool
SVCAT
VideoAnnEx
videoANT

6. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-6
TeLLNet
Learning
Layers
EU FP7 Learning Layers Project
 How can we scale up video
tagging to support informal
learning at the work place
 Objectives
– Mobile creation of videos
with learning materials
(physical-digital world)
– Tag the non-obvious for
informal learning in
communities (social media
layer)
– Scaffold meaningful learning
by exploiting semantic
tagging information (social
semantic layer)
 Two regional clusters
– Construction (Germany)
– Healthcare (UK) http://learning-layers.eu/

7. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-7
TeLLNet
Learning
Layers
Decoupled Multimedia Annotation
Information System Architecture
 3-tier architecture
 Cloud-enabled
solutions
 Advantages
– Separation of
concerns
– Scalability
– Rapid development &
deployment
– Application
customization

8. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-8
TeLLNet
Learning
Layers
Multimedia Metadata Web Services
Tethys (former i5Cloud)
LAS
SeViAnno MPEG-7
Metadata Services
Connectors
(REST, HTTP, Ajax)
User Management
Object Management
Session Management
Cloud Video Transcoder
Tethys Data Storage
Video Transcoding
Scaling Management
Cloud Video Upload
ROLE SDK
NRT
Collaboration
Widget/Space
Management
Streaming
Server

9. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-9
TeLLNet
Learning
Layers
Multimedia Metadata User Interface

10. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-10
TeLLNet
Learning
Layers
Near Real-Time (NRT) Collaboration
 Data structure dependent
– Tree-like data structures with operational transformations
– e.g. XML-based (MPEG-7)
 NRT updates propagation
– Underlying infrastructure and used protocols (e.g. XMPP)
 Preserving data consistency and user intention
– NRT video operations
– Concurrent updates

11. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-11
TeLLNet
Learning
Layers
Space (shared by multiple users)
SeViAnno 2.0 & ROLE Framework
Web application (composed of widgets)
Widget (collaborative web
component)
EU FP7 ROLE Project
http://role-sandbox.eu/

12. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-12
TeLLNet
Learning
Layers
User Interface Distribution
Limited space and
interaction
possibilities
Trend to using
multiple personal
devices
Lack of tools and
methodologies for
Web-based DUIs

13. Lehrstuhl Informatik 5
(Information Systems)
Prof. Dr. M. Jarke
I5-NiKl-0714-13
TeLLNet
Learning
Layers
Conclusion and Future Work
 Open Source Development and Cloud Deployment
(ask for details)
 Strong and scalable service infrastructure
 (Distributed) Web widget and mobile interfaces
 Challenges
– WebRTC infrastructure for update propagation
– Operational transformation algorithms for collaborative
semantic annotations
– Parallel video processing in the cloud (MapReduce)
– OpenID Connect (OIDC) for security and privacy