Second screen applications are becoming key for broadcasters exploiting the convergence of TV and Internet. Authoring such applications however remains costly. In this paper, we present a second screen authoring application that leverages multimedia content analytics and social media monitoring. A back-office is dedicated to easy and fast content ingestion, segmentation, description and enrichment with links to entities and related content. From the back-end, broadcasters can push enriched content to front-end applications providing customers with highlights, entity and content links, overviews of social network, etc. The demonstration operates on political debates ingested during the 2017 French presidential election, enabling insights on the debates.

1. EURECOM – Campus SophiaTech,
450 Route des Chappes, 06410 Biot FRANCE
www.eurecom.fr
NexGenTV: Providing Real-Time Insight during Political Debates
in a Second Screen Application
Olfa Ben Ahmed1, Gabriel Sargent2, Florian Garnier3, Benoit Huet1 Vincent Claveau2,
Laurence Couturier3, Raphaël Troncy1, Guillaume Gravier2, Philémon Bouzy3, Fabrice Leménorel4
benoit.huet@eurecom.fr, vincent.claveau@irisa.fr, laurence.couturier@avisto.com, fabrice@wildmoka.com
ACM MULTIMEDIA 2017
October 23-27, 2017, Mountain View, USA
(1)
(2)
(3)
(4)
Objectives
 Enrich viewers’ TV experience by providing relevant additional information and interactive content on their mobile devices.
 Leverage on latest multimedia content analytics and social media monitoring to fuel a second screen authoring platform.
 Provide an end to end platform from broadcast stream to front-end application on mobile devices.
Motivation
 Exploit the convergence of TV and Internet to enable Broadcasters to produce and deliver Second Screen Applications and facilitate Social TV.
Framework
Content Analysis Technologies
Functionalities to segment, describe, enrich TV streams based on content
and social network analytics:
 Face detection and recognition
 Detecting and identifying key persons within a TV program in real
time using deep facial representation
 Speaker Diarization
 Segmentation of the program based on speech turns
 Term Extraction
 Identification of the salient/representative terms describing a
sequence
 Content Hyperlinking
 Providing links to related material based on subtitles
 Tweet collection and Analysis
 Online social activity monitoring allows to detect important events
in the program and to detect the polarity of comments (sentiment
analysis)
Interfaces
 Back-Office
 Clip selection, assisted from speech and speaker turn
detection
 Clip enrichment, using content analysis and
exploiting an ontological description of political life
 Clip publication (pushed on the mobile app)
 Mobile App
 Clips pushed from the back-office appear on the
timeline and can be viewed along with their
description and additional hyperlinks
 Real-time stats are available for viewing during the
program: speaking time, visual appearance duration,
amount of reaction on social media, popularity of
candidates…
video
subtitles
Social Networks
Internet
tweets
clip
selection
Broadcaster
Operator
BACK-OFFICE
T0
Knowledge Base
NEXGENTV
Clip3Clip2Clip1
- Face Detection and Recognition
- Speaker Diarization
- Segmentation
- Keywords Detection
- Named Entity Recognition
- Event – Clip Association
- Tweet – Clip Association
- Named Entity based Search
- Topic based Search
- HyperLinking
End User
Mobile App
T-end
http://www.nexgentv.fr/