Presentation at the NYC Media Lab (NYCML2018). There is a growing demand for news videos online, with more consumers preferring to watch the news than read or listen to it. On the publisher side, there is a growing effort to use video summarization technology in order to create easy-to-consume previews (trailers) for different types of broadcast programs. How can we measure the quality of video summaries and their potential to misinform? This workshop will inform participants about automatic video summarization algorithms and how to produce more “representative” video summaries. The research presented is from the FAIRview project and is supported by the Digital News Innovation Fund (DNI Fund), which is part of the Google News Initiative.
2. Agenda for today
3:00 - 3:20: Introduction of Video Summarization Context
3:20 - 3:50: Work in groups to answer the following questions
(discussion document: http://bit.ly/fairview_discussion)
Q1: How to increase the user awareness (e.g. through explanations, visualizations,
interaction, etc) on the following two points:
○ the video summary “representativeness” compared to the original video
○ the (possible) video summary “misinformation potential” compared to the original video
Q2: What are adequate success metrics for video summaries?
○ How to measure the ‘representativeness’?
○ How to measure ‘misinformation potential’?
○ How to evaluate both points?
Answer these questions in the following interaction scenarios:
● while watching the video summary
● when browsing video search results
● when comparing two or more video summaries
● when creating the video summary
● other interaction scenarios
3:50 - 4:00: Summary and conclusions
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
3. video is 64% of Internet traffic
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
4. more Americans prefer to watch their news (46%)
than to read it (35%) or listen to it (17%)
http://www.journalism.org/2016/07/07/pathways-to-news/
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
5. 300h of video uploaded each min on YouTube alone
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
6. in 2020 it would take a person more than 5 million
years to watch the videos uploaded in a month
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
7. at some point it all looks the same
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
8. … tons of videos but difficult to choose what to watch
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
9. how to make videos consumable
in the age of information overload & declining attention span?
16. … e.g. micro-moments in video
for on-demand discovery search
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
17. … e.g. contextualized hyperlinks in video
for direct engagement
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
18. … creating bite size info nuggets (video snacks)
that can quickly be consumed, understood & shared
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
19. Let’s look at an example
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
20. Scenes from HBO Series: Big Little Lies
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
21. Scenes from 1 Episode
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
23. Frames for the selected sceneSelected: 1 Scene
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
24. All Concepts describing the FrameSelected: 1 Frame
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
25. all this results in a lot of
video, image and label data
… that could be organized in lots of different storylines
i.e. video snacks
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
30. … but how are these video stories created?
who selects what to include / exclude?
who chooses the summarization approaches?
what is the impact of different approaches?
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
31. … all these choices can amplify / diminish
a specific aspect or perspective in the original video,
and in this way introduce a bias
that can potentially lead to misinformation
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
32. FAIRView
● how to bring more awareness of all the perspectives,
topics and elements present in the original video
● what are indicators & evaluation criteria on how
these are represented in a video summary
● how to adapt existing summarization algorithms to
produce representative & explainable video
summaries
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
33. FAIRView
● study this problem in the context of news videos
● empower users with tools to evaluate
representativeness of videos
● gain a granular understanding of video content in
terms of perspectives, opinions, stories, etc.
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo
34. Work in Groups
3:20 - 3:50: Work in groups to answer the following questions
(discussion document: http://bit.ly/fairview_discussion)
Q1: How to increase the user awareness (e.g. through explanations, visualizations,
interaction, etc) on the following two points:
○ the video summary “representativeness” compared to the original video
○ the (possible) video summary “misinformation potential” compared to the original video
Q2: What are adequate success metrics for video summaries?
○ How to measure the ‘representativeness’?
○ How to measure ‘misinformation potential’?
○ How to evaluate both points?
Answer these questions in the following interaction scenarios:
● while watching the video summary
● when browsing video search results
● when comparing two or more video summaries
● when creating the video summary
● other interaction scenarios
3:50 - 4:00: Summary and conclusions
http://lora-aroyo.org https://www.slideshare.net/laroyo @laroyo