Take control of your SAP testing with UiPath Test Suite
Extracting Media Items from Multiple Social Networks
1. What Fresh Media Are You Looking
For? Extracting Media Items from
Multiple Social Networks
Giuseppe Rizzo1, Thomas Steiner2, Raphaël Troncy1,
Ruben Verborgh3, José Luis Redondo Garcia1
and Rik Van de Walle3
<raphael.troncy@eurecom.fr> / @rtroncy
1 EURECOM, France
2 Google & University Politècnica de Catalunya, Spain
3 IBBT Ghent, Belgium
2. Conferences and natural disaster
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -2
3. 29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -3
4. 29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -4
5. 29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -5
6. 29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -6
7. Some definitions
Media Item: a photo or a video that is shared on a social
network
Micropost: a text status message that can optionally
accompany a media item
Social Network: an online service that focuses on
building and reflecting social relationships among
people sharing interests or activities
Media Sharing Platforms: emphasis on sharing media but blurred
boundaries with social networks since users are encouraged to react
on media content (like, comment, favorite, etc.)
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -7
8. Social networks and media items
First-order support:
Posting requires the inclusion of a media item
Example: Flickr, YouTube
Second-order support:
Possibility to post media items but also text-only messages
Example: Facebook
Third-order support:
No direct support for media items but rely on third party applications
to host them
Example: Twitter before the introduction of native photo support
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -8
9. Media Collector (Server)
Composition of media item extractors (12 SNs)
Rely on search APIs + a fix 30s timeout window to provide results
Fallback on screen scraping when necessary (Twitter ecosystem)
Implemented as a NodeJS server
Serialize results in a common schema (JSON)
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan -9
10. Deep link
Permalink
Clean text for NLP
processing
Aggregate view of ALL
social interactions
12 Social Networks
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 10
11. Media Finder
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 11
12. Evaluation (1/3)
9 events occurring between 10 and 19 January 2012
Assad speech, CES Las Vegas, Costa Concordia Disaster, Cut the
Rope Launch, Dixville Notch, Free mobile launch, Blackout SOPA,
Ubuntu TV launch, Christian Wulff case
448 images + 143 videos
Photo-Sweeper CBIR-based image duplication detection software
Dataset heterogeneity:
Leaderboard banner (728x90) to a standard 3.1 mega pixels
(2048x1536) cell phone photo … no quadratic bitmaps shrinking
Hard problem!
Best settings for each event, no generic configuration, in order to
limit the number of duplicate misses and false positives
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 12
13. Evaluation (2/3)
User study to compare the relevance and
illustrativeness of the media galleries
One event: Google IO (“google i/o” + “io12”)
http://en.wikipedia.org/wiki/Google_io
Three systems:
Media Finder, Twitter Gallery, Teleportd
7 participants (6 male, 1 female) in 2 groups
MediaFinder Teleportd Twitter
Google i/o 108 (49%) 20 (9%) 96 (44%)
io12 69 (37%) 20 (10%) 98 (53%)
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 13
14. Evaluation (3/3)
Q1: How illustrative this gallery is for this event?
Q2: How visually diverse this gallery is for this event?
Lickert 7-scale: result http://goo.gl/QzSM6 + http://goo.gl/7ov6Q
google i/o io12
relevance Q1 Q2 relevance Q1 Q2
Media 0,28 2,35 2,72 0,21 2,05 2,24
Finder
Teleportd 0,05 0,30 0,37 0,04 0,35 0,59
Twitter 0,28 2,64 2,64 0,34 3,44 2,91
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 14
15. Demo: Grid view
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 15
16. Demo: Timeline view
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 16
17. Conclusion
Fresh media available on social networks
Ignored by general search engines …
… but ideal for building stories of events of our life
Media Server: a NodeJS server collecting media
items shared on social networks
Media Finder: a client-server architecture that
generates views of those media items
http://mediafinder.eurecom.fr/
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 17
18. Future Work
Image de-duplication:
Simple off-the-shelf tools using color, texture and shape
(Ramaiah and Mohan, IEEE RAICS’11)
Named Entity Recognition:
NERD: http://nerd.eurecom.fr/
Clustering and Storyfying:
Source and Temporal clustering
Visual clustering
Semantic clustering:
using named entities extracted in microposts
29/10/2012 - International Workshop on Socially-Aware Multimedia at ACM Multimedia 2012, Nara, Japan - 18