1. Understanding film
scholars’ annotation
behavior:
Supporting scholarship
by enhancing online film
annotations
Liliana Melgar-Estrada
PhD candidate in Information Sciences
Universidad Carlos III de Madrid
lmelgar@bib.uc3m.es
Digital Humanities Benelux Conference
June 12, 2014
1. Motivation and Research
framework
i. Context, motivation
ii. PhD thesis overview
iii. Background
research:
Perspectives in
(moving) image
indexing
2. Conceptual framework
i. Crowdsourcing
ii. Nichesourcing
3. Case study
i. Research questions
ii. Participants
iii. Method
iv. Findings and
discussion
4. Conclusions
2. PhD thesis in Information Science(s)
Information behavior studies
2
Wilson's 1999 nested model of information behaviour
“How and why can current indexing perspectives be used by film
archives to support film scholarship. ”
9. Study: Understanding film scholars’ annotation behavior
• RQ: How do film scholars behave as moving image
indexers?
• Theoretical framework: Information behavior
– cognitive theory for interactive information retrieval (IIR) put
forward by Ingwersen (1992, 1996, 2001)
• Method: Pia Borlund’s simulated work tasks
All participants got 2 different film clips (1 control clip), 1 short movie
and selected 1 movie themselves for each task in randomized order (in
total 2 clips and 2 movies)
1. Context independent and system-free task
2. Context dependent and system-free task
3. Context and system dependent
4. System dependent, context not clear
9
10. Participants
• Participants: 10 film scholars from 5 universities
in Madrid:
– Autónoma de Madrid (Art History dept.)
– Camilo José Cela University (Communication Sciences
dept.)
– Carlos III (Audiovisual Communication dept.)
– Complutense (Audiovisual Communication dept.)
– Rey Juan Carlos (Communication Sciences dept.)
• Film scholars with 10 or more years of experience in
publishing, teaching, editorial activities, festival
commitees, etc. 10
11. Finding I. Annotation styles
1. Descriptive texts (Para-texts) *
– Narrative structure + critical texts
– Interpretative clues for the reader
2. Combination of Descriptive/Critical texts and Keywords
3. Keywords / Tags / Descriptors
4. Plot synopsis
*(Gérard Genette, after commented by Nico de Klerk and studied in a recent book
edited by Pellatt, 2013)
11
12. Finding II. Unit of analysis and indexing
granularity
1. What do I have to describe? Clips vs entire movies
2. Annotation for access vs annotation for scholars’ own
research –Describing and searching tasks differ (Fidel,
1997)
12
RDA: Rules for Archival Description (Turner, 2009)
13. Finding III. Attributes
Sound, music
Narrative, story, plot, characters
Genre, movement, country
Influences, relations
Cinematographic, formal, stylistic features
CONTEXT, historical information
13
14. Finding IV. Willingness to participate
• 9 of the 10 participants said “Yes” to a proposal to
contribute to Nichesourcing initiatives by a film archive,
1 said “No”
• 1 of the scholars saying yes, hesitated of the success of
such idea: “Access is not about indexing or ordering, but
about selection and education”
• Scholars suggested: clarity, easiness, reward (not
necessary economic, but in the form of input and
feedback on the use of their voluntary work)
• The scholar who said “No” was very much aware of the
need of Indexing knowledge for any annotation task
14
15. Recommendations for Nichesourcing
• Tags or keywords alone are not enough for supporting
Film scholarship, different annotation styles should be
supported and “mined”
• Indexing knowledge should not be required from
domain experts. Any annotation system for scholars
has to clearly guide the indexing process
• The purpose and use of the annotations has to be
stated clearly
• Scholars should be provided with films they are familiar
with or are interested in (this is already done in
projects such as Naturalia/Accurator).
15
18. REFERENCES
De Boer, V., Hildebrand, M., Aroyo, L., De Leenheer, P., Dijkshoorn, C.,
Tesfa, B., & Schreiber, G. (2012). Nichesourcing: Harnessing the Power of
Crowds of Experts. Presented at the EKAW, Ireland. Retrieved from
http://ekaw2012.ekaw.org/node/101
Fidel, R. (1997). The image retrieval task: implications for the design and
evaluation of image databases. The New Review of Hypermedia and
Multimedia, 3. Retrieved from
http://www.ischool.washington.edu/fidelr/RayaPubs/ImageRetrieval%2
0Task.pdf
Geisler, G., Willard, G. & Ovalle, C. (2011). “A crowdsourcing framework
for the production and use of film and television data.” New Review of
Hypermedia and Multimedia, 17(1), 73-97.
Huurnink, B., Snoek, C. G. M., de Rijke, M., & Smeulders, A. W. M.
(2012). Content-Based Analysis Improves Audiovisual Archive Retrieval.
IEEE Transactions on Multimedia, 14(4), 1166–1178.
doi:10.1109/TMM.2012.2193561
Martorell, S., Canet, F., & Codina, L. (2014). Channelling academic
audiences: proposal for a social network for film studies researchers.
Hypertext.net, (12). doi:10.2436/20.8050.01.12 18
19. Patel, B. V., & Meshram, B. B. (2012). Content based video retrieval
systems. International Journal of UbiComp, 3(2), 13–30.
doi:10.5121/iju.2012.3202
Pellatt, V. (Ed.). (2013). Text, Extratext, Metatext and Paratext in
Translation - E-book | Krisostomus. Newcastle upon Tyne, UK:
Cambridge Scholars Publishing. Retrieved from
http://www.cambridgescholars.com/download/sample/60628
Ross, M., Grauer, M., & Freisleben, B. (2009). Digital Tools in Media
Studies: Analysis and Research : an Overview. Bielefeld: transcript
Verlag.
Turner, J. M. (2009). Moving Image Indexing. In Encyclopedia of
Library and Information Sciences, Third Edition. New York: Taylor &
Francis. Retrieved from
http://www.tandfonline.com/doi/abs/10.1081/E-ELIS3-120043268
Weinberger, D. (2008). Everything Is Miscellaneous: The Power of the
New Digital Disorder. New York: Henry Holt. 19
REFERENCES
20. Acknowledgements
This study was possible due to the participation
of 10 enthousiastic film scholars in Madrid,
specially to Marina Díaz López (Cervantes
Institute) who provided information and helped
in contacting most of them.
20
21. Image credits
1. http://social.technet.microsoft.com
2. Buster Keaton, Sherlok Jr. http://www.cinema.indiana.edu
3. Laaf, Meike. Internet Visionary Paul Otlet: Networked Knowledge, Decades
Before Google. http://www.spiegel.de
4. The Image Processing and Analysis Laboratory.
http://imag.pub.ro/en/lab/index_2.jpg
5. http://woordentikkertje.manbijthond.nl/
6. http://www.dutchgameawards.nl/2013/spotvogel/
7. The Media Ecology Project: http://sites.dartmouth.edu/mediaecology/
8. Larm: Audio Research Archive. http://larm.blogs.ku.dk/about-larm/
9. Bibliothéque National de France (Annotation Vidéo)
http://blog.bnf.fr/lecteurs/index.php/2013/09/annotation-video-un-mois-
dexperimentation-avec-les-usagers-en-salle-p/
21