The University of Sheffield, 24 June 2020 I Online research seminar hosted by STeMiS and the Digital Society Network.

1. HOW TO READ COMPUTER VISION-BASED NETWORKS?
Universidade Nova de Lisboa I iNOVA Media Lab
@jannajoceli ˚ thesocialplatforms.wordpress.com
Janna Joceli Omena
The University of Sheffield, 24 June 2020 I Online research seminar hosted by STeMiS and the Digital Society Network.

4. COMPUTER VISION

6. REPURPOSING

7. ❏
❏ The definitions & potentialities of computer vision API-based networks.
❏ What precedes the reading of these networks?
❏ What takes place with and in image-label & image-domain networks?
READ

8. IMAGE CLASSIFICATION
❏

10. COMPUTER VISION IMAGE-LABEL NETWORKS
❏ image-label

11. COMPUTER VISION IMAGE-LABEL NETWORKS
❏ image-label

12. Visual representation of “Portuguese” in stock image sites.
ShutterStock and Adobe Stock, January 2019.

14. Pro-impeachment protests in Brazil.
Hashtag engagement I 18 March 2016, Instagram.
Image-label network of the pro-impeachment protests.

15. The imagery of #femboy
Hashtag engagement I June 2015 - August 2017, Instagram.
Image-Label Network of #femboy

16. Image-Label Network of Microcephaly on Instagram.
mother
daughter
cool
The visuality of #microcefalia in Brazil
Hashtag engagement I June 2012 - October 2017, Instagram.

17. Images Vision API Labels

18. the rookie the amateur the expert
* Shutterstock query for austrian
this is outdoor
Microsoft
the rookie
they’re trees
IBM watson
the amateur
they’re black pines
Google Vision
the expert

22. Detecting the mood of Portuguese
Universities through Facebook
Timeline images.

26. SEARCHING & DETECTING THE SITES
❏

28. COMPUTER VISION IMAGE-DOMAIN NETWORKS
❏ image-domain

29. COMPUTER VISION IMAGE-DOMAIN NETWORKS
❏ image-domain

30. The history of “climate emergency” visuality and circulation
based on Google Image search results
2008 - July 2019

31. The circulation of bot visuality across platforms
Google Vision I Web detection I Full matching images I June 2019, Instagram

33. How does the visuality of Instagram & Tumblr
botted accounts travel across domains?

37. precedes

38. query design & grammars
data capture affordances
output file and metadata
building the network
[query]
hashtag engagement
Facebook timeline images
Video/apps/profile img thumbnails
usernames (e.g. botted accounts)
keywords (e.g. climate emergency)
[research software
or python scripts]
API calling
Scraping
output file and metadata
GEPHITABLE2NET
EXCEL & PYTHON
SCRIPTS
GOOGLE VISION API
Full matching images
Images URLS Related data

39. Culture of use
(node & edges)

41. with in

42. We see
Node size & colour
Node position
We understand

43. Isolated elements
Periphery
Mid-term
Centre
Node size I Degree Node size I in-degreeComputer Vision API-based Networks
Total number of co-occurrences of
labels used to describe images
XImage-label
(undirected graph)
Total number of co-occurrences of link
domains and images
Image-domain
based web detection - full matching
images (mixed graph)
Node size I out-degree
X
When a link domain hosts one
or more images
When an image appears in
one or more link domains
Node colour
Created attributes, e.g.
clusters, year, image host
Created attributes, e.g.
username, year or TLD, ccTLD

45. query design & grammars
data capture affordances
output file and metadata
building the network
281 visible
[purchase & list of hashtags]
460 invisible
[data analysis & co-tag nets]
[query]
botted accounts
[query]
botted accounts
241 visible
[purchase]
442 invisible
[note section & co-tag nets]
Scraper built by Jason Chao
[last 30 posts]
visible invisible total
4056 7579 11.635
visible invisible total
7082 12.093 19.175
GEPHITABLE2NETEXCEL
GOOGLE VISION API
Full matching images
Images URLS
analytical process
BIG SCREENPRINTED NETGEPHI WEB
analytical process
BIG SCREENPRINTED NETGEPHI WEB

46. BIG SCREENGEPHI OVERVIEWWEB
GEPHI DATA
LABORATORY
ANNOTATESPREADSHEET

47. image-label
Reading
image-domain
TYPE OF NETWORK WHAT WE READWHAT WE SEE
a static view of
a dynamic view of

48. Sistema de leitura de redes digitais multiplataforma
Digital Methods for Hashtag Engagement Research.
Call into the platform!
APIs de Visão Computacional: investigando
mediações algorítmicas a partir de estudo de bancos de imagens.
Cross-Platform Digital Networks:
Bots and the Black Market of Social Media Engagement
Interrogating Vision APIs
Reading Digital Networks: Climate Emergency, Bolsonaro & Bot Image Circulation by Vision API.