2. WIMMICS TEAM
▪ Inria
▪ CNRS
▪ University Côte D’Azur (UCA)
I3S
Web-Instrumented Man-Machine Interactions,
Communities and Semantics
3. MULTI-DISCIPLINARY TEAM
▪ 35~55 members
▪ ~15 nationalities
▪ 1 DR, 4 Professors
▪ 3CR, 3 Assistant professors
DR/Professors:
▪ Fabien GANDON, Inria, AI, KRR, Semantic Web, Social Web, K. Graphs
▪ Nhan LE THANH, UCA, Logics, KR, Emotions, Workflows, K. Graphs
▪ Peter SANDER, UCA, Web, Emotions
▪ Andrea TETTAMANZI, UCA, AI, Logics, Evo, Learning, Agents, K. Graphs
▪ Marco WINCKLER, UCA, Human-Computer Interaction, Web, K. Graphs
CR/Assistant Professors:
▪ Michel BUFFA, UCA, Web, Social Media, K. Graphs
▪ Elena CABRIO, UCA, NLP, KR, Linguistics, Q&A, Text Mining, K. Graphs
▪ Olivier CORBY, Inria, KR, AI, Sem. Web, Programming, K. Graphs
▪ Catherine FARON-ZUCKER, UCA, KR, AI, Semantic Web, K. Graphs
▪ Damien GRAUX, Inria, Linked Data, Sem. Web, Querying, K. Graphs
▪ Serena VILLATA, CNRS, AI, Argumentation, Licenses, Rights, K. Graphs
Research engineer: Franck MICHEL, CNRS, Linked Data, Integration, DB, K. Graphs
External:
▪ Andrei Ciortea (University of St. Gallen) Agents, WoT, Sem. Web, K. Graphs
▪ Nicolas DELAFORGE (Mnemotix) Sem. Web, KM, Integration, K. Graphs
▪ Alain GIBOIN, (Retired CR Inria), Interaction Design, KE, User & Task, K. Graphs
▪ Freddy LECUE (Thales, Montreal) AI, Logics, Mining, Big Data, S. Web , K. Graphs
5. CHALLENGES
typed graphs to analyze,
model, formalize and
implement social semantic
web applications for
epistemic communities
multidisciplinary approach for analyzing and modeling
▪the many aspects of intertwined information systems
▪communities of users and their interactions
formalizing and reasoning on these models using typed graphs
▪new analysis tools and indicators
▪new functionalities and better management
6. WEB GRAPHS
(meta)data of
the relations
and the
resources of the
web
…sites …social …of data …of services
+ + + +…
web…
= +
…semantics
+ + + +…
= +
typed
graphs
web
(graphs)
networks
(graphs)
linked data
(graphs)
workflows
(graphs)
schemas
(graphs)
7. URI, IRI, URL, HTTP URI
CONTRIBUTE TO DATA AND SCHEMATA STANDARDS ON THE WEB
JSON
RDF
JSON LD
N-Triple
N-Quad
Turtle/N3
TriG
RDFS
OWL
SPARQL
XML
HTML
RDF XML
HTTP
Linked Data
CSV-LD R2RML
GRDDL
RDFa
SHACL
LDP
9. The four research axes of Wimmics
-
contributing to research in AI and
Semantic Web along 4 axes:
1. Web-based user modeling and interaction
design
2. Social interactions and content analysis on
the Web
3. Knowledge extraction and representation
for and by linked data on the Web
4. Web-oriented and Web-dedicated artificial
intelligence algorithms
G2 H2
G1 H1
<
Gn Hn
12. PUBLISHING
▪ extract data (content, activity…)
▪ provide them as linked data
DBPEDIA.FR (extraction, end-point)
180 000 000 triples
models
Web architecture
[Cojan, Boyer et al.]
13. PUBLISHING
DBpedia.fr usage
number of queries per day
70 000 on average
2.5 millions max
185 377 686 RDF triples extracted and mapped
public dumps, endpoints, interfaces, APIs…
22. DBPEDIA & STTL
declarative transformation
language from RDF to text
formats (XML, JSON, HTML,
Latex, natural language, GML,
…) [Cojan, Corby, Faron-Zucker et al.]
23. COVID ON THE WEB
[Corby, Michel, Gazzotti, Winckler, et al. 2019]
▪ integrate multiple datasets in heterogeneous formats
▪ perform information extraction to enrich
▪ perform inferences and validation to improve
▪ provide a public end-point for reuse
▪ provide querying and visualization services
24. vs. use cases…
• Scenario 1: Help clinicians analyze clinical trials and take evidence-based decisions
• Scenario 3: Help missions heads from Cancer Institute elaborate research programs
to study the links between cancer and coronavirus
121
[Giboin, et al.]
25. COVID ON THE WEB
RDF
translator
Jupyter Notebook
Python, R & analytics
Corese
engine
query
&
infer
ACTA Web application
visualization of argument graphs
Corese portal
Data browsing
MGExplorer
Data visualization
Open Data publication
Zenodo, Github, Virtuoso
Named
Entities
extractor
s
Covid-on-the-Web
dataset
LOD
ACTA pipeline
extraction of argument graphs
1
1
2
2
1
3
3
3
vocabularies
& datasets
3
3
3
2
Covid-19
Open
Research
Dataset
Process data &
derive “smarter” data
Means to exploit data
[Michel, Gazzotti, Gandon et al.]
26. COVID ON THE WEB
Biomedical
researchers
& managers
Data analysts
…
RDF
translator
Jupyter Notebook
Python, R & analytics
Corese
engine
query
&
infer
ACTA Web application
visualization of argument graphs
Corese portal
Data browsing
MGExplorer
Data visualization
Open Data publication
Zenodo, Github, Virtuoso Applications
Named
Entities
extractor
s
dereference, query, download
query, browse, analyze, make sense
Covid-on-the-Web
dataset
LOD
ACTA pipeline
extraction of argument graphs
1
1
2
2
1
3
3
3
vocabularies
& datasets
3
3
3
2
Covid-19
Open
Research
Dataset
Process data &
derive “smarter” data
Means to exploit data Biomedical research
[Michel, Gazzotti, Gandon et al.]
27. Dataset description No. RDF triples
dataset description + definition of a few properties 170
articles metadata (title, authors, DOIs, journal etc.) 3 722 381
named entities identified by Entity-fishing in articles titles/abstracts 35 049 832
named entities identified by Entity-fishing in articles bodies 1 156 611 321
named entities identified by Bioportal Annotator in articles titles/abstracts 104 430 547
named entities identified by DBpedia Spotlight in articles titles/abstracts 65 359 664
argumentative components and PICO elements by ACTA from articles titles/abstracts 7 469 234
Total 1 361 451 364
125
31. PREDICT STUDENTS
▪ a model of the students' learning
▪ predict success or failure to questions
▪ features from KG representations
▪ Logistic Regression (LR) / Factorization Machines
(FM) / Deep Factorization Machines (DeepFM)
[Rodriguez-Rocha, Faron, Ettorre, Michel et al. 2020]
Answers
Questions
s: students identifiers
q: questions identifiers
r: responses identifiers
a: number of attempts
w: number of wins
T: questions text embeddings
Q: graph embeddings of the questions
R: graph embeddings of the answers
e: extra group of calculated features:
question_difficulty,student_ability,
student_ability_progressive,
student_ability_progressive_question_difficulty
Features
32. EDUMICS
▪ Ontology EduProgression: OWL modeling of scholar program
▪ Ontology RefEduclever: new education referential for Educlever
▪ Migration and persistence in graph databases
▪ Reasoning, query, interactions, recommendation
[Fokou, Faron et al. 2017]
34. QUESTION ROUTING
▪ emails to the customer service (eg 350000/day “Crédit Mutuel”)
▪ detect topics in order to “understand” a question
▪ 3 humans annotate 142 questions (Krippendorff’s Alpha 0,70)
▪ NLP and semantic processing for features extraction
▪ ML performance comparison for question classification
Naive Bayes, Sequential Minimal Optimisation (SMO),
Random Forest, RAndom k-labELsets (RAkEL)
[Gazzotti, et al. 2017]
NE
recognition
(L,T)
Removing
special
characters
Tokenization
(L,T)
Spell
Checking
(L,T)
Lemmatization
(L)
Vector
generation
BOW/N-gram
Replacement in documents
Consider as feature
Input
Document
ML
workflow
L: Language dependent - T: Text dependent
Unbalanced Topics
Metrics uni uni⨁bi uni+bi+tri uni⨁NE syn syn⨁hyper syn⨁NE
Hamming Loss 0,0381 0,0370 0,0374 0,0373 0,0399 0,0412 0,0405
36. SCIENTIFIC HERITAGE
▪ TAXREF Vocabulary
▪ Data extraction and
publication
[Tounsi, Callou, Michel, Pajo, Faron Zucker et al.]
37. rr:objectMap
1
1
0-1
0-1
1
0-1
0-1
0-1
0-1
1
1
rr:GraphMap
rr:graphMap
0-1
xrr:logicalSource
xrr:LogicalSource
xrr:query
Query String
rml:iterator Iteration pattern
rr:IRI, rr:BlankNode,rr:Literal,
xrr:RdfList, xrr:RdfBag,
xrr:RdfSeq, xrr:RdfAlt
reference expr.
xrr:nestedTermMap
xrr:NestedTermMap
rr:inverseExrpression
xrr:reference
reference expr.
reference expr.
rr:ObjectMap
HETEROGENEITY
xR2RML mapping language
and SPARQL query rewriting
[Michel et al.]
<AbstractQuery> ::= <AtomicQuery> | <Query> |
<Query> FILTER <SPARQL filter> | <Query> LIMIT <integer>
<Query> ::= <AbstractQuery> INNER JOIN <AbstractQuery> ON {v1, … vn} |
<AtomicQuery> AS child INNER JOIN <AtomicQuery> AS parent
ON child/<Ref> = parent/<Ref> |
<AbstractQuery> LEFT OUTER JOIN <AbstractQuery> ON {v1, … vn} |
<AbstractQuery> UNION <AbstractQuery>
<AtomicQuery> ::= {From, Project, Where, Limit}
<Ref> ::= a valid xR2RML data element reference
38. µSERVICES
Linked Data access to Web APIs.
[Michel et al.]
SPARQL Client
Service Logics
Web API
JSON-LD
Profile
SPARQL
INSERT/CONSTR
HTTP
query
JSON
response
Triple
store
SPARQL Micro-Service
(1)
(4) (2)
(3)
LD Client Web Server
(1’)
(4’)
http://example.org/photo/472495
39. LD µSERVICES
APIs as linked data
[Michel , et al.]
SPARQL micro-
service
SPARQL SD
graph
Shapes
graph
SPARQL engine
Web
API
(5)
40. LD µSERVICES
APIs as linked data
[Michel , et al.]
HTML
JDON-LD
</>
SPARQL micro-
service
(1)
(4) SPARQL query
LD-based
application
SPARQL SD
graph
Shapes
graph
SPARQL engine
Web
API
(5)
41. LD µSERVICES
APIs as linked data
[Michel , et al.]
HTML
JDON-LD
</>
SPARQL micro-
service
(1)
(4) SPARQL query
LD-based
application
SPARQL SD
graph
Shapes
graph
SPARQL engine
Web
API
(5)
&
43. UNCERTAINTY
▪ Representing uncertainty theories
▪ Publishing it with linked data
▪ Negotiating the theory over HTTP
▪ Combining uncertainty statements
[Djebri, Tettamanzi, Gandon, 2019]
44. UNCERTAINTY
publishing theories and calculi as linked data
[Djebri, Tettamanzi, Gandon, 2019]
prob:Probability a munc:UncertaintyApproach;
munc:hasUncertaintyFeature prob:probabilityValue;
munc:hasUncertaintyOperator prob:and.
prob:probabilityValue prob:and prob:multiplyProbability.
prob:Probability prob:probabilityValue
prob:and
ex:multiplyProbability
munc:hasUncertainty
Feature
munc:hasUncertainty
Operator
46. UNCERTAINTY
translate and negotiate theories
[Djebri et al 2019]
• Specify uncertainty in parameter linked to the format
• GET /some/resource HTTP/1.1
Accept:
text/turtle;uncertainty="http://example.com/Probability";q=0.8,
text/turtle;uncertainty="http://example.com/Possibility";q=0.2;
• Use uncertainty as a profile : prof-Conneg
• GET /some/resource HTTP/1.1
Accept: text/turtle;q=0.8;profile="prob:Probability",
text/turtle;q=0.2;profile="poss:Possibility"
• HEAD /some/resource HTTP/1.1
Accept: text/turtle;q=0.9,application/rdf+xml;q=0.5
Link: <http://example.com/Probability>; rel="profile" (RFC 6906)
• GET /some/resource HTTP/1.1
47. MoReWAIS Mobile Read Write Access and Intermittent to Semantic Web
France (Wimmics, Inria) – Senegal (LANI, UGB Saint-Louis) Project
explore the specificities (advantages and constraints) of mobile P2P knowledge
sharing and addressing its limitations (e.g. intermittent access, limited resources)
[Toure et al.]
&
48. MoRAI: Geographic and Semantic Overlay Network
• Three-level P2P architecture : mobile peers, super-peers and remote sources
• Random Peer Sampling (RPS) overlay +
Semantic Overlay Network (SON) +
Geographic Overlay Network (GON)
• Experimental validation/simulation
[Toure et al.]
49. CRAWLING
▪ Predict data availability
▪ Select features of URIs
▪ Learn crawling selection
(KNN/NaiveBayes/SVM)
▪ Online learning w. crawling
(FTRL-proximal algorithm)
[Huang, Gandon 2019]
50. QUERY
• automatically suggest relevant data sources to solve a query
• sets of path features: star, sink, chain
• approximate containment search: locality sensitive hashing
[Huang, Gandon 2020]
65. INTERACTION
design and evaluation
Favoris
Nouvelle recherche TEMPS
Debut test Free Jazz 24s
Free improvisation 33s
(fiche) Avant-garde 47s
John Coltrane (vidéo) 1min 28
Marc Ribot 2min11
(fiche) experimental music 2min18 2min23
Krautrock 2min31
(fiche) Progressive rock 2min37 2min39
Red (King Crimson album) 2m52 2min59
King
Crimson 3min05
(fiche) Jazz fusion 3min18
(fiche) Free Jazz 3min32 3min54
Sun Ra 4min18
(fiche) Hard bop 4min41 4min47
Charles
Mingus (vidéo) 5min29
(fiche) Third Stream (vidéo) 6min20
Bebop 7min19
Modal jazz 7min26
(fiche) Saxophone 7min51 7min55
Mel Collins
21st CenturySchizoid Band
Crimson Jazz Trio
(fiche)
King
Crimson
(fiche)
Robert
Fripp
Miles Davis
Thelonious Monk
(fiche) Blue Note Record
McCoy Tyner
(fiche) Modal Jazz
(fiche) Jazz
Chick Corea
(fiche) Jazz Fusion
Return to Forever
MahavishnuOrchestra
Shakti (band)
U.Srinivas
Bela Fleck
Flecktones
John McLaughlin (musician)
Dixie Dregs
FICHE Dixie Degs
T Lavitz
Jordan Rudess
Behold… The Arctopus
(fiche) Avant-garde metal
Unexpected
FICHE unexpected
Dream Theater
King
Crimson
(fiche) Jazz fusion
King
Crimson
TonyLevin
(fiche) Anderson Bruford Wakeman Howe
(fiche) Rike Wakeman (vidéo)
Fin test
[Palagi, Marie, Giboin et al.]
67. METHODS & CRITERIA
▪ interaction design and evaluation
▪ exploratory search process model
[Palagi, Giboin et al. 2018]
A. Define the search space
B. Query (re)formulation
C. Information gathering
D. Put some information aside
E. Pinpoint search
F. Change of goal(s)
G. Backward/forward steps
H. Browsing results
I. Results analysis
J. Stop the search session
Previous features Feature Next features
NA A B ; J
A ; F B G ; H ; I ; J
D ; E ; I C D ; E ; F ; G ; H ; J
E ; I D C ; F ; G ; J
G ; H ; I E C ; D ; F ; G ; J
C ; D ; E ; G ; H ; I F B ; H ; I ; J
B ; D ; E ; H ; I G E ; F ; H ; I ; J
B ; F ; G ; I H E ; F ; G ; ; I ; J
B ; F ; G ; H I C ; D ; E ; F ; G ; H ; J
all J NA
70. “
« a Web-Augmented Interaction (WAI) is a
user’s interaction with a system that is
improved by allowing the system to
access Web resources »
[Gandon, Giboin, WebSci17]
72. ALOOF: Web and Perception
[Cabrio, Basile et al.]
Semantic Web-Mining and Deep Vision for Lifelong Object Discovery (ICRA 2017)
Making Sense of Indoor Spaces using Semantic Web Mining and Situated Robot Perception (AnSWeR 2017)
73. ALOOF: robots learning by reading on the Web
Annie cuts the bread in the kitchen with her knife dbp:Knife aloof:Location dbp:Kitchen
[Cabrio, Basile et al.]
74. ALOOF: robots learning by reading on the Web
First Object Relation Knowledge Base:
46.212 co-mentions gave 49 tools, 14
rooms, 101 “possible location” relations,
Annie cuts the bread in the kitchen with her knife dbp:Knife aloof:Location dbp:Kitchen
[Cabrio, Basile et al.]
75. ALOOF: robots learning by reading on the Web
▪ First Object Relation Knowledge Base: 46212 co-mentions, 49 tools, 14 rooms, 101
“possible location” relations, 696 tuples <entity, relation, frame>
▪ Evaluation: 100 domestic instruments, 20 rooms, 2000 crowdsourcing judgements
▪ Shared between robots through a shared Web knowledge base
Annie cuts the bread in the kitchen with her knife dbp:Knife aloof:Location dbp:Kitchen
[Cabrio, Basile
et al. 2017]
76. ALOOF: RDF dataset about objects
[Cabrio, Basile et al.]
▪ common sense knowledge about objects: classification, prototypical locations
and actions
▪ knowledge extracted from natural language parsing, crowdsourcing,
distributional semantics, keyword linking, ...
82. SENTIMENTS
▪ sentiment recognition in search results
▪ Automatic detection of affect
▪ Automatically interplay emotions to sentiment polarity
▪ Broader sense of sentiment ( Valence, Arousal, Dominance)
▪ Use case: Brexit scenario
▪ Propaganda detection based on argumentative techniques
[Vorakitphan, Cabrio, Villata 2020]
83. OPINIONS
NLP, ML and arguments
to monitor online image
[Villata, Cabrio, et al.]
84. ARGUMENT MINING ON
CLINICAL TRIALS
▪ NLP, ML and arguments
▪ assist evidence-based medicine
▪ support doctors and clinicians
▪ identify doc. for certain disease
▪ analyze argumentative content
and PICO elements
[Mayer, Cabrio, Villata]
85. ARGUMENT MINING ON
POLITICAL SPEECHES
▪ NLP and Machine Learning.
▪ Support historians/social science scholars
▪ Analyze arguments in political speeches
▪ DISPUTool : 39 political debates,
last 50 years of US presidential
campaigns (1960-2016)
[Mayer, Cabrio, Villata]
94. FO → R GF GR
mapping modulo an ontology
car
vehicle
car(x)vehicle(x)
GF
GR
vehicle
car
O
RIF-BLD SPARQL RIFSPARQL
?x ?x
C C
List(T1. . . Tn) (T1’. . . Tn’)
OpenList(T1. . . Tn T)
External(op((T1. . . Tn))) Filter(op’ (T1’. . . Tn’))
T1 = T2 Filter(T1’ =T2’)
X # C X’ rdf:type C’
T1 ## T2 T1’ rdfs:subClassOf T2’
C(A1 ->V1 . . .An ->Vn)
C(T1 . . . Tn)
AND(A1. . . An) A1’. . . An’
Or(A1. . . An) {A1’} …UNION {An’}
OPTIONAL{B}
Exists ?x1 . . . ?xn (A) A’
Forall ?x1 . . . ?xn (H)
Forall ?x1 . . . ?xn (H:- B) CONSTRUCT { H’}
WHERE{ B’}
restrictions
equivalence no equivalence
extensions
95. FO → R GF GR
mapping modulo an ontology
car
vehicle
car(x)vehicle(x)
GF
GR
vehicle
car
O
truck
car
=
1
2
1 ,
, )
(
2
1
2
1
2
2
1
2
1
)
,
(
let
;
)
,
( t
t
t
t
t t
depth
H
c t
t
l
t
t
H
t
t c
( )
)
,
(
)
,
(
min
)
,
(
let
)
,
( 2
1
,
2
1
2
2
1 2
1
t
t
l
t
t
l
t
t
dist
H
t
t c
c H
H
t
t
t
t
c +
=
vehicle
car
O
truck
t1(x)t2(x) → d(t1,t2)< threshold
98. FO → R GD GQ
mapping modulo an ontology
Lymphoma
Cancer
Lymphoma(x) Cancer(x)
GD
GQ
Cancer
Lymphoma
O
[Corby et al.]
AI methods: knowledge graphs, ontology-based formalisms, querying, validating and reasoning
218
101. LDSCRIPT
a Linked Data Script Language
FUNCTION us:status(?x) {
IF (EXISTS { ?x ex:hasSpouse ?y }||EXISTS { ?y ex:hasSpouse ?x },
ex:Married, ex:Single) }
[Corby, Faron Zucker, Gandon, ISWC 2017]
102. SPARQL ENDPOINT ACCESS CONTROL
Protect SPARQL endpoint from hostile actions
Set of protected Features:
SPARQL Update, Load RDF data, Service clause
Set of Access Rights:
PUBLIC, PROTECTED, PRIVATE
Assign Access Rights to Features:
SPARQL Update -> PRIVATE
Service <http://fr.dbpedia.org> -> PROTECTED
Assign Access Right to User Action:
User query -> PUBLIC
PUBLIC action cannot access PRIVATE Feature
[Corby, 2021]
103. RDF & SPARQL ACCESS CONTROL
Assign Access Right to RDF triples and SPARQL
Queries
• SPARQL Query has access to subset of RDF triples
• RDF Graph extended with Access Rights
• SPARQL Interpreter extended with Access Rights
e.g.
Assign Access Rights to RDF triples according to URIs or namespaces
URI foaf:address -> PRIVATE
Namespace foaf: -> PUBLIC
select * where { ?x ?p ?y } -> PUBLIC
Query can access PUBLIC foaf:name
Query cannot access PRIVATE foaf:address
[Corby, 2021]
106. This project has received funding from the European Union's Horizon 2020
research and innovation programme under grant agreement 825619.
ONTOLOGY FOR AI ITSELF
▪ ontology and metadata of AI resources
▪ SHACL to validate AI4EU these RDF graphs
▪ online endpoint http://corese.inria.fr
▪ predefined SPARQL queries, SHACL shapes, display
[Corby et al., 2019]
107. mining interesting association rules
AI methods: clustering + community detection + dimensionality
reduction (auto-encoder) + Frequent Pattern Growth
[Cadorel, Tettamanzi]
241
[WI-IAT 2020]
108. mining interesting association rules
AI methods: clustering + community detection + dimensionality
reduction (auto-encoder) + Frequent Pattern Growth
• hidden patterns to enrich the dataset
• novel hypotheses for biomedical research
[Cadorel, Tettamanzi]
242
[WI-IAT 2020]
109. mining interesting association rules
AI methods: clustering + community detection + dimensionality
reduction (auto-encoder) + Frequent Pattern Growth
• hidden patterns to enrich the dataset
• novel hypotheses for biomedical research
• error detection in the dataset
• relevant clusters & communities for navigation
[Cadorel, Tettamanzi]
243
[WI-IAT 2020]
112. DEONTICS
Legal Rules on the Semantic Web
OWL + Named Graphs + SPARQL Rules
Named Graph (state of affair) Subject Predicate Object
http://ns.inria.fr/nrv-inst#StateOfAffairs1 Tom http://ns.inria.fr/nrv-inst#activity driving at 100km/h
http://ns.inria.fr/nrv-inst#StateOfAffairs1 Tom http://www.w3.org/2000/01/rdf-schema#label Tom
http://ns.inria.fr/nrv-inst#StateOfAffairs1 can't drive over 90km http://www.w3.org/1999/02/22-rdf-syntax-ns#type violated requirement
http://ns.inria.fr/nrv-inst#StateOfAffairs1 can't drive over 90km has for violation http://ns.inria.fr/nrv-inst#StateOfAffairs1
http://ns.inria.fr/nrv-inst#StateOfAffairs1 driving at 100km/h http://ns.inria.fr/nrv-inst#speed 100
http://ns.inria.fr/nrv-inst#StateOfAffairs1 driving at 100km/h http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://ns.inria.fr/nrv-inst#Driving
http://ns.inria.fr/nrv-inst#StateOfAffairs1 driving at 100km/h http://www.w3.org/2000/01/rdf-schema#label "driving at 100km/h"@en
Named Graph (state of affair) Subject Predicate Object
http://ns.inria.fr/nrv-inst#StateOfAffairs2 Jim http://ns.inria.fr/nrv-inst#activity driving at 90km/h
http://ns.inria.fr/nrv-inst#StateOfAffairs2 Jim http://www.w3.org/2000/01/rdf-schema#label Jim
http://ns.inria.fr/nrv-inst#StateOfAffairs2 can't drive over 90km http://www.w3.org/1999/02/22-rdf-syntax-ns#type compliant requirement
http://ns.inria.fr/nrv-inst#StateOfAffairs2 can't drive over 90km has for compliance http://ns.inria.fr/nrv-inst#StateOfAffairs2
http://ns.inria.fr/nrv-inst#StateOfAffairs2 driving at 90km/h http://ns.inria.fr/nrv-inst#speed 90
http://ns.inria.fr/nrv-inst#StateOfAffairs2 driving at 90km/h http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://ns.inria.fr/nrv-inst#Driving
http://ns.inria.fr/nrv-inst#StateOfAffairs2 driving at 90km/h http://www.w3.org/2000/01/rdf-schema#label "driving at 90km/h"@en
[Gandon et al.]
114. PREDICT HOSPITALIZATION
▪ Predict hospitalization from
Physician’s records classification
▪ Augment records data with
Web knowledge graphs
▪ Study impact on prediction
[Gazzotti, Faron, Gandon et al. 2020]
Sexe Date Cause CISP2 ... History Observations
H 25/04/2012 vaccin-antitétanique A44 ... Appendicite EN CP - Bon état général - auscult
pulm libre; bdc rég sans souffle -
tympans ok-
Element Number
Patients
Consultations
Past medical history
Biometric data
Semiotics
Diagnosis
Row of prescribed drugs
Symptoms
Health care procedures
Additional examination
Paramedical prescription
Observations/notes
55 823
364 684
187 290
293 908
250 669
117 442
847 422
23 488
11 850
871 590
17 222
56 143
(1)
(2)
PRIMEGE
115. Image Metadata Score
portrait
50350012455
C:Jocondejoconde0138m503501_d0012455-000_p.jpg
cheval:
0.999
Image Metadata Score
figure (saint Eloi de Noyon, évêque, en pied, bénédiction,
vêtement liturgique, mitre, attribut, cheval, marteau, outil :
ferronnerie)
000SC022652
C:/Joconde/joconde0355/m079806_bsa0030101_p.jpg
cheval:
0.006
MonaLIA
▪ reason & query on RDF to build training sets.
▪ transfer learning & CNN classifiers on targeted
categories (topics, techniques, etc.)
▪ reason & query RDF of results to address
silence, noise and explain
350 000 images
of artworks
RDF metadata based
on external thesauri
Joconde database from French museums
(1)
(3)
[Bobasheva, Gandon, Precioso, 2021]
(2)
116. MonaLIA 2.0 Approach
• SPARQL+RDFS+SKOS on metadata to extract training and
test subsets of images
• create labeled training and test sets including the “narrower”
categories according to Garnier Thesaurus
• create “missing” links between some categories
• balance number of training images per class
• filter out certain categories and images
• Train Multi-Label Deep Learning classifier
• select state-of-the-art pre-trained CNN model
• adapt the model to multi-label classification
• fine-tune model on artwork images
• optimize model hyperparameters for best performance
• Apply trained model and extend metadata
• run all the images through the trained classifier
• record the prediction score as RDF triples
• SPARQL on extended metadata to search the database
(Maasai & Wimmics)
117. Detecting “noise”
By querying the extended metadata for the objects with low scores we
can detect the “noise” in the represented subject annotation
Image Metadata Score
figure (saint Eloi de Noyon, évêque, en pied, bénédiction, vêtement
liturgique, mitre, attribut, cheval, marteau, outil : ferronnerie)
000SC022652
C:/Joconde/joconde0355/m079806_bsa0030101_p.jpg
cheval: 0.006
figures bibliques (Vierge à l'Enfant, à mi-corps, assis, Enfant Jésus : nu,
livre);fond de paysage (colline, cours d'eau, barque, cavalier)
000PE027041
C:/Joconde/joconde0001/m503604_90ee1719_p.jpg
cheval: 0.009
scène (satirique : Bismarck Otto von : Gargantua, repas, cheval, boisson :
vin)
5002E006121
C:/Joconde/joconde0074/m500202_atpico-g70128_p.jpg
cheval: 0.011
118. Detecting “silence”
By querying the extended metadata for the object with high scores and
without object mentioned in annotation we can detect the “silence” in the
annotation
Image Metadata Score
portrait
50350012455
C:Jocondejoconde0138m503501_d0012455-000_p.jpg
cheval: 0.999
scène historique (guerre de siège : Lawfeld, Louis XV, Saxe maréchal de,
bataille rangée)
000PE004371
C:Jocondejoconde0634m507704_79ee519_p.jpg
cheval: 0.999
figure (sainte Jeanne d'Arc, jeune fille, équestre passant, armure,
asque, épée)
M0301000355
C:Jocondejoconde0617m030106_007305_p.jpg
cheval: 0.997
119. Ranking of search results
Running the same query on the Extended Joconde database and sorting by
score gives a better result putting the image in the second place
Image Metadata Score
représentation animalière (épagneul, debout)
M0341003743
C:Jocondejoconde0534m034186_006932_p.jpg
chien: 0.994
scène (chasse : lévrier, lièvre)
M0810001165
C:Jocondejoconde0466m081003_028491_p.jpg
chien: 0.993
représentation animalière (mise à mort, gros gibier : sanglier, chasse à
courre, chien)
00000105149
C:Jocondejoconde0107m505206_oa817_p.jpg
chien: 0.990
120. Hypermedia MAS
▪ Bridging Web architecture and Multi-Agent Systems architecture
▪ Hypermedia Communities of People and Autonomous Agents
▪ Define an architectural style for Hypermedia MAS
▪ Define declarative languages and mechanisms for specifying, enacting, and
regulating interactions among people and autonomous agents in
Hypermedia MAS
▪ Develop an open-source software infrastructure for Hypermedia MAS that
enables the deployment of hybrid communities on the Web
▪ Demonstrate the deployment of prototypical hybrid communities in two
application areas: (i) Industry 4.0 and (ii) tackling online disinformation.
http://hyperagents.gitlab.emse.fr/#
125. Toward a Web of Programs
“We have the potential for every HTML document to be a
computer — and for it to be programmable. Because the thing
about a Turing complete computer is that … anything you can
imagine doing, you should be able to program.”
(Tim Berners-Lee, 2015)
128. Make the Web AI-friendly
content, links, metadata, etc.
data, knowledge, etc.
AI Web bots: chat bots, recommenders, facilitators, etc.
configuration, parameters, embeddings, services,
communication, etc.
140. WIMMICS
1. user & interaction design
2. communities & social networks
3. linked data & semantic Web
4. reasoning & analyzing
epistemic hybrid communities
linked data
usages and introspection
contributions and traces
142. WIMMICS
Web-instrumented man-machine interactions, communities and semantics
Fabien Gandon - @fabien_gandon - http://fabien.info
he who controls metadata, controls the web
and through the world-wide web many things in our world.
Technical details: http://bit.ly/wimmics-papers