This talk is in the Semantic Web research field. It is about discussing how to evaluate the relevance of statements by comparing the contexts derived from online ontologies, to the ontology under evolution. It includes an evaluation and comparison of 2 approaches: an overlap-based approach and pattern-based approach, showing how patterns outperform the base techniques. It also includes the integration of the work in the ontology evolution tool Evolva.

1. Using Ontological Contexts to Assess the
Relevance of Statements in Ontology Evolution
Fouad Zablith, Mathieu d'Aquin, Marta Sabou, Enrico Motta
Knowledge Media Institute (KMi),
The Open University, UK

2. Scenario
Domain Data:
New Concepts Ontology Evolution
Device
Component Computer
Printer Tablet
Monitor Desktop
Mouse Laptop
Computer Products
Ontology

3. Online Ontologies for Relation Discovery
Device
Component Computer
Mouse
Monitor Desktop

4. Online Ontologies for Relation Discovery
Component
Printer
Device
Component Computer
Mouse
Monitor Desktop

5. Scenario
Domain Data:
New Concepts Ontology Evolution
Device
Component Computer
Printer Tablet
Monitor Desktop
Mouse Laptop
Computer Products
Ontology

6. Scenario
Domain Data:
New Concepts Ontology Evolution
Device
Component Computer
Printer Tablet
Monitor Desktop
Mouse Laptop
Computer Products
Ontology

7. Scenario
Domain Data:
New Concepts Ontology Evolution
Device
Component Computer
Printer Tablet
Monitor Desktop
Mouse Laptop
Engine?
Computer Products
Ontology

10. Related Work
• There exist many tools for consistency checking.
However, relevance is usually left for the user
• Existing relevance techniques based on statistical
measures (e.g. TF.IDF) do not take the ontology into
consideration

11. Relevance and Context
• In cognitive science [1], it is acknowledged that:
– Information exchange between two entities requires an
agreement on the context used
– “An input is relevant to an individual when it connects with
background information he has available that yields
conclusions that matter to him”
• To assess the relevance of a statement, we need to look
at the context in which it appears
• Online ontologies can provide such a context
1. D. Sperber and D. Wilson. Relevance. 1986.

12. Contexts Generation
Device
Component Computer
Mouse
Monitor Desktop

13. Process Overview
Rel(s) = X
Statement
Context Relevance
Analysis Measure
Online ontology
Selection & Extraction

15. Overlap Based Approach
• Overlap analysis is based on checking to what extend the
statement context overlaps with the ontology context.
• In this case, the more shared concepts the contexts have
with respect to the size of the ontology, the more relevant
a statement would be.
• Overlap relevance confidence formula:

16. Overlap based limitations
• The ontology structure is not taken into consideration.
• All statements in the same context have the same
relevance confidence e.g:
Confoverlap(<proposal, subClass, Document>, OntoSem, SWRC) = 0.2535
Confoverlap(<capture, subClass, Event>, OntoSem, SWRC) = 0.2535
• Using big ontologies (e.g. Cyc) as context, would not
reflect relevance appropriately

17. Pattern Based Approach
• Identifies specific structural situations that give indication
of relevance, supported by a confidence value. For
example:

18. Pattern Based Approach
• Identifies specific structural situations that give indication
of relevance, supported by a confidence value. For
example:
Contexts: ISWC.owl vs. SWRC.owl

19. Experimental Data
• We identified the patterns based on a collection of
statements evaluated by experts in 3 domains
• We used our ontology Evolution tool Evolva to process
text documents and identify new statements to add to the
ontology
Domain Ontology Corpus # Statements
Academic SWRC: KMi News: 251
http://ontoware.org/frs/ http://
download.php/354/ swrc news.kmi.open.ac.uk/11/
updated v0.7.1.owl
Fishery Biosphere: Fishery 124
http://kmi-web06.open.ac.uk: Website:
8081/cupboard/ ontology/
http://fishonline.org/
Experiment1/biosphere?rdf
Music Music: Music Blog: http:// 341
http://pingthesemanticweb.com/ blog.allmusic.com/
ontology/ mo/
musicontology.rdfs

20. Experimental Data: Experts’ Evaluation

21. Relevance Patterns

22. Pattern 2
Pattern 3
Pattern 4
Pattern 5

23. Performance Measure
Experts’ Answers Interpretation:
Relevant = 1
Don’t know = 0.5
Irrelevant = 0
Examples:
<Squid, subClass, Mollusk> = {Relevant, Relevant, Don’t Know} = 2.5
<Prawn, subClass, Arthropod> = {Relevant, Don’t know, Irrelevant} = 1.5
<Fisherman, subClass, Animal> = {Irrelevant, Irrelevant, Don’t know} = 0.5
$%%&'&()*2" 3/*42"5*/6" 1&'&()*2"
!" $%%&'&()*+&" 1&'&()*+&" #"
,-%&.-/'0" ,-%&.-/'0"

24. Datasets Experts Evaluation
Relevant Irrelevant Don’t Know
Academic 1 9 86 5
Academic 2 16 80 4
Fishery 36 58 6
Music 23 72 5

25. Results

26. What does it mean?
Overlap
Random Pattern
= Relevant s
= Irrelevant s
= Don’t know s

27. User Support

28. Conclusions and Future Work
• We provide a method to assess the relevance of statements
with respect to an ontology
• Our pattern-based approach shows promising results,
outperforming the baseline techniques
• We implemented our work in Evolva, to assess the relevance
of changes in ontology Evolution
• Next steps:
– Identifying further relevance patterns
– Improving the selection of context from online ontologies
– Devising a technique for automatic threshold detection
– Evaluating the impact on usage within the Evolva tool

29. Thank you!
@fzablith
f.zablith@open.ac.uk
http://evolva.kmi.open.ac.uk