Detecting, Measuring and Representing Vagueness in Ontologies

1. Detecting, Analyzing and Representing Vagueness in
Ontologies for Facilitating Data Reuse
Dr. Panos Alexopoulos
Semantic Applications Research Manager
iSOCO S.A.
University of Aberdeen
1/4/2014
1 l S eptember 10, 2014

2. What will I talk about
Vagueness and Ontologies
Research Questions and (some) Answers
Ongoing & Future Work

3. What will I talk about
Vagueness and Ontologies
Research Questions and (some) Answers
Ongoing & Future Work

4. Vagueness
„Vagueness is a semantic phenomenon where predicates admit
borderline cases, namely cases where it is not determinately true
that the predicate applies or not“
[Shapiro, 2006]
This happens when predicates have blurred
boundaries:
•What’s the threshold number of years
separating old and not old films?
•What are the exact criteria that distinguish
modern restaurants from non-modern?

5. Where does vagueness appear in an ontology?
Vague Classes
●A class is vague if it admits
borderline cases, i.e. if there are (or
could be) individuals for which it
is indeterminate whether they
instantiate the class.
● “TallPerson”, “StrategicClient”
Vague Datatypes
Vague Relations
●A relation is vague if there are (or could
be) pairs of individuals for which it is
indeterminate whether they stand in the
relation.
● “isExpertInSubject”, “belongsToGenre”
●Datatypes whose value range consists of a set of vague terms.
● E.g. “RestaurantPriceRange” when this comprises the terms “cheap”, “moderate” and
“expensive”.

6. Vagueness in human communication
I am telling you this is a
strategic client for the firm
with large-budget projects!
Come on, $300,000 in
two years is hardly a
large budget!

7. When is vagueness in an ontology a problem?
● Defining instances: Vagueness will cause disagreement among experts
when defining class and relation instances.
● Utilizing Vague Facts in Ontology-Based Systems: Reasoning results
might not meet users’ expectations.
● Integrating Vague Semantic Information: The merging of particular
vague elements can lead to data that will not be valid for all its users.
● Reusing Vague Datasets: The intended meaning of the dataset’s vague
elements might not be compatible to the one needed in a particular
application context.

8. What will I talk about
Vagueness and Ontologies
Research Questions and (some) Answers
Ongoing & Future Work

9. Research Questions
Can we automatically detect vague ontological
definitions?
Vagueness
Detection
Vagueness
Measurement
Vagueness
Explicitness
Can we quantify the amount and importance of
vagueness in an ontology and/or a dataset?
Can we make the intended meaning of vague
elements in ontologies more explicit than it
typically is?

10. Q1: Vagueness Detection
Problem Definition
● Can we automatically determine whether an ontology entity (class,
relation etc.) is vague or not?
● For example, “StrategicClient” as “A client that has a high value
for the company” is vague!
● “AmericanCompany” as “A company that has legal status in the
Unites States” is not!

11. Q1: Vagueness Detection
Approach
● We train a supervised classifier that may distinguish between vague
and non-vague word senses, using corresponding examples from
Wordnet.
● We use this classifier to determine whether a given ontology element
definition is vague or not.

12. Q1: Vagueness Detection
Data
● 2,000 adjective senses from WordNet.
● 1,000 vague
● 1,000 non-vague
Vague Senses Non Vague Senses
• Abnormal: not normal, not typical or usual or
regularor conforming to a norm
● Inter-agreement of vague/non-vague annotation among 3 human
judges was 0.64 (Cohen’s Kappa)
• Compound: composed of more than one part
• Impenitent: impervious to moral persuasion • Biweekly: occurring every two weeks.
• Notorious: known widely and usually
unfavorably
• Irregular: falling below the manufacturer's
standard
• Aroused: emotionally aroused • Outermost: situated at the farthest possible point
from a center.

13. Q1: Vagueness Detection
Training & Evaluation
● 80% of the data used to train a multinomial Naive Bayes classifier.
● We removed stop words and we used the bag of words assumption to
represent each instance.
● The remaining 20% of the data was used as a test set.
● Classification accuracy was 84%!

14. Q1: Vagueness Detection
Comparison with Subjectivity Analyzer
● We also used a subjective sense classifier to classify the dataset’s
senses as subjective or objective.
● From the 1000 vague senses, only 167 were classified as subjective
while from the 1000 non-vague ones 993.
● This shows that treating vagueness in the same way as subjectiveness is
not really effective.

15. Q1: Vagueness Detection
Detecting Vagueness in CiTO Ontology
● As an ontology use case we considered CiTO, an ontology that enables
characterization of the nature or type of citations.
● CiTO consists primarily of relations, many of which are vague (e.g.,
plagiarizes).
● We selected 44 relations and we had 3 human judges manually
classify them as vague or not.
● Then we applied the Wordnet-trained vagueness classifier on the
textual definitions of the same relations.

16. Q1: Vagueness Detection
Detecting Vagueness in CiTO Ontology
Vague Relations Non Vague Relations
• plagiarizes: A property indicating
that the author of the citing entity
plagiarizes the cited entity, by
including textual or other elements
from the cited entity without formal
acknowledgement of their source
• sharesAuthorInstitutionWith:
Each entity has at least one author
that shares a common institutional
affiliation with an author of the other
entity
• citesAsAuthority: The citing entity
cites the cited entity as one that
provides an authoritative
description or definition of the
subject under discussion.
• providesDataFor: The cited entity
presents data that are used in work
described in the citing entity.

17. Q1: Vagueness Detection
Detecting Vagueness in CiTO Ontology
● Classification Results:
● 82% relations were correctly classified as vague/non-vague
● 94% accuracy for non-vague relations.
● 74% accuracy for vague relations.
● Again, we classified the same relations with the subjectivity classifier:
● 40% of vague/non-vague relations were classified as
subjective/objective respectively.
● 94% of non-vague were classified as objective.
● 7% of vague relations were classified as subjective.

18. Q2: Vagueness Measurement
Problem Definition & Proposal
● Can we use metrics to quantify the existence and importance of
vagueness in ontologies and semantic data?
● E.g. can we somehow claim that ontology A is more vague than
ontology B and have a number to support this claim?
● Some initial metrics we have thought:
● Vagueness Spread
● Vagueness Intensity

19. Q2: Vagueness Measurement
Vagueness Spread
● The ratio of the number of ontological elements (classes, relations and
datatypes) that are vague, divided by the total number of elements.
● This metric reflects the extent to which vagueness is present in the
ontology.
● It also provides an indication of the ontology's potential
comprehensibility and shareability:
● An ontology with a high value of vagueness spread is less explicit
and shareable than an ontology with a low value.
● Calculation of this metric can be done semi-automatically using the
vagueness detector we have described.

20. Q2: Vagueness Measurement
Vagueness Intensity
● The degree to which the ontology's users disagree on the validity of the
(potential) instances of the ontology elements.
● The higher this disagreement is for an element, the more problems the
element is likely to cause.
● Metric Calculation:
● Consider a sample set of vague element instances,
● Have potential ontology users denote whether and to what extent
they believe these instances are valid
● Measure the inter-agreement between users (e.g. by using Cohen’s
kappa)

21. Q3: Vagueness Explicitness
Problem Definition
● Can we make the intended meaning of vague elements in ontologies
more explicit than it typically is?
● For example, ontologies typically do not explicitly state which of their
elements are vague and which are not.
● Why is this a problem?
● “Fat Person” as “A person weighing many kilos” is vague!
● “Fat Person” as “A person with a BMI greater than 25” is not!
● What are the odds that a typical ontology user will immediately see
this distinction without prior notice?

22. Q3: Vagueness Explicitness
Our Proposal: Vagueness Aware Ontologies
Vagueness-aware ontologies are “ontologies
whose vague entities are accompanied by
meta-information that describes the nature
and characteristics of their vagueness”

23. Q3: Vagueness Explicitness
What should be made explicit?
Vagueness Existence E.g. “Tall Person” is vague.
E.g. “Low Budget” has quantitative
vagueness while “Expert Consultant”
qualitative.
“E.g. “Strategic Client" is vague in the
dimension of the generated revenue”
“E.g. “Strategic Client" is vague in the
dimension of the generated revenue
in the context of Financial Reporting”
E.g. “Strategic Client" is vague in the
dimension of the generated revenue
according to the Financial Manager.
Vagueness Type
Vagueness dimensions
Applicability Context
Entity Creator

24. Q3: Vagueness Explicitness
The Vagueness Ontology

25. Vagueness Ontology
Supported Competency Questions
● Q1: What entities have been explicitly defined either as vague or non-vague?
● Q2: What entities are vague, in what contexts and according to whom?
● Q3: What entities have been defined both as vague and non-vague at the same
time and why?
● Q4: What entities of a specific type (e.g., classes) have been defined either as
vague or non-vague?
● Q5: What entities are characterised by a specific vagueness type?
● Q6: What entities have quantitative vagueness, in what dimensions and what is
the context of their dimensions (if any)?

26. Vagueness Ontology
Example Scenario
● The object property isExpertInResearchArea is considered
vague by John Doe in the context of researcher hiring.
● Moreover, it is quantitatively vague since, for him, expertise is
judged by the number of publications and projects.
● These two different dimensions he thinks are applicable in
different contexts:
● Number of relevant publications in Academia
● Number of relevant projects in Industry.

27. Vagueness Ontology
Example Scenario

28. Vagueness Ontology
Intended Usage
● Vagueness Ontology is meant to be used by both producers and
consumers of ontologies and semantic data.
● The former to annotate the vague part of their produced
ontologies with relevant vagueness metainformation
● The latter to query this metainformation and use it to make a
better use of the data.
● The annotation task should ideally take place in the course of the
ontology's construction and evolution process.

29. Vagueness Ontology
Usage and Benefits
Vagueness Ontology Usage Expected Benefits
Structuring Data
with a Vague
Ontology
• Communicate the meaning of the vague
elements to the domain experts.
• Use the metamodel to characterize the created
data's vagueness.
• Make the job of the experts
easier and faster and reduce
disagreements among them.
Utilizing Vague
Semantic Data in an
Ontology-Based
System
• Check which data is vague.
• Use the properties of the vague elements to
provide vagueness-related explanations to the
users.
• Know a priori which data
may affect the system's
effectiveness.
Integrating Vague
Semantic Datasets
• Compare same vague elements across datasets
according to their vagueness type and
dimensions
• Know a priori which data
may affect the system's
effectiveness.
Evaluating Vague
Semantic Datasets
for Reuse
• Query the metamodel to check the vagueness
compatibility of the dataset with the intended
domain and application scenario.
• Avoid re-using (parts of)
datasets that are not
compatible to own
interpretation of vagueness.

30. Vagueness Ontology
Documentation and Resources
Documentation: http://www.essepuntato.it/2013/10/vagueness/documentation.html
Examples: http://www.essepuntato.it/2013/10/vagueness/examples.html

31. What will I talk about
Vagueness and Ontologies
Research Questions and (some) Answers
Ongoing & Future Work

32. Ongoing and Future Work
Creation of Vagueness-Aware Ontologies
● Develop ontology authoring tool that:
1. Takes as input an ontology.
2. Detects automatically vague entities.
3. Guides the user into annotating them with the Vagueness
Ontology in a Q&A manner.
4. Gives as output a vagueness annotation for the given
ontology.

33. Ongoing and Future Work
Vagueness-Based Evaluation of Ontologies
● Refine/expand/evaluate the current vagueness metrics we have
defined.
● Devise methods and tools for their effective and efficient
calculation.
Reasoning with Vagueness-Aware Ontologies
● Identify vagueness-related reasoning tasks that are useful when
working with vague ontologies.
● Expand/revise the Vagueness Ontology accordingly.

34. Thank you for your attention!
Dr. Panos Alexopoulos
Semantic Applications Research Manager
Email: palexopoulos@isoco.com
Web: www.panosalexopoulos.com
LinkedIn: www.linkedin.com/in/panosalexopoulos
Twitter: @PAlexop