The expression and management of uncertainty, both in the information and in the operations that manipulate it, is a critical issue in those systems that work with physical environments. Measurement uncertainty can be due to several factors, such as unreliable data sources, tolerance in the measurements, or the inability to determine if a certain event has actually happened or not. In particular, this contribution focuses on the expression of one kind of uncertainty, namely the confidence on the model elements, i.e., the degree of belief that we have on their occurrence, and on how such an uncertainty can be managed and propagated through model transformations, whose rules can also be subject to uncertainty.
Expressing Confidence in Model and Model Transformation Elements
1. Expressing Confidence
in Models and in MT elements
Loli Burgueño1,2, Manuel F. Bertoa1, Nathalie Moreno1, Antonio Vallecillo1
1Universidad de Malaga, Spain
2Open University of Catalonia, Spain
October 17, 2018
1
2. Motivation
Uncertainty in Engineering Disciplines
1. Engineers naturally think about uncertainty
associated with measured values
2. Uncertainty is explicitly defined in their models
and considered in model-based simulations
3. Precise notations permit representing and
operating with uncertain values and confidences
2
3. Uncertainty (in Science and Engineering)
1. It applies to predictions of future events, estimations, physical
measurements, or unknown properties of a system, due to:
Underspecification Design U.
Lack of knowledge of the system actual behavior or underlying physics
Variability and lack of precision in measurements
Numerical approximations because values are too costly to measure
Associated properties not directly measurable/accessible (Estimations)
2. Measurement Uncertainty: A set of possible states or outcomes where
probabilities are assigned to each possible state or outcome.
3. The ISO document "Guide to the Expression of Uncertainty in
Measurement" (GUM)
describes the procedure for calculating measurement uncertainty,
as used by most Engineering Disciplines (but Software, until very recently)
[GUM] JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in measurement.
http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf
Epistemic U.
Aleatory U.
Uncertainty: Quality or state that involves imperfect and/or unknown information
3
4. Representation of Uncertainty
1) Aleatory Uncertainty (Measurement Uncertainty) [GUM]
Uncertainty of the result of a measurement 𝑥𝑥 expressed as a standard deviation 𝑢𝑢
of the possible variation of the values of 𝑥𝑥.
Representation: 𝒙𝒙 ± 𝒖𝒖 or 𝑥𝑥, 𝑢𝑢
Examples:
2) Epistemic Uncertainty (Confidence)
Predicates representing statements on the system or beliefs are assigned a value
(the level of confidence given to them)
Probability, Possibility (Fuzzy) or Plausability (Dempster-Schafer theory)
[GUM] JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in measurement.
http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf
• Normal distribution: (𝑥𝑥, 𝜎𝜎) with mean 𝑥𝑥, and
and standard deviation 𝜎𝜎
• Interval 𝑎𝑎, 𝑏𝑏 : Uniform distribution is assumed
(𝑥𝑥, 𝑢𝑢) with 𝑥𝑥 =
𝑎𝑎+𝑏𝑏
2
, 𝑢𝑢 =
(𝑏𝑏−𝑎𝑎)
2 3
4
5. Motivating example: A Surveillance System
Drones ensure that no unidentified object gets close to the area they protect
If a drone detects that an unidentified object is moving at a speed higher than
30 m/s and gets closer than 1000 m to its position the drone identifies it as a
threat, and shoots at it.
5
7. Surveillance System (Aleatory uncertainty)
Neither the metamodel nor the transformation that specifies its behavior consider:
Precision in measurements and movements
Tolerance of mechanical parts
Tolerance of shooting instruments 7
8. Surveillance System (Aleatory uncertainty)
Solution: Introduce Measurement Uncertainty (i.e., use uncertain attributes)
Real UReal represented by a pair (x, u)
Boolean UBoolean represented by a pair (b, c)
M. F. Bertoa, N. Moreno, G. Barquero, L. Burgueño, J. Troya, A. Vallecillo: “Expressing Measurement Uncertainty in
OCL/UML Datatypes.” In Proc of ECMFA 2018: 46-62, 2018.
8
10. Our research questions in this paper
How to deal with epistemic uncertainty?
In the model elements
In the model transformation that specify the system behavior
10
11. Our Contribution
We treat a special kind of (epistemic) uncertainty: confidence
Confidence refers to the quality of being certain about something
e.g., up to what extent something is true or will happen.
We assign confidence to model elements and to model transformation rules
Confidence in model elements:
degree of belief that we have on the actual existence of the entity in reality
(e.g., an event modeled in the system has indeed happened)
Confidence in model transformation rules:
Degree of belief that we have on the rule itself and on its effects
(e.g., what the rule specifies is indeed correct)
11
12. What if unreliable sources and degradation of values due to the propagation of
uncertainty, cause the transformation to
Generate objects that do not exist in reality, or
Miss the generation of objects that do exist in reality?
Surveillance System (Epistemic uncertainty)
12
13. Example: An UnidentifiedObject is shot only if:
Its confidence is higher than 0.65, and
It has not already been shot with a confidence of hitting the target higher than 0.95
Surveillance System (Epistemic uncertainty)
13
14. Surveillance System (Epistemic uncertainty)
Confidence is present in the
different phases of the rule:
Selection
Matching
Production
14
16. Confidence in Model Elements and in Model Transformations
1. Model Elements
Classes/objects
Relationships
2. Model Transformations
Rules
16
17. Confidence in Model Elements and Model Transformations
1. Model Elements
Classes/objects
Relationships
2. Model Tranformations
Rules
17
18. Performance
Implementation with LinTra
Java-based MT platform
Libraries for uncertain types have been integrated
Three case studies
1. Surveillance
2. Social media
3. Smart home
18
22. Conclusions and Future Work
We have shown how to represent and manage epistemic uncertainty in model
elements and in model transformations
Obtain more feedback on the features and scalability of our approach by means of
more and larger case studies
High-Order Transformations (HOT) to include confidence in existing MTs
Integration with MT languages such as ATL
22
23. Expressing Confidence
in Models and in MT elements
October 17, 2018
Loli Burgueño1,2, Manuel F. Bertoa1, Nathalie Moreno1, Antonio Vallecillo1
1Universidad de Malaga, Spain
2Open University of Catalonia, Spain
23