1. 1/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
Jan Claes
Supervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Investigating the process of process modeling
and its relation to modeling quality
The Role of Structure Serialization
2. 2/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 1 â INTRODUCTION
Research gaps
3. 3/33
www.janclaes.info
Context
ďIncreasing complexity of organizations
(globalization, customization, cost-effectiveness, âŚ)
ďProcess orientation
(efficiency, responsiveness, differentiation)
ďProcess models
(representing process steps and execution constraints)
ďProcess of Process Modeling
(translate mental image of process into formal model)
4. 4/33
www.janclaes.info
Research gaps
ď GAP 1. Need for accurate measurements
ď GAP 2. Need for detailed, yet cognitive effective
visualizations
ď GAP 3. Knowledge about how people construct
process models (=PPM)
ď GAP 4. Knowledge about relation between PPM
and model quality
ď GAP 5. Need for practical process modeling methods
ď GAP 6. Knowledge about process modeling
challenges
5. 5/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 2 â VISUALIZATION
PPMChart
6. 6/33
www.janclaes.info
Study 1 â Visualization
ďCurrent techniques
ď§ Too high-level (Modeling Phase Diagrams)
ď§ Not cognitive effective (Dotted Chart)
ďDesign method
ď§ 9 principles of cognitive effective visualization
ďEvaluation method
ď§ Qualitative evaluation with 6 academic researchers
ď§ Modeling pattern discovery
7. 7/33
www.janclaes.info
Study 1 â Visualization
ďCognitive effective visualization design principles
ď§ Visual expressiveness (maximal use of graphical variables)
ď§ Perceptual discriminability (visual matches conceptual distance)
ď§ Graphic economy (maximal six values per variable)
ď§ Dual coding (combine graphics with text)
ď§ Semiotic clarity (exactly one symbol per exactly one concept)
ď§ Semantic transparency (intuitiveness through natural mapping)
ď§ Complexity management (modularization and hierarchical structuring)
ď§ Cognitive integration (easy integration with other charts/models)
ď§ Cognitive fit (fit with task and user)
10. 10/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 3 â EXPLORATION
Relation with quality
11. 11/33
www.janclaes.info
Study 2 â Exploration
ďRelation between modeling patterns and
process model quality
ďExploration method
ď§ Compare PPMCharts with process models
ď§ Discover links
ďEvaluation method
ď§ Measure definition
ď§ Quantitative data collection
ď§ T-tests
12. 12/33
www.janclaes.info
Study 2 â Exploration
Fast
modeling
Slow
modeling
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Serialization
Chunked
modeling
Structuredness Movement Speed
Based on dataset of 40 unique modeling executions
13. 13/33
www.janclaes.info
Study 2 â Exploration
ďConjecture 1: Structured modeling
results in
understandable models
ďConjecture 2: A high number of move operations
results in
less understandable models
ďConjecture 3: Slow modeling
results in
less understandable models
14. 14/33
www.janclaes.info
ď§ Structuredness
⢠MaxSimulBlock
⢠PercNumBlockAsAWhole
ď§ Speed
⢠TotTime
⢠TotCreateTime
ď§ Movement
⢠AvgMoveOnMovedElements
⢠PercNumElementsWithMoves
Study 2 â Exploration
ďMeasurement
ď§ Model quality
⢠Perspicuity
a model that is unambiguously interpretable and can be made sound
with only small adaptations based on minimal assumptions on the
modelerâs intentions with the model
15. 15/33
www.janclaes.info
Study 2 â Exploration
T-test
t=-2,231 (p=0,028)
T-test
t=2,199 (p=0,030)
Based on dataset of 103 unique modeling executions
16. 16/33
www.janclaes.info
Study 2 â Exploration
T-test
t=-1,984 (p=0,049)
T-test
t=0,457 (p=0,648)
Based on dataset of 103 unique modeling executions
17. 17/33
www.janclaes.info
Study 2 â Exploration
T-test
t=-2,183 (p=0,031)
T-test
t=2,505 (p=0,014)
Based on dataset of 103 unique modeling executions
18. 18/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 4 â THEORISATION
Structured Process Modeling Theory (SPMT)
19. 19/33
www.janclaes.info
Study 3 â Theorization
ďExplanatory theory
ďTheory building method
ď§ 6 observations, 3 impressions (induction)
ď§ Explanation via existing theories (deduction)
ďEvaluation method
ď§ Assessment of novelty, parsimony, consistency,
plausibility, credibility, and transferability
ď§ Inconclusive empirical results, but open-world
assumption
20. 20/33
www.janclaes.info
Study 3 â Theorization
Combined
Flow-oriented Aspect-oriented
Undirected
âModeling
stylesâ
Based on dataset of 118 unique modeling executions
21. 21/33
www.janclaes.info
Study 3 â Theorization
ď Observation 1. Almost all modelers paused frequently
during the modeling process
ď Observation 2. A large group can be categorized as
âflow-oriented process modelingâ
ď Observation 3. A smaller group can be categorized as
âaspect-oriented process modelingâ
ď Observation 4. Another large group used a combination
of both former styles
ď Observation 5. Another small group can be categorized
as âundirected process modelingâ
ď Observation 6. The âundirectedâ sessions lasted longer
than the other approaches
Based on dataset of 118 unique modeling executions
22. 22/33
www.janclaes.info
Study 3 â Theorization
ď Impression 1. Modelers need serialization of the modeling
process to deal with its complexity
ď Impression 2. Structured serializing of the modeling process
helps avoiding âmistakesâ
ď Impression 3. Structured serializing does not support every
modeler to avoid âmistakesâ to the same extent
Based on dataset of 118 unique modeling executions
23. 23/33
www.janclaes.info
Study 3 â Theorization
ďCognitive Load Theory
ď§ Working memory capacity is limited
ď§ Working memory overload causes decrease in
⢠Effectiveness (i.e., more mistakes)
⢠Efficiency (i.e., more time and effort)
⢠Learning
ďCognitive Fit Theory
ď§ Load is lower when there is a fit
⢠Between representation, tool or strategy on the one hand
⢠And task or modeler on the other hand
24. 24/33
www.janclaes.info
Study 3 â Theorization
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term+ +â
input material representation fit
working memory capacity
extraneous cognitive load germane cognitive load
cognitive schema construction
process model quality overall construction time
cognitive overload
intrinsic cognitive load
++
+
+++
â
task complexity
+
prior knowledge
ââ
â â
25. 25/33
www.janclaes.info
Study 3 â Theorization
A B A determines BA B The more A, the more B+ A B The more A, the less Bâ A B A translates into B
learning style
degree of serialization
adopted serialization style
field-dependency need for structure
â +
course of intrinsic cognitive load
for process modeling phases
course of intrinsic cognitive load
for aggregation phases
course of cognitive overload
course of intrinsic cognitive load
for strategy building phases
+ + +
serialization style fitstructuredness of serialization
â ââ â
1 2 3
26. 26/33
www.janclaes.info
Study 3 â Theorization
ďNovelty (uses existing theories in fundamental new way)
ďParsimony (11 constructs, 15 associations)
ďConsistency (can explain additional observations)
ďPlausibility (accurate and profound explanation)
ďCredibility (building blocks are established theories)
ďTransferability (problem solving in general)
ďFalsifiability (inconclusive, but open-world assumed)
ďUtility (only on longer term)
Consistency based on dataset of 143 unique modeling executions
27. 27/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 5 â CONCLUSION
Summary & Future work
28. 28/33
www.janclaes.info
Studies
Research
Cycle 3
Exploration
Research Cycle 1
Structured Process Modeling Theory
(SPMT)
RC6
SPMT
measures RC5
Cognitive
measures
RC4
Perspicuity
Engineering
Cycle 1
PPMChart
RC2
Modeling
styles
RC 7. Design validation
RC 7. Research design
EC 2. Problem investigation
RC 3. Problem investigation
RC 3. Research design
RC 3. Design validation
RC 3. Evaluation
RC 4. Evaluation
RC 4. Problem investigation
RC 4. Research design
RC 4. Design validation
RC 4. Research
RC 3. Research
EC 1. Problem investigation
EC 1. Solution design
EC 1. Design validation
EC 1. Implementation
EC 1. Evaluation
RC 1. Problem investigation
RC 2. Problem investigation
RC 2. Research design
RC 2. Design validation
RC 2. Research
RC 2. Evaluation
RC 1. Research design
RC 1. Design validation
EC 2. Evaluation
RC 8. Evaluation
RC 8. Research
RC 8. Design validation
RC 8. Research design
RC 8. Problem investigation
RC 7. Evaluation
RC 7. Research
RC 7. Problem investigation
EC 2. Design validation
EC 2. Solution design
RC 1. Evaluation
RC 1. Research
RC 6. Evaluation
RC 6. Research
RC 6. Design validation
RC 6. Research design
RC 6. Problem investigation
RC 5. Evaluation
RC 5. Research
RC 5. Design validation RC 5. Research design
RC 5. Problem investigation
EC 2. Implementation
EC2
Structured Process
Modeling Method
(SPMM)
RC8
Training
RC7
Influenceability
of method
29. 29/33
www.janclaes.info
Studies
Study 1. Visualization
⢠EC1. How can the operations of the process of process modeling
be presented in a cognitive effective and efficient way?
⢠RC2. How do people construct process models in terms of
modeling styles?
PPMChart
Research instrument
(visualization)
Study 3. Theorization
⢠RC1. Why do people struggle with the complexity of process
modeling?
⢠RC2. How do people construct process models in terms of
modeling patterns?
SPMT
Theory â type II
(explanation)
Study 2. Exploration
⢠RC3. How are process and product of modeling related?
⢠RC4. How to measure (syntax) errors with cognitive origin?
⢠RC2. How do people construct process models in terms of
modeling patterns?
Process vs. product
Conjectures
(exploration)
GAP 2
GAP 3
GAP 6
GAP 3
GAP 4
GAP 1
GAP 3
30. 30/33
www.janclaes.info
Future work
Study 5. Tool support
⢠EC3. How to support measurement of cognitive profile?
⢠EC4. How to support measurement of modeling effectiveness
and efficiency?
⢠EC5. How to support the SPMM
SPMTool
Tool support
(implementation)
GAP 1
GAP 1
GAP 5
Study 4. Method
⢠EC2. How to create process models in an effective and efficient
way?
⢠RC7. Is it possible to change a modelerâs approach towards
process modeling?
⢠RC8. How to transform the SPMT into a prescriptive theory?
SPMM
Practical method
(prescription)
GAP 5
GAP 6
GAP 5
31. 31/33
www.janclaes.info
Key publications
ďPublications in international journals
ď§ Indexed by Web Of Science
⢠J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, A visual
analysis of the process of process modeling, Information Systems and e-Business
Management, Vol 13(1), p. 147-190, 2015.
ď§ Under review
⢠J. Claes, I. Vanderfeesten, F. Gailly, P. Grefen, G. Poels, The Structured Process
Modeling Theory (SPMT) A cognitive view on why and how modelers benefit from
structuring the process of process modeling, resubmitted after revision to Information
Systems Frontiers.
32. 32/33
www.janclaes.info
Key publications
ďPublications in international conference
proceedings
ď§ Indexed by Web Of Science
⢠J. Claes, I. Vanderfeesten, H.A. Reijers, J. Pinggera, M. Weidlich, S. Zugal, D. Fahland,
B. Weber, J. Mendling, G. Poels, Tying Process Model Quality to the Modeling Process:
The Impact of Structuring, Movement, and Speed, Proc. BPM '12, LNCS 7481,
Springer, 2012, p. 33-48.
⢠J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, Visualizing the
Process of Process Modeling with PPMCharts, Proc. BPM '12 Workshops, LNBIP 132,
Springer, 2012, p. 744-755.
⢠J. Claes, F. Gailly, G. Poels, Cognitive Aspects of Structured Process Modeling, Proc.
CAiSE '13 Workshops, LNBIP 148, Springer, p. 168-173, 2013.
33. 33/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
Thanks for your attention!
Do you have any questions?
Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium
Editor's Notes
Moody, D. L. (2009). The âPhysicsâ of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering. Software Engineering, IEEE Transactions on, 35(6), 756â779.
Moody, D. L. (2009). The âPhysicsâ of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering. Software Engineering, IEEE Transactions on, 35(6), 756â779.
WMC capacity: Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63(2), 81â97.
CLT: Sweller, J., Van MerriĂŤnboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251â296.
CFT: Vessey, I., & Galletta, D. (1991). Cognitive Fit: An Empirical Study of Information Acquisition. Information Systems Research, 2(1), 63â84.