Lecture delivered at the Second Latin-American Summer School in Business Process Management, Bogota, Colombia, 28 June 2017 - http://ii-las-bpm.uniandes.edu.co/

1. Business Process
Monitoring and Mining
Marlon Dumas
University of Tartu, Estonia
marlon.dumas@ut.ee
II Latin-American BPM Summer School, 28/06/2017

2. 3 months later
2

3. 1. Any process is better than no process
2. A good process is better than a bad process
3. Even a good process can be improved
4. Any good process eventually becomes a bad process
• […unless continuously cared for]
• Michael Hammer
Back to basics…
3

4. 4

5. Preamble
Introduction to Process
Performance Measures

6. Process performance
If you had to choose between two services,
you would typically choose the one that is:
• F…
• C…
• B…

7. Process performance
If you had to choose between two services,
you would typically choose the one that is:
• Faster
• Cheaper
• Better

8. Process performance
Process
performance
Time
CostQuality

9. Processing
time
Waiting
time
Cycle
time
Time measures
9
Time taken by
value-adding
steps
Time between start and
completion of a process
instance
Time taken by non-
value-adding steps

10. Processing
cost
Cost of
waste
Per-
Instance
Cost
Cost measures
10
Cost of value-
adding activities
Cost of a process
instance
Cost of non-value-
adding activities

11. Material cost
•Cost of tangible or intangible
resources used per process instance
Resource cost
•Cost of person-hours employed per
process instance
Typical components of cost
11

12. Time spent
per resource
on process
work
Time
available per
resource for
process work
Resource
utilization
Resource utilization
12
Resource utilization = 60%
 on average resources are idle 40% of their allocated time

13. Resource
utilization Waiting time
Resource utilization vs. waiting time
13
Typically, when resource utilization > 90%
 Waiting time increases steeply

14. Product quality
• Defect rate = cases with positive outcome / all cases
Delivery quality
• On-time delivery rate
• Cycle time variance
Customer satisfaction
• Customer feedback score
Quality
14

15. For each performance measure, define targets
ST30 > 99%
For each objective, identify variable(s) and aggregation
method  performance measure
Variable: customer served in
< 30 min.
Aggregation method:
percentage
Measure: ST30 = % of
customers served in < 30 min.
For each process, formulate process performance objectives
Customer should be served always in a timely manner
Identifying performance measures
15

16. Financial Customer
Internal
business
process
Innovation
& learning
Organizing Performance Measures
Balanced scorecard
16
Cost measures
Quality & time
measures
Efficiency
measures
Technology
leadership,
Staff satisfaction

17. Supply Chain Operations Reference Model (SCOR)
• Performance measures for supply chain management
processes
American Productivity and Quality Council (APQC)
• Performance measures and benchmarks for processes in the
Process Classification Framework (PCF)
IT Infrastructure Library (ITIL)
• Performance measures for IT service management processes
Process performance reference
models
17

18. Consider the prescription fulfillment process of CVS
Pharmacy summarized in the handout attached
What performance measures could we define for this
process?
Teamwork

19. Business Process Monitoring
Dashboards & reports
Process miningEvent
stream
DB logs
Event
log

20. Process
Dashboards
Operational
dashboards
(runtime)
Tactical
dashboards
(historical)
Strategic
dashboards
(historical)
Types of process dashboards

21. Operational process dashboards
• Aimed at process workers & operational managers
• Emphasis on monitoring (detect-and-respond), e.g.:
- Work-in-progress
- Problematic cases – e.g. overdue/at-risk cases
- Resource load

22. • Aimed at process owners / managers
• Emphasis on analysis and management
• E.g. detecting bottlenecks
• Typical process performance indicators
• Cycle times
• Error rates
• Resource utilization
Tactical dashboards

23. Tactical Performance Dashboard
@ Australian Insurer

24. • Aimed at executives & managers
• Emphasis on linking process performance to strategic
objectives
Strategic dashboards

25. Manage
Unplanned
Outages
Manage
Emergencies &
Disasters
Manage Work
Programming &
Resourcing
Manage
Procurement
Customer
Satisfaction
0.5 0.55 - 0.2
Customer
Complaint
0.6 - - 0.5
Customer
Feedback
0.4 - - 0.8
Connection Less
Than Agreed Time
0.3 0.6 0.7 -
Key Performance
Process
Strategic Performance Dashboard
@ Australian Utilities Provider

26. Process: Manage Emergencies & Disasters
Process: Manage Procurement
Process: Manage Unplanned Outages
Overall Process Performance
Financial People
Customer
Excellence
Operational
Excellence
Risk
Management
Health
& Safety
Customer
Satisfaction
Customer
Complaint
Customer
Rating (%)
Customer
Loyalty Index
Average Time
Spent on Plan
1st Layer
Key Result
Area
2nd Layer
Key Performance
Satisfied
Customer Index
Market
Share (%)
3rd & 4th Layer
Process Performance
Measures
0.65
0.6 0.7
0.7 0.6 0.8
0.4 0.8
0.5 0.4 0.5 0.8 0.4
0.54
0.58
0.67

27. Sketch operational and tactical process monitoring
dashboards for CVS Pharmacy’s prescription
fulfillment process.
Consider the viewpoints of each stakeholder in the
process.
Teamwork

29. Business Process Monitoring
Dashboards & reports
Process miningEvent
stream
DB logs
Event
log

30. Process Mining
31
/
event log
discovered model
Discovery
Conformance
Deviance
Difference
diagnostics
Performance
input model
Enhanced model
event log’

31. Event logs structure: minimum
requirements
Concrete formats:
• Comma-Separated Values (CSV)
• XES (XML format)

32. Automated Process Discovery
33
Enter Loan
Application
Retrieve
Applicant
Data
Compute
Installments
Approve
Simple
Application
Approve
Complex
Application
Notify
Rejection
Notify
Eligibility
CID Task Time Stamp …
13219 Enter Loan Application 2007-11-09 T 11:20:10 -
13219 Retrieve Applicant Data 2007-11-09 T 11:22:15 -
13220 Enter Loan Application 2007-11-09 T 11:22:40 -
13219 Compute Installments 2007-11-09 T 11:22:45 -
13219 Notify Eligibility 2007-11-09 T 11:23:00 -
13219 Approve Simple Application 2007-11-09 T 11:24:30 -
13220 Compute Installements 2007-11-09 T 11:24:35 -
… … … …

33. Process Mining Tools
Open-source
• Apromore
• ProM
Lightweight
• Disco
Mid-range
• Minit
• myInvenio
• QPR Process
Analyzer
• Signavio Process
Intelligence
• StereoLOGIC
Discovery Analyst
Heavyweight
• ARIS Process
Performance
Manager
• Celonis Process
Mining
• Perceptive Process
Mining (Lexmark)
• Interstage Process
Discovery (Fujitsu)
34

34. Apromore.org

35. Fluxicon Disco
36

36. Process Maps
(a.k.a. Directly-Follows Graphs)
• A process map of an event log is a graph where:
• Each activity is represented by one node
• An arc from activity A to activity B means that B is directly
followed by A in at least one trace in the log
• Arcs in a process map can be annotated with:
• Absolute frequency: how many times B directly follows A?
• Relative frequency: in what percentage of times when A is
executed, it is directly followed by B?
• Time: What is the average time between the occurrence of A
and the occurrence of B?
37

37. Process Maps – Example
38
Event log:
10: a,b,c,g,e,h
10: a,b,c,f,g,h
10: a,b,d,g,e,h
10: a,b,d,e,g,h
10: a,b,e,c,g,h
10: a,b,e,d,g,h
10: a,c,b,e,g,h
10: a,c,b,f,g,h
10: a,d,b,e,g,h
10: a,d,b,f,g,h

38. Process Maps – Exercise
Case
ID Task Name Originator Timestamp
Case
ID Task Name Originator Timestamp
1 File Fine Anne 20-07-2004 14:00:00 3 Reminder John 21-08-2004 10:00:00
2 File Fine Anne 20-07-2004 15:00:00 2 Process Payment system 22-08-2004 09:05:00
1 Send Bill system 20-07-2004 15:05:00 2 Close case system 22-08-2004 09:06:00
2 Send Bill system 20-07-2004 15:07:00 4 Reminder John 22-08-2004 15:10:00
3 File Fine Anne 21-07-2004 10:00:00 4 Reminder Mary 22-08-2004 17:10:00
3 Send Bill system 21-07-2004 14:00:00 4 Process Payment system 29-08-2004 14:01:00
4 File Fine Anne 22-07-2004 11:00:00 4 Close Case system 29-08-2004 17:30:00
4 Send Bill system 22-07-2004 11:10:00 3 Reminder John 21-09-2004 10:00:00
1
Process
Payment system 24-07-2004 15:05:00 3 Reminder John 21-10-2004 10:00:00
1 Close Case system 24-07-2004 15:06:00 3 Process Payment system 25-10-2004 14:00:00
2 Reminder Mary 20-08-2004 10:00:00 3 Close Case system 25-10-2004 14:01:00
39

39. Process Maps in Disco
• Disco (and other commercial process mining tools)
use process maps to visualize event logs
• It also provides operations to filter the process
map:
• By activities
• By paths
• Let’s try it out: http://fluxicon.com/disco
40

40. Discovering (BPMN) Process Models
• Alpha miner (α-miner)
• Simple, limited, not robust
• Heuristics miner
• Robust to noise, fast, often a good tradeoff but can
produce incorrect models
• Inductive miner (ProM v6)
• Ensures that models are block-structured & correct
• Structured miner (Apromore)
• Improves heuristics miner to produce maximally block-
structured process models
41

41. α-miner
Basic Idea: Ordering relations
• Direct succession:
x>y iff for some case
x is directly followed
by y.
• Causality: xy iff
x>y and not y>x.
• Parallel: x||y iff x>y
and y>x
• Unrelated: x#y iff
not x>y and not y>x.
case 1 : task A
case 2 : task A
case 3 : task A
case 3 : task B
case 1 : task B
case 1 : task C
case 2 : task C
case 4 : task A
case 2 : task B
...
A>B
A>C
B>C
B>D
C>B
C>D
E>F
AB
AC
BD
CD
EF
B||C
C||B
ABCD
ACBD
EF
42

42. Basic Idea: Example
43

43. Basic Idea: Example
44

44. Basic Idea: Footprints
45

45. α-Algorithm
• Idea (a)
a  b


46. α-Algorithm
• Idea (b)
a b, a c and b # c


47. α-Algorithm
• Idea (c)
b d, c d and b # c


48. α-Algorithm
• Idea (d)
a b, a c and b || c


49. α-Algorithm
• Idea (e)
b d, c d and b || c


50. Process Model Discovery – Exercise
Case
ID Task Name Originator Timestamp
Case
ID Task Name Originator Timestamp
1 File Fine Anne 20-07-2004 14:00:00 3 Reminder John 21-08-2004 10:00:00
2 File Fine Anne 20-07-2004 15:00:00 2 Process Payment system 22-08-2004 09:05:00
1 Send Bill system 20-07-2004 15:05:00 2 Close case system 22-08-2004 09:06:00
2 Send Bill system 20-07-2004 15:07:00 4 Reminder John 22-08-2004 15:10:00
3 File Fine Anne 21-07-2004 10:00:00 4 Reminder Mary 22-08-2004 17:10:00
3 Send Bill system 21-07-2004 14:00:00 4 Process Payment system 29-08-2004 14:01:00
4 File Fine Anne 22-07-2004 11:00:00 4 Close Case system 29-08-2004 17:30:00
4 Send Bill system 22-07-2004 11:10:00 3 Reminder John 21-09-2004 10:00:00
1
Process
Payment system 24-07-2004 15:05:00 3 Reminder John 21-10-2004 10:00:00
1 Close Case system 24-07-2004 15:06:00 3 Process Payment system 25-10-2004 14:00:00
2 Reminder Mary 20-08-2004 10:00:00 3 Close Case system 25-10-2004 14:01:00
51

51. Limitations of alpha miner
Completeness
All possible traces of the process (model)
need to be in the log
Short loops
c>b and b>c implies c||b and b||c
instead of cb and bc
Self-loops
b>b and not b>b implies bb (impossible!)

52. Heuristic Miner
Process Map

53. Heuristic Miner

54. Heuristic Miner

55. Heuristic Miner
56
What’s the
corresponding
process model?

56. Automated Process Discovery
Automated
process
discovery
method
Simplicity
minimal size & structural
complexity
Precision
does not parse traces
not in the log
Fitness
parses the traces of the log
Generalization
parses traces of the
process not included in
the log
57

57. Accuracy of Automated Process
Discovery
58
Process
Model
Log
Lack of
fitness
Lack of
precision
Lack of
generalization

58. Process Discovery Algorithms:
The Two Worlds
High-Fitness
High-Precision
Heuristic Miner
Fodina Miner
High-Fitness
Low-Complexity

59. Process Model discovered with
Heuristics Miner

60. Process Model discovered with
Inductive Miner
• Structured by construction
• Based on process tree

61. Process Discovery Algorithms:
The Two Worlds
High-Fitness
High-Precision
Heuristic Miner
Fodina Miner
High-Fitness
Low-Complexity
Inductive Miner
Evolutionary
Tree Miner

62. Process Model discovered with
Structured Miner

63. Process Discovery Algorithms
High-Fitness
High-Precision
Low-Complexity
Structured
Miner

64. Automated Process Discovery in Apromore
Structured Miner + Subprocess Identification
Apromore.org

65. Conformance Checking
66
≠

66. Conformance Checking:
Unfitting vs. Additional Behavior
Unfitting behaviour:
• Task C is optional (i.e. may be skipped) in the log
Additional behavior:
• The cycle including IGDF is not observed in the log
Event log:
ABCDEH
ACBDEH
ABCDFH
ACBDFH
ABDEH
ABDFH

67. Conformance Checking in Apromore
68
Full demo at:
https://www.youtube.com/watch?v=3d00pORc9X8

68. Given two logs, find the differences and root causes for
variation or deviance between the two logs
Deviance & Variance Mining
≠

69. Simple claims and quick Simple claims and slow
Deviance Mining via Model Differencing
Model
Delta
Analysis
S. Suriadi et al.: Understanding Process Behaviours in a Large Insurance Company in Australia: A Case Study. CAiSE 2013

70. Deviance Mining via Log Delta Analysis
L1 - Short stay
448 cases
7329 events
L2 - Long stay
363 cases
7496 events
Log Delta Analysis
In L1, “Nursing Primary Assessment”
is repeated after “Medical Assign”
and “Triage Request”, while in L2 it is
not
…
N.R. van Beest, L. Garcia-Banuelos, M. Dumas, M. La Rosa, Log Delta Analysis: Interpretable Differencing of Business Process Event Logs.
BPM 2015: 386-405

71. Deviance Mining Demo

72. • Insurance
- Suncorp, Australia
• Government
- Qld Treasury & Trade, Australia
• Health
- AMC Hospital, The Netherlands
- São Sebastião Hospital, Portugal
- Chania Hospital, Greece
- EHR Workflow Inc., USA
• Transport
- ANA Airports, Portugal
- Busan Port, South Korea
- Kuehne + Nagel, Switzerland-Germany
• Electronics
- Phillips, The Netherlands
• Banking, construction… etc.
Process Mining: Where is it used?

73. Case Study: Suncorp Group
• General & life insurance, banking, superannuation and
investments management
• 9M customers
• 16K employees
• $85 billion in assets

74. Each process is varied by product & brand…
End to end insurance process
Source: Guidewire reference models
Total process variants: 3,000+
30
variations
500
tasks
Home      
Motor        
Commercial     
Liability     
CTP / WC      
Suncorp Insurance

75. Processing Problem
OK
OK Good
Bad Expected
Performance
Line

76. Discover and analyse actual organisational processes from data
Main result
Key patterns that explain lower performance identified
Simple Claim and Quick Simple Claim and Slow
Deviance & Variance Mining
MODEL

77. 1. Frame & Plan the Problem
2. Collect the Data
3. Analyze: Look for Patterns
4. Interpret & Create Insights
5. Create Business Impact
Wil van der Aalst, 2012
Process Mining Methodology

78. 1. Plan & Frame the Problem
• Frame a top-level question or phenomenon:
- How and why does customer experiences with our order-to-cash
processes diverge (geographically, product-wise, temporally)?
- Why does the process perform poorly (bottlenecks, slow handovers)?
- Why do we have frequent defects or performance deviance?
• Refine problem into:
- Sub-questions
- Identify success criteria and metrics
• Identify needed resources, get buy-in, plan remaining phases

79. 1. Plan & Frame the Problem – Suncorp
• Often “simple” claims take an unexpectedly long time to complete:
- What distinguishes the processing of simple claims completed on-
time, and simple claims not completed on time?
- What early predictors can be used to determine that a given “simple”
claim will not be completed on time?
• Define what a “simple” claim is
• Create awareness of the extent of the problem
Resources:
• 2 part-time Business Analysts, 1 DB Administrator, 1 Executive
Manager (sponsor)
• 1 full-time data scientist
Timeframe: 4 months

80. • Find relevant data sources
- Information systems, SAP, Oracle, BPM Systems…
- Identify process-related entities and their identifiers and
map entities to relevant processes in the process
architecture
• Extract traces
- Collect records associated with process entities
- Group records by process identifier to produce “traces”
- Export traces into standard format (XES or MXML)
• Clean
- Filter irrelevant events
- Combine equivalent events
- Filter out traces of infrequent variants if not relevant
2. Collect the data

81. 2. Collect the data: minimum requirements

82. • Discover the real process from the logs
• Calculate process metrics
- Cycle times, waiting times, error rates…
• Explore frequent paths
• Discover types of cases (good vs bad)
• Identify process deviances and early predictors
• …
3. Analyze – look for patterns

83. How likely is it that a running
process will become “deviant”?
Will it end up in
a negative
outcome?
Will it fail to
meet its SLAs in
the next 24
hours?
Will it generate
abnormal
effort, costs or
rework?
Beyond Deviance Mining:
Predictive Process Monitoring

84. Predictive Process Monitoring –
Concept
85
• Process Mining is mainly tactical
• Predictive Process Monitoring is mainly
operational

85. Predictive Process Monitoring –
Detailed View
PAGE 86
Current situation
• What is the next activity for
this case?
• When is this next activity going
to take place?
• How long is this case still going
to take until it is finished?
• What is the outcome of this
case? Is the compensation
going to be paid? Or rejected?

86. Event log
Classifier
/
Outcome
Predictions
Attributes
Traces
Predictive Process Monitoring:
General Approach
87
Event log
Regressor /
structured
predictor
Future “paths”
prediction
Attributes
Traces

87. Predictive Monitoring Example:
Debt Recovery Process
88
Debt repayment due Call the debtor Send a reminder Payment received

88. Debt repayment due Call the debtor Send a reminder Send a warning Call the debtor Call the debtor
Send to external debt
collection agency
Call the debtor
Send a reminder Send a warning Call the debtor Call the debtorCall the debtor
Call the debtor
Call the debtor
Call the debtor
Call the debtor Call the debtor
Predictive Monitoring Example:
Debt Recovery Process
89

89. Predictive Process Monitoring for
Debt Collection
90Irene Teinemaa, Marlon Dumas, Fabrizio Maria Maggi, Chiara Di Francescomarino: Predictive Business Process Monitoring with Structured
and Unstructured Data. Proc. of BPM 2016, pp. 401-417.

90. Text mining
91

91. Predictive Process Monitoring for
Debt Collection
92Classifier
Encoding of
textual dataCase attributesEvents
Will repay
in 60 days
or not?
> 80%
accuracy

92. Predictive Accuracy
• Feasibility of predictive monitoring heavily depends
on the accuracy of the predictive models
• Accuracy of an outcome-based predictive
monitoring technique measured via F-score
• Or Area Under the ROC curve (AUC) – more robust
93

93. Accuracy of outcome-based predictions
• Precision:
• Recall:
• F-Score:
• Earliness:
predicted
deviant normal
deviant TP FN
normal FP TN
actual
94

94. Predictive Accuracy
• Feasibility of predictive monitoring heavily depends
on the accuracy of the predictive models
• Accuracy of outcome-based predictive monitoring
models measured via F-score
• Or Area Under the ROC curve (AUC) – more robust
• Accuracy of KPI-based predictive monitoring
models measured via Mean Absolute Error (MAE)
• Or Root-Mean Square Error (RMSE)
95

95. Event log Nirdizati Training
Training Validation
Predictor Dashboard
Nirdizati Runtime
Enterprise Information
System
Predictions
stream
Predictive
model(s)
Event stream Event stream
Nirdizati.com
Open-Source Predictive Process Monitoring

96. Nirdizati: Model Training

97. Nirdizati: Predictive Dashboard

98. Predictive Process Monitoring:
Summary & Additional Links
99
Nirdizati https://github.com/nirdizati/
Outcome
Prediction
Clustering-based method:
http://goo.gl/ykozBf
Index-based methd:
https://goo.gl/BQFk7k
Index-based with text:
https://goo.gl/a2DoWT
Next task &
remaining
path
LTSM-based:
https://goo.gl/mkQDyy
Remaining
time
LSTM-based:
https://goo.gl/mkQDyy

99. 100