This presentation discusses and analyzes the opportunity to approach a typical TQM qualitative technique such as Root-Cause Analysis (RCA), expressed with the well-known Ishikawa (or Fishbone) diagrams, in a quantitative manner. Adding a control measure at the end of each ‘cause bone’ can help decision-makers in their determination of corrective/improvement actions to take in terms of how much to introduce into the related action plan in the way of resources. The ISO 15939 Measurement Information Model can be the appropriate technique to help derive such measures, as it overcomes the intrinsic limitations of Orthogonal Defect Classification (ODC) by providing direct process improvement at ML2 on Measurement & Analysis (ME) and at ML3 on Organizational Process Focus (OPF), in particular taking care of the specific practice SP1.4 concerning the establishment of the organization’s measurement repository), as well as on the General Practice GP2.8 (Monitor & Control the Process) across all the process areas involved in each single cause-effect analysis. The communication issue is discussed, an alternative way to express Ishikawa diagrams is proposed (Mind Maps), and how Mind Maps can facilitate the diffusion of RCA and its quantitative variant (Q-RCA) into organizations, and subsequently be used by organizations, is shown.

1. 5° Software Measurement
European Forum:
Milan, 28-30 May 2008
Strengthening CMMI Maturity Levels with a
Quantitative Approach to Root-Cause Analysis
Luigi Buglione
Engineering.it / ETS Montréal
luigi.buglione@eng.it
www.eng.it SMEF 2008 – L.Buglione © 2008

2. Goals:
Goals
 G1. Discuss and analyze the opportunity to approach RCA in a
quantitative manner
 G2. Using Q-RCA in a visual, more appealing manner using Mind
Maps, in order to overcome possible communication problems
 G3. Show some possible usages of the “visual Q-RCA” with a
discussion of expected benefits
www.eng.it SMEF 2008 – L.Buglione © 2008 2

3. Agenda
x Introduction
 A short summary about RCA
x Root-Cause Analysis (RCA) and Process Improvement
 Causal Analysis & Resolution and CMMI: high or low maturity process?
 Impacts on Process Rating and subsequent improvement actions
x Joining RCA and ‘Open’ Measurement
 Measuring RCA by ODC: some limitations
 Extending the measurable RCA scope of application
 From GQM to ISO 15939 Measurement Information Model (MIM)
x Visualizing and Communicating RCA
 SPI models and the Communication process
 Fishbone diagrams: some problems and limitations
 Mind Maps: origins & perspectives
 Possible Applications of Q-RCA using Mind Maps
x Conclusions & Prospects
www.eng.it SMEF 2008 – L.Buglione © 2008 3

4. Introduction
Some short notes on RCA
• From Staged towards Continuous representations in Maturity Models
 broader awareness about the path towards a ‘real’ process improvement
 appraisal of each single process of interest for an organization
• Most known and applied MM and RCA
 CMMI has both representations
 ISO/IEC 15504 (aka SPICE) has only the continuous representation
• Root-Cause Analysis (RCA) and Maturity Levels (ML)
 CMMI has a ML5 process area (CAR – Causal Analysis & Resolution)
 ISO 9001:2000 asks for RCA as a basic requirement to be satisfied
 Approx. equivalence between ISO 9001 certified companies and CMMI ML2-3
 Discrepancy among the RCA’s positioning in CMMI and ISO 9001
• Possible solutions
 Adopt a continuous representation in MM (no influence of staged view on PI)
 Shift down CAR-related assets/issues across levels:
− Move CAR at ML2 in MM (re-alignment of assumptions for proper appraisals)
− Move GP5.2 to GP2.x (being a basic mechanism to evolve towards higher CL)
www.eng.it SMEF 2008 – L.Buglione © 2008 4

5. Introduction
Some short notes on RCA
• Q: RCA is a typical qualitative TQM tool. Is it possible to
evolve it as a quantitative tool?
• A: possible solutions
 Make RCA more quantitative, linking it to organizational PAL’s
− PAL (Process Asset Libraries)
− Those ones typically demanded in CMMI’s OPD/OPF process
areas
 Provide some tips for spreading its use in organizations
− Communication
− Visual aids
−…
www.eng.it SMEF 2008 – L.Buglione © 2008 5

6. Agenda
x Introduction
 A short summary about RCA
x Root-Cause Analysis (RCA) and Process Improvement
 Causal Analysis & Resolution and CMMI: high or low maturity process?
 Impacts on Process Rating and subsequent improvement actions
x Joining RCA and ‘Open’ Measurement
 Measuring RCA by ODC: some limitations
 Extending the measurable RCA scope of application
 From GQM to ISO 15939 Measurement Information Model (MIM)
x Visualizing and Communicating RCA
 SPI models and the Communication process
 Fishbone diagrams: some problems and limitations
 Mind Maps: origins & perspectives
 Possible Applications of Q-RCA using Mind Maps
x Conclusions & Prospects
www.eng.it SMEF 2008 – L.Buglione © 2008 6

7. RCA & Process Improvement
CAR and CMMI: high or low maturity process?
• RCA and Total Quality Management (TQM)
 RCA is one of the ‘old’ TQM tools (more qualitative)
 aka ‘Fishbone’ or ‘Ishikawa’ diagram
 Objective: to derive, in a back-forward manner, related causes for an effect
 Visually represented as a ‘fishbone’
Elaboration from: Leszak M., Perry D.E. & Stoll D., A Case Study in Root Cause Defect
Analysis, Proceedings of the 22nd International Conference on Software Engineering (ICSE
2000), Limerick (Ireland), June 4-11 2000, ISBN 1581132069, pp.428-437
www.eng.it SMEF 2008 – L.Buglione © 2008 7

8. RCA & Process Improvement
CAR and CMMI: high or low maturity process?
• Nature of the tool
 Core analysis technique within an improvement path
 It summarizes more TQM techniques into one
− brainstorming, affinity diagram, Pareto analysis
 Foundation mechanism for establishing any action plan
• RCA and ML
 CMMI has a ML5 process area (CAR – Causal Analysis & Resolution)
 RCA is as a basic requirement to be satisfied for ISO 9001:2000 certification
 Approx. equivalence between ISO 9001 certified companies and CMMI ML2-3
 Discrepancy among the RCA’s positioning in CMMI and ISO 9001
 Opened a CR to SEI on it in 2007  partly received in the CMMI-SVC
‘Problem Management’ PA draft (ML2)
www.eng.it SMEF 2008 – L.Buglione © 2008 8

9. RCA & Process Improvement
Impacts on Process Rating and subsequent improvement actions
• Q: how can CAR and DAR be two ML apart, being so
intimately related?
• Q: Being RCA executed in any project (even if not in a
graphical notation) and evincible from any meeting report,
shouldn’t it be at ML2?
• Q: if CAR is a basic mechanism for process improvement,
shouldn’t it be positioned before statistical control
mechanisms (ML4/GP4.x)?
• A: the apparent impact on process and organizational ratings is less
than the real one, because:
 the current CMMI ‘staged’ path does not reflect a company’s priorities,
and
 CAR is relegated where it will be evaluated by few ones
www.eng.it SMEF 2008 – L.Buglione © 2008 9

10. Agenda
x Introduction
 A short summary about RCA
x Root-Cause Analysis (RCA) and Process Improvement
 Causal Analysis & Resolution and CMMI: high or low maturity process?
 Impacts on Process Rating and subsequent improvement actions
x Joining RCA and ‘Open’ Measurement
 Measuring RCA by ODC: some limitations
 Extending the measurable RCA scope of application
 From GQM to ISO 15939 Measurement Information Model (MIM)
x Visualizing and Communicating RCA
 SPI models and the Communication process
 Fishbone diagrams: some problems and limitations
 Mind Maps: origins & perspectives
 Possible Applications of Q-RCA using Mind Maps
x Conclusions & Prospects
www.eng.it SMEF 2008 – L.Buglione © 2008 10

11. Joining RCA and ‘Open’ Measurement
Measuring RCA by ODC: some limitations
• Measuring RCA by ODC
 ODC: IBM’s Orthogonal Defect Classification (R.Chillarege, 1992)
 ODC proposed a standard taxonomy of causes for software testing process by
the number of defects detected
 Pros: quantitative usage of RCA
 Limitations:
− allow benchmarks but applying such taxonomy
− not directly covered other SLC phases
www.eng.it SMEF 2008 – L.Buglione © 2008 11

12. Joining RCA and ‘Open’ Measurement
Extending the measurable RCA scope of application
• RCA, QM and GQM
 By analogy, here a comparison with a three-tier schema
 Addition of a third layer to RCA with measures will allow to quantify causes 
Q-RCA (Quantitative RCA)
Level Quality Model (QM) Root-Cause Analysis GQM
(RCA)
0 Product Effect Goal
1 Characteristic Cause Question
2 Sub-Characteristic Sub-Cause Sub-question
3 Measure (Measure) Metric (Measure)
• Possible advantages with Q-RCA
 spread the ODC lesson to all the facts of interest in a project
 less subjectivity in assigning priorities to causes for tasks in action plans
 stimulate an easier data gathering across projects and in the organization
 reduce CONQ (Cost of non Quality) in m/l term and optimize COQ (Cost of
Quality)
 easy applicable to all processes
www.eng.it SMEF 2008 – L.Buglione © 2008 12

13. Joining RCA and ‘Open’ Measurement
Extending the measurable RCA scope of application
• Q: as an intermediate goal, can Q-RCA be made automatic (and easy
to be used)?
www.eng.it SMEF 2008 – L.Buglione © 2008 13

14. Joining RCA and ‘Open’ Measurement
From GQM to ISO 15939 MIM
Goal
Implicit
models
Interpretation
Definition
Question
Q1 Q2 Q3 Q4
Metric
M1 M2 M3 M4 M5 M6 M7
• GQM (Goal-Question-Metric)
• MIM (Measurement Information Model)
www.eng.it SMEF 2008 – L.Buglione © 2008 14

15. Agenda
x Introduction
 A short summary about RCA
x Root-Cause Analysis (RCA) and Process Improvement
 Causal Analysis & Resolution and CMMI: high or low maturity process?
 Impacts on Process Rating and subsequent improvement actions
x Joining RCA and ‘Open’ Measurement
 Measuring RCA by ODC: some limitations
 Extending the measurable RCA scope of application
 From GQM to ISO 15939 Measurement Information Model (MIM)
x Visualizing and Communicating RCA
 SPI models and the Communication process
 Fishbone diagrams: some problems and limitations
 Mind Maps: origins & perspectives
 Possible Applications of Q-RCA using Mind Maps
x Conclusions & Prospects
www.eng.it SMEF 2008 – L.Buglione © 2008 15

16. Visualizing & Communicating RCA
SPI models and the Communication Process: some drawbacks…
• Communication issues in MM
 Partly addressed, even if fundamental
− CMMI GP2.7 on stakeholders’
involvement
 Heavily documented and discussed
how communication pitfalls represent a
major problem which may make a
process improvement initiative
unsuccessful
www.eng.it SMEF 2008 – L.Buglione © 2008 16

17. Visualizing & Communicating RCA
… and possible solutions
• Possible solutions to reduce resistance to change
 introduction of organizational improvement models (i.e. IDEAL) and related
analysis tools such as RCA, using it also as a communication tool
 a daily/more current use of RCA will necessarily increase the quality of
communication among the different stakeholders
 Q-RCA can improve the quantitative view on the effects to be analyzed
www.eng.it SMEF 2008 – L.Buglione © 2008 17

18. Visualizing & Communicating RCA
Fishbone diagrams: some problems & limitations
• Appraisal of appropriate resources for each process of interest
 CMMI GP2.3 (Provide Resources)
• Available types
 free-hand drawing with a presentation tool
− i.e. MS-Powerpoint; OpenOffice Impress, …
 specific drawing tools
− i.e. RCAXPress; Freemind, …
Generic drawing tools with RCA templates
− i.e. MS-Visio; SmartDraw, …
• Difficulties to connect with other org. Information Systems
• Increased effort for maintenance of RCA charts reduce its
adoption
• …which solution to satisfy all these requirements at a time?
www.eng.it SMEF 2008 – L.Buglione © 2008 18

19. Visualizing & Communicating RCA
Mind Maps: origins & perspectives
• Definition: “A mind map is a diagram used to represent words, ideas,
tasks or other items linked to and arranged radially around a central key
word or idea […]” (from Wikipedia)
• Origins
 at least from the 300BC on (i.e.
Porphyry of Tyros, a noted thinker)
 more recently proposed by
psychologist Tony Buzan
• Perspectives
 several possible usages, with the
common aim to represent concepts
grouping them by some criteria
 brainstorming, creating a hierarchy
of concepts, note-taking, etc…
www.eng.it SMEF 2008 – L.Buglione © 2008 19

20. Visualizing & Communicating RCA
From RCA to Mind Maps (1/5)
www.eng.it SMEF 2008 – L.Buglione © 2008 20

21. Visualizing & Communicating RCA
From RCA to Mind Maps (2/5)
www.eng.it SMEF 2008 – L.Buglione © 2008 21

22. Visualizing & Communicating RCA
From RCA to Mind Maps (3/5)
www.eng.it SMEF 2008 – L.Buglione © 2008 22

23. Visualizing & Communicating RCA
From RCA to Mind Maps (4/5)
www.eng.it SMEF 2008 – L.Buglione © 2008 23

24. Visualizing & Communicating RCA
From RCA to Mind Maps (5/5)
www.eng.it SMEF 2008 – L.Buglione © 2008 24

25. Visualizing & Communicating RCA
Possible applications of Q-RCA using Mind Maps
 Meeting Reports (3 pages)
www.eng.it SMEF 2008 – L.Buglione © 2008 25

26. Visualizing & Communicating RCA
Possible applications of Q-RCA using Mind Maps
 Within a wiki
www.eng.it SMEF 2008 – L.Buglione © 2008 26

27. Visualizing & Communicating RCA
Possible applications of Q-RCA using Mind Maps
 At the project closure (‘post-mortem analysis’)
The ‘measure’
bone level
Example taken from: www.scatolepensanti.it
www.eng.it SMEF 2008 – L.Buglione © 2008 27

28. Visualizing & Communicating RCA
Measurement construct with MIM: an example
Information Need Evaluate the completeness level for a project’s User Requirements (UR)
Measurable Concept Completeness of URs
Relevant Entities 1. URs produced within the current iteration
Attributes 1. Text of URs
2. List of changes to insert in the new, refined version of the URs
Base Measures 1. Number of UR to be re-worked per iteration
2. Number of URs written per iteration
Measurement Method 1. Count the number of URs to be re-worked in an iteration
2. Count the overall number of URs formulated in an iteration
Type of Measurement Method 1. Objective
2. Objective
Scale 1. Integers from zero to infinity
2. Integers from zero to infinity
Type of Scale 1. Ratio
2. Ratio
Unit of Measurement 1. URs
2. URs
Derived Measure Percentage of URs to be reworked per iteration
Measurement Function Divide the number of URs to be reworked per iteration by the total number of URs
for the iteration
Indicator UR completeness level
Model Compute thresholds using values of reworked URs per project and customer
Decision Criteria Results outside the established threshold require further investigation
See ISO/IEC 15939:2007, Appendix A
www.eng.it SMEF 2008 – L.Buglione © 2008 28

29. Agenda
x Introduction
 A short summary about RCA
x Root-Cause Analysis (RCA) and Process Improvement
 Causal Analysis & Resolution and CMMI: high or low maturity process?
 Impacts on Process Rating and subsequent improvement actions
x Joining RCA and ‘Open’ Measurement
 Measuring RCA by ODC: some limitations
 Extending the measurable RCA scope of application
 From GQM to ISO 15939 Measurement Information Model (MIM)
x Visualizing and Communicating RCA
 SPI models and the Communication process
 Fishbone diagrams: some problems and limitations
 Mind Maps: origins & perspectives
 Possible Applications of Q-RCA using Mind Maps
x Conclusions & Prospects
www.eng.it SMEF 2008 – L.Buglione © 2008 29

30. Conclusions & Prospects
• One of the foundation processes for continuous improvement is cause-
effect analysis
 Aka Root-Cause Analysis (RCA) or Ishikawa diagram
• Well-known SPI models (i.e. CMMI) positioned RCA at the highest level
 ML5 from a process viewpoint (staged view)
 GP5.2 from an appraisal viewpoint
• Direct effects from appraisal results are…
 RCA will be not often evaluated
 An underevaluation of the real capability of an organization
 It is not aligned with the ISO 9000/9001 principles & requirements
• Possible solutions
 Place RCA at ML2/GP2.x in SPI models
 Evolve RCA as a quantitative tool (Q-RCA), based on GQM principles
 Adopt RCA using a visual approach, using mind maps
• Q-RCA possible advantages
 More objective evaluation of resources for tasks in the action plan
 Reduce project stakeholders’ resistance, providing more information
 Direct linkage of Q-RCA with the adopted information systems
www.eng.it SMEF 2008 – L.Buglione © 2008 30

31. …and a bit of ‘murphology’
• COMMERCE LAW: Excess of analysis causes paralysis
•
LAW OF COMMUNICATIONS: The inevitable result of
improved and enlarged communications between different
levels in a hierarchy is a vastly increased area of
misunderstanding
• FIFTH LAW OF DECISION MAKING: Decisions are justified
by the benefits to the organization, but they are MADE by
considering the benefits to the decision-makers
• SECOND LAW OF COMMUNICATION: The information
conveyed is less important than the impression
www.eng.it SMEF 2008 – L.Buglione © 2008 31

32. Q&A
Thanks for your attention!
www.eng.it SMEF 2008 – L.Buglione © 2008 32

33. LuigiBuglione Engineering.it S.p.A.
t +39 06 83074472 Via Riccardo Morandi, 32
m +39 335 1214813 I-00148 Rome (Italy)
luigi.buglione@eng.it www.eng-it.it
www.eng.it SMEF 2008 – L.Buglione © 2008