Risk management using risk+ (v5)

1

INCREASING THE
PROBABILITY OF
PROGRAM SUCCESS
USING RISK+

Glen B. Alleman A workshop on the principles and practices of Risk+ and
Niwot Ridge LLC increasing the Probability of Program Success

A Warning
We’re going to cover a lot of material in 3 hours

2

Risk involves uncertainty. Uncertainty involves probability.

3

4 Douglas Adams, Hitchhiker's Guide to the Galaxy

MOTIVATION?

Your motivation?
Your motivation is your pay packet on Friday.
Now get on with it.

– Noel Coward, English actor, dramatist, &
songwriter (1899 – 1973)

5

 We have to know the underlying statistical behavior of the
processes driving the project
 This means cost, schedule, and technical performance
measures with probabilistic models
 We need to know how these three statistical drivers are
coupled
 What drives what?
 What are the multipliers between each random
6
variable?

Let’s start with the basics
7

Remember High School Statistics
8

The IMS is a collection of probabilistic
processes all coupled together
9

What does this really mean?
10

 In building a risk tolerant IMS,
we’re interested in the
probability of a successful
outcome…
 “What is the probability of
making a desired completion
date?”
 But the underlying statistics of
the tasks influence this
probability
 The statistics of the tasks, their arrangement in a network of
tasks and correlation define how this probability based
estimated developed.

There are real problems with those pesky
Unknowns that get in the way of progress

Imprint of a bird on our west facing family room second story
window on a bright afternoon
11 The Bird survived

The “units of measure” of Risk
12

 These classifications can be used to avoid asking the
“3 point” question for each task.
 Anchoring and Adjustment† of all estimating
processes produces a bias.
 Knowing this is necessary for credible estimates.
Classification Uncertainty Overrun
1 Routine, been done before Low 0% to 2%
2 Routine, but possible difficulties Medium to Low 2% to 5%
3 Development, with little technical difficulty Medium 5% to 10%
4 Development, but some technical difficulty Medium High 10% to 15%
5 Significant effort, technical challenge High 15% to 25%
6 No experience in this area Very High 25% to 50%
† Tversky and Khanemann Anchoring and Adjustment

We’re looking for knowledge of what is going to
happen in he future, with a known level of confidence

13
Harvard main library

15 What is Monte Carlo Simulation?
With some principals behind us, let’s see how to use
Risk+ to address the problem of forecasting the future of
schedule and cost performance.

A Quick Look At Monte Carlo
16

George Louis Leclerc,
Comte de Buffon, asked
what was the probability
that the needle would fall
across one of the lines,
marked here in green.
That outcome will occur
only if A  l sin

17
Los Alamos Science, Special Issue 1987

Monte Carlo Simulation
18

 Monte Carlo Simulation is named
after the city, in Monaco, of casinos
on the French Rivera.
 Monte Carlo …
 Examines all paths not just the critical
path.
 Provides an accurate (true) estimate
of completion:
 Overall duration distribution
 Confidence interval (accuracy
range)
 Sensitivity analysis of interacting tasks
 Varied activity distribution types – not restricted to a single distribution
 Schedule logic can include branching – both probabilistic and conditional
 When resource loaded schedules are used – provides integrated cost and
schedule probabilistic model.

20 What Are We Really After?
We need to answer the question …
What is the confidence we will complete “on or
before” at date and “at or below” at cost?
This is the question that should be asked and
answered on a periodic basis.
We need to have Schedule and Cost margin to
protect the deliverables and our Budget At
Completion.

Here is some advice on how
to depict this margin and
where to place this margin.
No matter how we show
manage these two elements
in the IMS, if we don’t have
margin we are late and
over budget before we
start.

http://www.ndia.org/Divisions/Divisions/Procurement/Documents/PMSCommittee/CommitteeDocuments/
21
WhitePapers/NDIAScheduleMarginWhitePaperFinal-2010(2).pdf

Confidence levels for margin change
as the program proceeds
22

As the program
proceeds we
want to have
 Increased
accuracy
 Reduced
schedule risk
 Increasing
visual
confirmation Current Estimate Confidence

that success
can be
reached

Our REAL goal here is to Manage
Margin using probabilistic models
23

 Programmatic Margin is  Margin that is not used in the
added between Development, IMS for risk mitigation will be
Production and Integration & moved to the next sequence
Test phases of risk alternatives
 Risk Margin is added to the  This enables us to buy back
IMS where risk alternatives schedule margin for activities
are identified further downstream
 This enables us to control the
Downstream
ripple effect of schedule shifts
Plan B
Duration of Plan B < Plan A + Margin Activities shifted to
left 2 days
on Margin activities
Plan B
3 Days Margin Used

Plan A
5 Days Margin

First Identified Risk Alternative in IMS Plan A 5 Days Margin

Second Identified Risk 2 days will be added
to this margin task
Alternative in IMS to bring schedule
back on track

Sensitivity Analysis
24

 The schedule sensitivity of a task measures the
closeness with which change in the task duration
matches change in the project duration over the
simulation.
 This closeness is the correlation between changes in
individual activities and their impacts on other
activities.
 A task with high schedule sensitivity is more likely to
be a major driver of the project duration than a
lower ranked task.

: Models of the Schedule

Task Criticality Analysis
25

 A measure of the frequency that an activity in the
project schedule is critical (Total Float = 0) in a
simulation
 If a task is critical in 500 of the 1,000 iterations of
the simulation, it has a Criticality Index of 0.5
 The higher the criticality index, the more certain it is
that the task will always be critical in the project


Cruciality shows each task’s tolerance
to risk
26

 Cruciality = Schedule Sensitivity x Criticality
 Schedule Sensitivity can be statistically misleading:
A task with high sensitivity may not be on or near the
critical path.
 Thus a reduction in that task’s duration may have little
effect on the project duration.
 Cruciality sharpens the analytical focus:
 It highlights critical or near–critical activities with high.
 Schedule Sensitivity
 These tasks are most likely to drive project duration.


Guiding the Risk Factor Process means
weighting each level of risk
27

 For tasks marked “Low” a reasonable
approach is to score the maximum
Min Most Max
10% greater than the minimum. Likely
 The “Most Likely” is then scored as a
geometric progression for the Low 1.0 1.04 1.10
remaining categories with a common Low+ 1.0 1.06 1.15
ratio of 1.5
Moderate 1.0 1.09 1.24
 Tasks marked “Very High” are bound Moderate+ 1.0 1.14 1.36
at 200% of minimum.
High 1.0 1.20 1.55
 No viable project manager would like
a task grow to three times the planned High+ 1.0 1.30 1.85
duration without intervention Very High 1.0 1.46 2.30
 The geometric progress is somewhat Very High+ 1.0 1.68 3.00
arbitrary but it should be used instead
of a linear progression
: Examples of Monte Carlo

Progressive Risk Factors
28

 A geometric progression (1.534) of risk can be
used.
 The phrases associated with increasing risk have
been shown at the Naval Research Laboratory to
correlate with an engineers “sense” of increasing
risk.


Risk Factor Attributes
29

 The “narrative” for each risk
factor needs to be developed.
 Each description is dependent
on…
 Discipline
 Program stage
 Complexity
 Historical data
 Current “risk state” of the program

 This is currently missing from our
efforts to quantify schedule and
cost risk.

Accuracy
30

 Given a specified final cost or project duration, what is
the probability of achieving this cost or duration?
 Frequentist approach
 Over many different projects, four out of five will cost less
or be completed in less time than the specified cost or
duration.
 Bayesian approach
 We would be willing to bet at 4 to 1 odds that the project
will be under the 80% point in cost or duration.
 Accuracy is needed to plan reserves.
 Accuracy is needed when comparing competing
proposals.

: What is the Purpose of Project Risk Analysis?

Structured Thinking
31

 All estimates will be in error to some degree of
variance.
 Trying to quantify these errors will result in bounds too
wide to be useful for decision making.
 Risk analysis should be used to
 Think about different aspects of the project
 Try to put numbers against probabilities and impacts
 Discuss with colleagues the different ideas and perceptions

 Thinking things through carefully results in
 Which elements of the programmatic and technical risk are
represented in the IMS.
 The process becomes more valuable than the numbers.
: What is the Purpose of Project Risk Analysis?

To properly use Schedule Margin†
32

 Work must be represented in single units – either
task or work packages.
 The overall schedule margin must be related to the
variation of individual units of work.
 The importance of the units of work must be shared
among all participants (ordinal ranking of work and
its risk).
 The schedule must be reasonable in some units of
measure shared by all the participants.
† “Protecting Earned Value Schedules with Schedule Margin,” Newbold, Budd, and Budd,
http://www.prochain.com/pm/articles/ProtectingEVSchedules.pdf

Let’s Apply a Monte Carlo Simulation Tool
The Monte Carlo trolley, or FERMIAC,
was invented by Enrico Fermi and
constructed by Percy King. The drums
on the trolley were set according to
the material being traversed and a
random choice between fast and slow
neutrons.
Another random digit was used to
determine the direction of motion,
and a third was selected to give the
distance to the next collision. The
trolley was then operated by moving
it across a two dimensional scale
drawing of the nuclear device or
reactor assembly being studied.
The trolley drew a path as it rolled,
stopping for changes in drum settings
whenever a material boundary was
crossed. This infant computer was
used for about two years to
determine, among other things, the
change in neutron population with
33 time in numerous types of nuclear
systems.

35 A Small Diversion
Most Likely Isn’t Likely to be the Most Likely

When we say “most likely” what do we think this
actually means?
If you pick the wrong meaning, your Monte
Carlo model will be seriously flawed.

The problem with “Most Likelies”
36

 For each activity the “best” estimate is …
 The “most likely” duration – the mode of the distribution of
durations? (Mode is the number that appears most often)
 It’s 50th percentile duration – the median of the distribution?
(Median is the number in the middle of all the numbers)
 It’s expected duration – the mean of the distribution? (Mean
is the average of all the numbers)
 These definitions lead to values that are almost always
different from each other.
 Rolling up the “best” estimate of completion is almost
never one of these.

Durations are Probability Estimates not
Single Point Values
37

 We know this because…
 “Best” estimate is not the only possible estimate, so other
estimates must be considered “worse.”
 Common use of the phrase “most likely duration” assumes
that other possible durations are “less likely.”
 “Mean,” “median,” and “mode” are statistical terms
characteristic of probability distributions.
 This implies activity distributions have probability
distributions
 They are random variables drawn from the probability
distribution function (pdf).
 “Actual” project duration is an uncertain quality that can
be modeled as a sum of random variables
 The pdf may be known or unknown.

3 Task Most Likely ≠ Project Most Likely
38

 PERT assumes
probability
distribution of the
project times is the
same as the tasks
on the critical
path.
 Because other
paths can become
critical paths, PERT
consistently
underestimates the
project completion
time. 1+1=3
: Managing Uncertainty in the IMS

Probability Distribution Function is the
Lifeblood of good planning
39

 Probability of
occurrence as a
function of the
number of
samples.
 “The number of
times a task
duration
appears in a
Monte Carlo
simulation.”

: Managing Uncertainty in the IMS

Remember the quote about statistics
40

Lies, Damn Lies, and Statistics
– Benjamin Disraeli

But we know better, we know that
any estimate without a variance is
not trustworthy.
We know that the variances have
to be calibrated from past
performance to be credible

42 A “Real World” Schedule Analysis
One should expect that the expected
can be prevented, but the unexpected
should have been expected.
— Augustine Law XLV

This is a must own book for everyone in our business. It defines
fundamental Laws of program and business management, which
are many times ignored – like the one above

Our Starting Point
43

 Risk+ Installed
 Let’s define the
needed fields
 These are used
by Risk+ to hold
information and
run the
application.
 If there are conflicts, you can make changes in Risk+
to work around your fields.

A Simple IMS
44

By simple it means serial cascaded work efforts.

Initial Field Usage
45

 Minimum Remaining Duration
 The duration that is least you’d
expect this task to complete in
 Most Likely Remaining
Duration
 The ML (Mode) of the duration
 Maximum Remaining Duration
 The duration that is the most
you’d expect this task to
complete in
 Task Reporting ID
 The tasks we want to watch

Define a View and Table for Risk+
46

Start with the Gantt View and Entry Table
Set up both to match the Risk+ field usage

Use the default if
there are no field
conflicts

Fields used for Risk+ example
47

Let’s actually “doing something”
48

 Initialize the Most Likely.
 This sets the Most Likely
duration to the same value
that is in the “Duration” field
of your IMS.
 The “planned duration” now
becomes the ML duration.
 If this “planned duration” is
bogus then your model will be
as well.
 Choose wisely.

Now the ML = DURATION step
49

 All the DURATION
values have been
moved to the ML field.
 But remember our
discussion of the ML’s
 Choose them carefully

 The next we’ll set the
upper and lower limits
of that ML value
 Using risk factors.
 OK, 3 point estimates
if you have to.

Let’s do this the simple way
50

 Let’s pick MEDIUM
confidence.
 MEDIUM means
 –25%
 +25%
 And a NORMAL
(Gaussian) curve

Let’s have Risk+ do something for us
51

 Enter a “1” in the RPT field (Number 1)

 This marks that ROW in the schedule as a work
activity we want to see the Monte Carlo output for

Now we’re ready to run
52

 The RISK ANALYSIS command
starts the process going.
 Let’s make 200 iteration and
look at the DURARTION
ANALYSIS for the activities we
are watching.

This is nice but what actually is Risk +
doing?
53

 Risk+ is picking a random number from under the normal
distribution within the range of the
 Least remaining and most remaining
 This is not some ordinary random number it is chosen through
an algorithm called the Latin Hypercube - more on that later.
 Risk+ then plugs that number into the “real” DURATION
field and does that for all the DURATIONS in the
schedule
 Then the F9 key is pressed and the date is recorded for
the finish of UID 41.
 This is done 200 times and a histogram of all the dates
that appeared for those 200 time is recorded.

And We Get
54

Date: 11/29/2011 4:32:17 PM Completion Std Deviation: 2.06 days
Samples: 500 95% Confidence Interval: 0.18 days
Unique ID: 19 Each bar represents 1 day
Name: End Work Package 3

0.20 1.0 Completion Probability Table
0.18 0.9
Cumulative Probability

Prob Date Prob Date
0.16 0.8
0.05 Wed 3/7/12 0.55 Tue 3/13/12
0.14 0.7 0.10 Thu 3/8/12 0.60 Tue 3/13/12
Frequency

0.12 0.6 0.15 Thu 3/8/12 0.65 Tue 3/13/12
0.10 0.5 0.20 Fri 3/9/12 0.70 Wed 3/14/12
0.08 0.4 0.25 Fri 3/9/12 0.75 Wed 3/14/12
0.06 0.3 0.30 Fri 3/9/12 0.80 Wed 3/14/12
0.35 Mon 3/12/12 0.85 Thu 3/15/12
0.04 0.2
0.40 Mon 3/12/12 0.90 Thu 3/15/12
0.02 0.1 0.45 Mon 3/12/12 0.95 Fri 3/16/12
Fri 3/2/12 Mon 3/12/12 Tue 3/20/12
0.50 Mon 3/12/12 1.00 Tue 3/20/12
Completion Date

Learning to Speak in Risk+
55

 Risk +shows use the probability of finish “on or
before” a date
 It does NOT show the probability of success.
 But even the “on or before” term is loaded with
special meaning.
 It means for the 500 iterations of Risk+ using the
upper and lower bounds of the duration, drawn
from the probability density function (pdf) with the
Normal (Gaussian) shape, 60% of the finish dates
were recorded to be on or before 3/12/12.

Medium confidence for a large project
56

Unique ID: 17 Each bar represents 2 days
Name: (SA) Systems Requirements Completed

0.20 0.9

Prob Date Prob Date
0.17 0.8
0.05 Fri 5/4/12 0.55 Thu 5/17/12
0.7 0.10 Wed 5/9/12 0.60 Thu 5/17/12
Frequency

0.15
0.6 0.15 Thu 5/10/12 0.65 Fri 5/18/12
0.13
0.5 0.20 Fri 5/11/12 0.70 Mon 5/21/12
0.10 0.25 Mon 5/14/12 0.75 Mon 5/21/12
0.4
0.08
0.3 0.30 Mon 5/14/12 0.80 Tue 5/22/12
0.05 0.35 Tue 5/15/12 0.85 Wed 5/23/12
0.2
0.40 Tue 5/15/12 0.90 Thu 5/24/12
0.03 0.1 0.45 Wed 5/16/12 0.95 Mon 5/28/12
Wed 5/2/12 Wed 5/16/12 Mon 6/4/12
0.50 Wed 5/16/12 1.00 Mon 6/4/12
Completion Date

Low confidence for a large project
57

Unique ID: 17 Each bar represents 3 days
Name: (SA) Systems Requirements Completed
0.9
0.14

Prob Date Prob Date
0.8
0.12 0.05 Thu 5/3/12 0.55 Fri 5/25/12
0.7 0.10 Tue 5/8/12 0.60 Mon 5/28/12
Frequency

0.10 0.6 0.15 Wed 5/9/12 0.65 Wed 5/30/12
0.08 0.5 0.20 Mon 5/14/12 0.70 Wed 5/30/12
0.4 0.25 Tue 5/15/12 0.75 Fri 6/1/12
0.06
0.3 0.30 Thu 5/17/12 0.80 Mon 6/4/12
0.04 0.35 Fri 5/18/12 0.85 Wed 6/6/12
0.2
0.02
0.40 Mon 5/21/12 0.90 Fri 6/8/12
0.1 0.45 Wed 5/23/12 0.95 Thu 6/14/12
Tue 4/24/12 Thu 5/24/12 Wed 6/27/12
0.50 Wed 5/23/12 1.00 Wed 6/27/12
Completion Date

Let’s run some
simulations

58

60 Basic Principles of Probabilistic Cost
Now that the schedule can be produced using
probabilistic methods, it’s time to talk about the cost.
Cost does not have a linear relationship with
schedule unfortunately.

: Basic Principles of Probabilistic Cost

Basic Principles with Probabilistic Cost
Estimating are coupled with scheduling
61

 Cost estimates usually involve many CERs
 Each of these CERs has uncertainty (standard error)
 CER input variables have uncertainty (technical uncertainty)
 Must combine CER uncertainty with technical uncertainty for
many CERs in an estimate
 Usually cannot be done arithmetically; must use simulation to roll
up costs derived from Monte Carlo samples
 Add and multiply probability distributions rather than numbers
 Statistically combining many uncertain, or randomly varying, numbers
 Monte Carlo simulation
 Take random sample from each CER and input parameter, add and
multiply as necessary, then record total system cost as a single sample
 Repeat the procedure thousands of times to develop a frequency
histogram of the total system cost samples
 This becomes the probability distribution of total system cost

The Cost Probability Distributions as a
function of the weighted cost drivers
62

Combined Cost Modeling
and Technical Uncertainty

Cost = a + bXc
Cost Modeling Uncertainty

Cost
Estimate

Historical data point
$
Cost estimating relationship

Technical Uncertainty Standard percent error bounds

Cost Driver (Weight)


The Risk Adjusted Cost Estimate
Connected To The IMS
63

 In the risk–adjusted cost estimate, we now combine
discrete risk events and the uncertainty of the input
distributions with the uncertainty of the CERs
 Since the input distributions tend to be right–skewed,
the expected cost tends to be larger than the baseline
estimate
 In addition, the risk–adjusted cost distribution tends to
be wider than the baseline estimate
 The difference between the expected cost of the risk–
adjusted estimate and the expected cost of the
baseline estimate is, by definition, the amount of RISK
dollars included in the risk–adjusted estimate


Baseline versus Risk Adjusted Cost
Estimates Usually Show a Cost Increase
64

Baseline vs. Risk-Adjusted Estimates
Baseline:
Mean = $102.6M
Std Dev = $29.8M

Risk–Adjusted:
Mean = $122.6M
Likelihood

Std Dev = $42.8M

0 50 100 150 200 250 300 350
FY$M

The S–Curve for Cost Modeling
65 Cumulative Distribution Function

100%

90%

80%

80th percentile

70%
50th percentile $153.5M
60% $114.7M

50%
Baseline Estimate
Mean $102.6M Risk–adjusted
40% Estimate Mean
$122.6M
30%

20%

10%

0%
$60 $80 $100 $120 $140 $160 $180 $200
FY00$M


The Real Question Always Returns to…
“But How Much Does It Cost? Really?”
66

 This is impossible to answer precisely
 Decision–makers and cost analysts should always think
of a cost estimate as a probability distribution, NOT as
a deterministic number
 The best we can provide is the probability distribution –
If we think we can be any more precise, we’re fooling
ourselves
 It is up to the decision–maker to decide where he/she
wants to set the budget
 The probability distribution provides a quantitative
basis for making this determination
 Low budget = high probability of overrun
 High budget = low probability of overrun


68 Some More Parts to using Risk+
Just having the pictures is necessary, but knowing
what they mean is required.
Making changes to the IMS to increase the
Probability of Program Success is the primary
outcome from Monte Carlo Simulation.

Without Integrating $, Time, and TPM
you’re driving in the rearview mirror
69

Technical
Performance (TPM)

Risk Management Demands a Well Defined Process

Statistics of a Triangle Distribution
71

50% of all possible values are
under this area of the curve. This
is the definition of the median

Minimum Maximum
1000 hrs 6830 hrs
Mode = 2000 hrs Mean = 3879 hrs
Median = 3415 hrs

Basic Statistics

TPM Trends & Responses directly
impact risk and credibility of the IMS
72

Design Model
ROM in Proposal Detailed Design Model
Bench Scale Model Measurement
Technical Performance Measure

28kg
Prototype Measurement
Vehicle Weight

Flight 1st Article
26kg

25kg

23kg

CA SFR SRR PDR CDR TRR

Dr. Falk Chart – modified

Not A Mitigation Plan
Mitigation is too late, the risk has
turned into an issue. The money
has been spent, and the time has
passed.
73

Ordinal versus Cardinal
74

Ordinal Cardinal
A variable is ordinally measurable A variable is cardinally measurable
if ranking is possible for values of if a given interval between
the variable. For example, a gold measures has a consistent meaning,
medal reflects superior performance i.e., if the measure corresponds to
to a silver or bronze medal in the points along a straight line. For
Olympics, or you may prefer French example, height, output, and income
toast to waffles, and waffles to oat are cardinally measurable.
bran muffins. All variables that are
cardinally measurable are also
ordinally measurable, although the
reverse may not be true.

Correcting Ordinal Risk Scales
75

 Classify and calibrate risk ranking in units
meaningful to the decision makers
 Risk rank 1, 2, 3, 4, is NOT sufficient
 The Risk Rank must have a measurable value connected
to the actual behavior of the system being assessed
 Calibration coefficients between ordinal probability
and consequences should also be used.
 Ordinal analysis assumes ordering of the risks.
 Cardinal analysis provides objective measures of
probability and consequential impact.

Level Likelihood Value
Never multiply Likelihood E
E Near Certainty E ≥ 90%
by outcome. They are not
“numbers,” they a D Highly Likely 74% ≤ D ≤ 90% D

probability distributions. C Likely 40% ≤ C ≤ 60% C
Only convolution is
B Low Likelihood 20% ≤ B ≤ 40% B
possible
A Not Likely A ≤ 20% A

These are Cardinal measures of probability of occurrence and A B C D E
consequential impact
Level Technical Performance Schedule Cost
Minimal or no consequence to
A Minimal or no impact Minimal or no impact
technical performance.
Budget increase or unit
Minor reduction in technical
B Able to meet key dates production cost increases.
performance or supportability.
< (1% of Budget)
Moderate reduction in Minor schedule slip.
technical performance or Able to meet key
C production cost increase
supportability with limited milestones with no
< (5% of Budget)
impact on program objectives. schedule float.
Significant degradation in
technical performance or Program critical path
D production cost increase
major shortfall in affected
< (10% of Budget)
supportability.
Cannot meet key Exceeds budget increase or
Severe degradation in technical
E program milestones. unit production cost
performance. 76
Slip > X months threshold

Example of
Ordinal Probability Complexity Scale†
77

Definition of the Ordinal Scale Ranking Scale Level
Greater than 20% of the interface design has been
E
altered because of modifications to the ICD’s.
Greater than 15% but less than 20% of the interface
design has been altered because of modifications of the D
ICD’s.
Greater than 10% but less than 15% of the interface
design has been altered because of modifications of the C
ICD’s.
At least 5% but less than 10% of the interface design has
B
been altered because of modifications of the ICD’s.
At least 5% of the interface design has been altered
A
because of modifications of the ICD’s.
† Effective Risk Management: Some Keys to Success, Edmund Conrow, AIAA Press, 2003

A “real” risk Ordinal Ranking Table
78

Risk
Percent Variance Interpretation of Risk Ranking
Rank
Normal business, technical & manufacturing
A – 5% ≤ A ≤ 10%
processes are applied
Normal business & technical processes are
B – 5% ≤ B≤ 15% applied; new or innovative manufacturing
processes
Flight software development & certification
C – 5% ≤ C ≤ 35%
processes
Build & qualification of flight components,
D – 10% ≤ D ≤ 25%
subsystems & systems
E – 10% ≤ E ≤ 35% Flight software qualification
F – 5% ≤ F ≤ 175% ISS thermal vacuum acceptance testing

Project Train Wrecks Occur When There is…
 Inattention to budgetary
responsibilities
 Work authorizations that are
not always followed
 Issues with Budget and data
reconciliation
 Lack of an integrated
management system
 Baseline fluctuations and
frequent replanning
 Untimely and unrealistic Latest Revised
 Current period and retroactive
Estimates (LRE)
changes
 Progress not monitored in a regular and
 Improper use of management
consistent manner
reserve
 Lack of vertical and horizontal traceability
 EV techniques that do not
cost and schedule data for corrective action
reflect actual performance
 Lack of internal surveillance and controls
 Lack of predictive variance
 Managerial actions not demonstrated using
analysis
Earned Value 79

Our Final Check List
80

 Set up the Risk+ fields, flags, views, and tables for the
program standard IMS.
 Build an IMS that passes the DCMA 14 Point Assessment
with all GREEN.
 Build the Ordinal Risk Ranking table for the various risk
categories on the program.
 Assign risk ranking to each activities in the IMS, with the
variances defined in the Ordinal Table.
 Run Risk+ to see the confidence in the deliverables.
 Develop the needed schedule margin to protect the
delivery to at least the 80% confidence level.

Advice from the school of hard knocks
81

 Put margin in front of
critical deliverables.
 Build a margin burn
down chart and
allocate schedule
margin just like you do
MR for the PMB.
 This real world advice
is counter to the
current DCMA
guidance.

83 Putting This New Knowledge To Work

Managing margin is what Risk+ is all
about
84

CP Total Float
Float Erosion: Critical Path Time Usage Acceptable Rate of Float Erosion
Linear (CP Total Float )
100

80 Time Now
October 31, 2005
Critical Path - Time Reserve

60

40 Spacecraft
Contract Delivery
December 10, 2007

20

0

-20

-40

How much margin do we need?
85

The Missing Link: Schedule Margin Management, Rick Price, PS–10, PMI–CPM EVM World 2008

Deterministic versus Probabilistic
86

Baseline
Plan
Sep 2011

Oct 2011
Current Plan
with risks is the
Ready deterministic schedule
Nov 2001
Early Plan
Margin
Dec 2011 Launch
20% Risk
Period Margin
Jan 2012
Mean Current Plan
with risks is the
Feb 2012 stochastic schedule
The probability
distribution can
80%
Mar 2012 vary as a Missed
function of time
Launch
ATLO

Period
CDR
PDR

FRR
SRR

Apr 2012

References
89

 “Protecting Earned Value with Schedule Margin,”
http://www.prochain.com/pm/articles/ProtectingEVSchedules.pdf
 Depicting Schedule Margin in the Integrated Master Schedule,
http://www.ndia.org/Divisions/Divisions/Procurement/Documents/PMSCommittee/C
ommitteeDocuments/WhitePapers/NDIAScheduleMarginWhitePaperFinal-
2010(2).pdf
 Effective Risk Management: Some Keys to Success, Second Edition, Edmund Conrow,
AIAA Press.
 How to Lie with Statistics, Darrell Huff, Norton, 1954 (Available in paper back at
any good book store)
 DID DI–MGMT–81650 “A management method for accommodating schedule
contingencies. It is a designated buffer and shall be identified separately and
considered part of the baseline.

References
90

 Interfacing Risk and Earned Value Management, Association for Project Management,
150 West Wycombe Road, High Wycombe, Buckinghamshire, HP12 3AE, United
Kingdom.
 Practice Standard for Earned Value Management, Second Edition, Project
Management Institute, 2011.
 Effective Opportunity Management for Projects, David Hillson, Taylor and Francis,
2004.
 Measuring Time: Improving Project Performance Using Earned Value, Mario
Vanhoucke, Springer, 2009.
 Performance Based Earned Value, Paul Solomon and Ralph Young, Wiley, 2007.
 Effective Risk Management: Some Keys to Success, Edmund Conrow, AIAA Press,
2003.

Niwot Ridge LLC
(: 303.241.9633
4347 Pebble Beach Drive
-: glen.alleman@niwotridge.com
Niwot, Colorado 80503

91

Risk management using risk+ (v5)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Risk management using risk+ (v5)

Similar to Risk management using risk+ (v5) (20)

More from Glen Alleman

More from Glen Alleman (20)

Recently uploaded

Recently uploaded (20)

Risk management using risk+ (v5)