A retrospective looking at the implementation of Cost of Delay as a means to improve decision-making, improve the sense of urgency and encourage the breaking down of work as well as scheduling or prioritisation using CD3.

1. Cost of Delay: A retrospective
Implementing an economic decision framework
Joshua Arnold
LESS2012 – Tallinn, Estonia

2. Safety First!
These are the views of Joshua Arnold
• My experience, viewpoint, thoughts
• Trust you to respect the client: don‟t share specifics
@joshuajames
joshua@ecolojic.com
uk.linkedin.com/in/joshuaa

3. Together > Σ(parts)

4. “The Don”

5. Context & Scope
Company, Culture, Technology, Process

6. A Container Logistics
Company
• >600 vessels
• 3,800,000 TEU
• 35,000 port calls
• $26B Revenue

7. People
• Logical Reasoning
• High sense of urgency
• Open, but structured
• Willing to learn
• Generalists

8. The IT Department
• ~$500m Budget
• ~$100m Development
• Thin outsourcing
• Legacy & SOA
• Standard process?
Demand > Supply

9. Scope: Lightbulb to live
Prioritise new Reduce batching Break down Limit the Work
ideas quickly to smooth flow the work in Progress
Triage Dynamic Refine Realise Release
Priority
<1 week List
Prioritise by Manage Get the point Increase
knowing the capacity with of writing quality with
Cost of Delay a pull system code quickly fast feedback
The end-to-end innovation process

10. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

11. Oct 2010
600 Long value pipeline
Cycle-time from Lightbulb to Live
# Requirements
400
200
0
Days

12. Oct 2010
Work-in-progress Analysis
State # RQ
Idea! 729
New 897
Being Drafted 416
Ready for Review 422
Ready for Guesstimation 181
Ready for Prioritization Analysis 2980 335
Ready for Estimation 68
Ready for Authorization 219
Authorized Estimation 502 215
Development Initiated 242
Development Complete Development 326 84
Testing (total of all states) Testing 395 395
On Hold Waiting 471 471
* Requirements currently work-in-progress

13. Oct 2010
Demand is increasing
IT Development Budget
(forecast)
Scaling up might help but
won‟t be enough to deal
with the forecast increase
in demand
2010 2011

16. SLT: Prune the Tree: Selected Lean/Agile Practices

17. Business Case Details
What impact is this problem having on our customers/the business/the employees? What are the key
deliverables to be expected? What other indirect benefits may arise from this work?
> $9m by delivering functionality earlier due to delivering in smaller more frequent increments.
Development expenditure in 2010 is $62m.
MLSP Benefits Tracking team found that the hard benefit to cost ratio for IT investment was 1.8.
Therefore the current hard benefits (over three years) for expenditure of $62m is approximately 1.8 X $62m = $111.6m
Therefore the benefits per year are approximately $111.6 ÷ 3 = 37.2
By delivering half of the functionality in half of the time, we release value earlier and gain benefits earlier. The formula for
this is: Additional benefits = ½(1-Cn/Co) x Original Benefits – where Cn=New Cycle Time, and Co= Old Cycle Time.
When Cn/Co = ½ then the additional benefits are ¼ of the original benefits = ¼ x $37.2m = $9.3m
> $4m by improving prioritization of work
Key assumptions:
• Development budget continues at 2010 levels ($62m)
• Project have same hard benefits:cost ratio as the MLSP Benefits Tracking team measured in 2009 (1.8)
• Benefits derived from an increase in value without any significant change in the cost of IT development.
• Improvement in prioritisation will have a similar effect (+3.4%) as that seen on X-leap project.
Soft benefits:
> Flow improvements – a measurable improvement in cycle time, enabling MLIT to be more efficient – delivering an increase in
development throughput with the same resource.
> Quality improvements – a measurable improvement in VoC due to increased feedback cycles ensuring alignment of solution
to expectations
Nov 2010
All rights reserved © 2008

18. Planning next steps
• Objective: begin to apply lean product development techniques to a
specific portfolio:
• Probably focus on AMC as pathfinder portfolio because:
– Contains System A
• steady flow of development,
• “notoriously” slow cycle time with significant waiting waste
• will be a visible win to the whole organization if we improve this
– Contains other smaller applications too
– Close to coaching team
– Enthusiastic Portfolio and Delivery Manager
Dec 2010

19. “I thought
you were
nuts trying it
on [Sys A]”
Board Member

20. Dec 2010
Seeking Business Partner support…
1. Participation
• Steering Group
• Kaizen Group
2. Help us to reduce work package size
• Express benefits for each Feature (not just Release level)
• Break large Requirements down into smaller ones
• Help adjust the approval process & funding model so that we approve/fund
individual requirements/features.
3. Test the prioritisation approach
• Help in putting together a backlog
• Use Cost of Delay / Duration as an initial priority order

21. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

22. Jan 2011 Improve Prioritisation

23. Jan 2011
Dynamic Priority List
1. Safe waiting place
2. Pull system
• Flexible
• Visible
• Value driven
To improve E2E
speed, needed Faster
“Triage”

24. Jan 2011
A framework for estimating benefits
Increase Revenue associated with either
improving our profit margin or
Revenue increasing our market share
Protect Sustaining current market share or
Revenue revenue figures
Reduce Costs that we are currently
Costs incurring, that can be reduced
Avoid Costs we are not currently incurring
Costs but may do without action

25. Jan 2011
Identify benefit types
If we can‟t work out the value, is it worthless?
- Need some idea of value in order to compare
WHY? WHY?
Increase Protect Reduce Avoid
Revenue Revenue Costs Costs

26. Jan 2011
Getting to $ figures
A couple of tactics…
1. Make the value equal to cost of alternatives
Example: Automating a process
The value of automating the XYZ process is at least equal
to the current cost of doing it manually (plus the value of
doing it faster and without human error)

27. Jan 2011
Getting to $ figures
A couple of tactics…
2. Estimate the value of the effects of the change
Example: Improvements to invoicing accuracy
We want to improve invoice clarity and accuracy, in order to:
• Make it easier for customers to pay the correct amount
(protect revenue)
• Without delays (increase revenue) and
• Use less employee time processing customer inquiries and
complaints (reduce costs).

28. Feb 2011
Jan/Feb 2011 – Key Learnings
• Adding work-in-progress limits effectively reveals problems but can choke
flow if set too agressively
• Need to actively ensure all stakeholders feel this is managed appropriately
• Need a firm hand
• A new prioritisation approach creates winners & losers
• Losers need active management (KPIs can make this worse)
• Unclear role definitions/expectations hampers the taking of ownership by
the permanent team
• Too much of the change is still being driven by the Lean-product-development coaches
• Need to repeat it many times for it to stick
• Lean Product Development concepts are deceptively simple but applying them consistently take
time for people to understand

29. Feb 2011
What are we trying to accomplish?
development pipeline
^
^
ideas

30. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

31. Mar 2011
Cost of Delay
Putting a price-tag on time
How that value
$ Business decays over time Information
Value of the discovery value
feature
Cost of
Delay
While you may ignore economics, it won‟t ignore you

32. Mar 2011
Cost of Delay
Putting a price-tag on time
$ Benefits Realised
Time
For ideas with a very long-life, with peak unaffected by delay

33. Mar 2011
Cost of Delay
Putting a price-tag on time
$ Benefits Realised
Late Entry Time
For ideas with a very long-life, with peak unaffected by delay

34. Mar 2011
Cost of Delay
Putting a price-tag on time
$ Benefits Realised
Cost of
Delay
Late Entry Time
For ideas with a very long-life, with peak unaffected by delay

35. Mar 2011
Cost of Delay
Putting a price-tag on time
$ Benefits Realised
Reduced Peak
Cost of
Delay
Late Entry Time
Short benefits horizon, and reduced peak due to late delivery

36. Mar 2011
Cost of Delay
Putting a price-tag on time
Reduced
$ Benefits Realised
Peak
Cost of
Delay
Late Entry Time
For ideas with a very long-life, with reduced peak due to later delivery

37. Mar 2011
Cost of Delay
Putting a price-tag on time
$ Benefits Realised
Cost of
Delay
Late Entry Time
For ideas with a very long-life, with peak unaffected by delay

38. Mar 2011
Cost of Delay – Example 1
RQ-9076
Improve invoice accuracy leading to:
• Reduction in number of customers paying late, worth an
additional $4,000,000 per annum
• Reduction in number of calls currently costing 5 FTEs at
$20k per FTE
Increase Revenue: $4,000,000 p.a.
Reduce Cost: $100,000 p.a.
Delaying this requirement by 1 week is worth $4.1m/52
weeks
Cost of Delay = $78,846 per week

39. Mar 2011
Cost of Delay – Example 2
RQ-9077
Automating a process to satisfy new regulation that will be
effective from 1st Sept 2012, in order to:
• Avoid the additional manual processing resource which is
estimated to cost about 20 FTEs at $20k per FTE
Avoid Cost: $400,000 p.a.
It‟s going to take about 13 weeks to automate, so delaying
the start by 1 week beyond the last responsible moment
of 1st June 2012 is worth $0.4m/52 weeks
Cost of Delay = $7,692 per week

40. Mar 2011
Cost of Delay – Example 3
RQ-5942
Change required to satisfy a new customs regulation
effective immediately, in order to avoid:
• Fine estimated to be about $XXX,XXX per vessel per week
x XX vessels that are calling at certain ports.
• Estimated probability of being fined ~20%. The total
benefits have been calculated as $0.Xm x XX x 20% =
$Xm per week
Avoid Cost: $X,XXX,XXX per week
Cost of Delay = $Xm per week

41. Mar 2011
Trade off decisions
CoD as an aid for decision making
Cost of Delay Alternative solution:
= $10k/week
Cost Increase:
$5,000
Delivered:
2 weeks earlier
Should we do this?

42. Mar 2011
Trade off decisions
CoD as an aid for decision making
Cost of Delay Tier 2 Infrastructure:
= $10k/week
Cost Increase:
$20,000
Delivered:
4 weeks earlier
Should we do this?

43. Mar 2011
Trade off decisions
CoD as an aid for decision making
Cost of Delay Vendor B:
= $10k/week
Cost Reduced:
$50,000
Delivered:
10 weeks later
Should we do this?

44. Mar 2011
Scheduling decisions
If these take the same amount of time to deliver:
Requirement Cost of Delay
RQ-9076 $78,846/week
RQ-9077 $7,692/week
RQ-5942 $X,000,000/week
When the time required to deliver varies, then use:
Cost of Delay Divided by Duration (CD3)

45. Mar 2011
CD3: Cost of Delay Divided by Duration
How value decays
Business value over time Information
of the feature discovery value
Cost of Delay
CD3
Score Duration

46. Mar 2011
FIFO Queuing method
First In First Out
Project Duration Cost of Delay CD3
A 5 $1 0.2
B 1 $4 4
A
C 2 $5 2.5
Cost of
B
Delay
$50
C
Time

47. Mar 2011
CD3 Queuing method
Cost of Delay Divided by Duration
Project Duration Cost of Delay CD3
A 5 $1 0.2
B 1 $4 4
C 2 $5 2.5
B
Cost of
Delay This example shows
27% reduction in Delay cost
C compared to using CoD only
84% reduction in Delay cost
$8 A compared to FiFO
Time

48. Mar 2011
Cost of Delay Divided By Duration (CD3)
The mathematics…
Two possible features, A and B that have:
• Cost of delay of CODa and and CODb ($ per week)
• Each blocks the pipeline for Ta and Tb (weeks)
Scenario 1 (B then A) Scenario 2 (A then B)
B A
Tb A Ta B
Opportunity Cost: CODa x Tb Opportunity Cost: CODb x Ta
If Scenario 2 is better than Scenario 1, then the opportunity cost incurred by
choosing Scenario 1 will be bigger than the first:
• CODa x Tb > CODb x Ta, rearranging this (divide everything by Ta x Tb)...
• CODa/Ta > CODb/Tb

49. Mar 2011
Why Duration, not Cost?
• Dividing by cost assumes that the limiting factor is
available funding
• In most product development settings, it is time and
the capacity of the development pipeline that is the
limiting factor - time is an irreplaceable resource
• Knowing how long the machine will be busy,
blocked/unavailable is more valuable information

50. Mar 2011
Splitting benefits
Break large business cases into smaller pieces of value:
• 70% of functionality delivered is never or rarely used
• Large batches tend to snowball: include everything
imaginable
• Discovery: getting the requirements right up front is
impossible
Breaking things down into smaller parts has other benefits…

51. Mar 2011
Prioritisation in the real world
• Sometimes the data doesn‟t tell the full picture
• Business & IT Strategy can be used as additional
levers
In the real world:
• Start with CD3 as a means of initial triage
• Make a manual adjustment to Cost of Delay to reflect
• State reasons for adjusting

52. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

53. Jul 2011
End of “pilot mode”
Process Measures
Enable smaller batches
 From 12wks between releases to 7wks by Sep
Establish dynamic prioritisation
 Process established by April
Improve prioritisation Process established by April

Actively manage WIP
 Process established by April
Reduce size of requirements
 No max, to max 5 guesstimate points by April
Get to prioritisation faster
 From 100 days to 7 days by April Currently 14 days
Results Measures
Cycle time from Pull Baseline: 210 days, Goal 90 days. 102 days by July
Requires 12 months to gain data but should get to 140 days
 by July by summing process segments.
Voice of Customer Requires 12 months to gain data. Quick survey to CUS BPO
 staff will give indication (Apr)
Improved ROI Requires 12 months to gain data Measurement has begun

54. Nov-
LPD Roadmap Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug
Jan
Mobilise & Analyse M+A
Teams
Pilot (CSBPO & AMC) Red: Core team
Yellow: Team 2
Pilot A (System A) Envision + Realise (A) Green: Team 3
Pilot B (System C) B
Pilot New Proj. (XX) New Project
Wider IT Roll out
Sys D Assessment A
Engage OPS + FIN M
System B 1
OPS Applications 2
D2I & FBR 3
DCF 4
N&P Applications Key Assumptions: 5
• 8 Groups of Applications (+Pilot) 6
SAL Applications
• Most effective to order by BPO
Large Project X • 3 support teams required 7
• A team can start supporting a new
Other smaller Apps Business Area every 3 months 8
Stabilise & Kaizen
VFQ Education Value, Flow, Quality Education
Kaizen Coaching Follow-up Coaching and Ad-hoc advice

55. May 2011
Jorge & Matt working
with the Sys C Team

56. September 2011.
Kicking off the Sys B Team

57. System B LPD Rollout
Improve
Prioritisation
FSCM Rollout to one area
Triage Refine Realise Release Live

58. November 3 FINBPO Prioritisation Session
I.

59. Vendor team in Kolkata

60. System B – (SAP Finance)
125+
RQs
requirements evaluated using Cost-of-Delay
25
Released
with CoD
120+ employees coached in process and practises
60+ attending daily stand-ups
Releases
4
6 Kanban boards 3
Kaizen groups
in operation

62. SOA Project Team: Problem
statements for validation, valuation
and prioritisation using CoD & CD3

65. London teams
From: Özlem Yüce
Subject: Update from the London team
Date: 1 February 2012
Hi all,
We had the kick off session this
afternoon; BPOs, DM team and the
vendor participated. The session was
interactive and the team is very engaged.
Especially the BPOs are impatient about
applying COD which is very good.
Tomorrow we plan to have the COD
presentation. We will also have the first
prioritisation session (relative) and start
identifying the data to be gathered for
COD calculations before the end of this
week.
Ozzie & Jorgie

66. Ops Applications

67. Enterprise Architecture
Head of Enterprise Architecture

68. Senior Management
Sent: Monday, May 07, 2012 1:52 PM
We are eagerly awaiting a change to [System A] and I cannot
understand why it takes this amount of time to go through the review
process!?
The cost of delay for this change is USD XM per week
With kind regards
Head of Operations BPO

69. Author: Joshua Arnold

70. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

71. Benefits for I.T.
Headache: Controlling demand for scarce resources
• Create clear focus on priorities
• „less yelling and screaming‟ data-driven, more visible
• Manage dependencies between teams
• Change the conversation…
Delivering
“on time”
Delivering
value quickly
Cutting
I.T. costs

72. Benefits for the wider organisation
Headache: Getting changes fast enough
• Create focus across the
business on the most
profitable ideas or
problems
• Recognise urgency in
prioritisation
• Uses a language that
everyone can understand
• Encourages focus on
reducing time to market

73. Improved visibility of benefits
$44.80
6x
Average GCSS FACT
(Before) (After)
Benefits per dollar invested

74. System A: Value distribution
(truncated)
$2,800,000
$2,600,000
A small number of
$2,400,000 features have a very
$2,200,000 high Cost of Delay
$2,000,000 Previous
$1,800,000 Average
$1,600,000
$1,400,000
Cost of Delay / week
$1,200,000
$1,000,000
$800,000
$600,000
$400,000
$200,000
$0
0 10 20 30 40 50 60 70 80
Requirements sorted by Cost of Delay

75. System A: Value by quartile
Average $ Benefits
Per Requirement
Next 25%
Next 25% Bottom
Top 25% of RQs 25%
$220/wk
$18,600/wk $5,200/wk
$230,000/wk

76. System B: Value distribution
100000
90000
80:20
80000 Pareto Principle
“the vital few”
70000
60000
Cost of Delay
50000
(US$ per week)
40000
30000
20000
10000
0
0 10 20 30 40 50 60
Requirements

77. Improved visibility of benefits
How do we know the changes are an improvement?
10x
6x
Average Sys A Sys B
(Before) (After)
Benefits per dollar invested

78. Making the case
Pilot: getting to $ benefits
Pilot: Cost of Delay and CD3
Roll out to other teams
Results to date
What went well?
What should we do differently?
What did we learn?
What still puzzles us?

79. What went well?
1. Piloting with one application/business area
2. Quid pro quo - quality and speed
3. Flexibility via Dynamic PL
4. Softly, softly, catchy, monkey approach
5. Relative first; using economics to inform
6. Four benefit types help guide thinking about value
7. Dial up the use of numbers
8. Drive to dollars (what data is needed?)
9. Manual to Automated
10.CoD at Feature Level

80. What should we do differently?
1. Apply CoD/CD3 to support and maintenance work first?
2. In parallel?
3. Get EA using it earlier?
4. Make the CoD for each team/project more visible?
5. Get help to work out the Ave. Lifetime Value of a customer?
6. Leverage segmentation data, geographical customer value?
7. Teach more about benefits that are probabilistic in nature?
8. Coach senior stakeholders – train them to ask the Q?
9. Visualise the value/ROI for each delivery streams earlier?
10. Develop guidance on refining ideas, value mining?

81. What did we learn?
1. It works! Synergy with other Lean-Agile practices
2. Getting to numbers and $ isn’t as hard as you think
3. The four buckets cover well the type of benefits IT delivers
4. It works for Support and Maintenance, upgrades etc.
5. Assumptions can be standardised more than you think
6. CoD for dependencies between features and teams works
7. Some people really want to standardise duration units
8. It changes the focus and conversation between people
9. Senior stakeholders get it (and use it to escalate)!
10.CD3 encourages breaking down of requirements

82. What still puzzles us?
1. Will it survive without on-going support. How?
2. How to easily support other benefit profiles?
3. Dependencies between delivery streams which Duration?
4. Do items on the DPL need to have integrity?
Duplicates? Real Options? Linking and updating of
dupes?
5. Does Portfolio level CD3 need common units for duration?
6. CD3 for the Refine/breaking down stage?
7. How to handle Information Value in a consistent way
8. How to consistently represent Technical Debt using CoD
9. Do we need to do CoD at project level – scope?
10. How to speed up the feedback loop to know it‟s working?

83. Final thought
• Typical situation in product development
• Go hunting: try looking at the „Fuzzy Front End‟
PoC Dev & Test
Captured Go Live!
(24 hrs) (82 hrs)
18 weeks waiting 11 weeks waiting 9 weeks waiting
Cost of Delay = $XXX,XXX per week

84. Questions? Feedback?
@joshuajames
joshua@ecolojic.com
uk.linkedin.com/in/joshuaa