To Estimate or Not to Estimate, Is that the Question? (2017 Better Software Conference)

# b e t t e r s o f t w a r e c o n | @ m a t t p h i l i p
TO ESTIMATE OR  
NOT TO ESTIMATE
Is that the Question?

@mattphilip #bettersoftwarecon
HOFSTADTER’S LAW
“It always takes longer than you expect,
even when you take into account
Hofstadter’s Law.”

PARKINSON’S LAW
Work expands so as to ﬁll the time
available for its completion.

“POOR ESTIMATION SKILLS”

THE SPECTRUM OF ESTIMATING
NeverEstimateAnything
AlwaysEstimateEverything
Estimate Default Behavior Don’t estimate
Eﬀort Factors All sources of variation
Intuition Approach Data
Deterministic Forecast Type Probabilistic
Tasks in hours Level of Detail Work items in days
High Eﬀort Spent Low
Estimating better Improvement Aim Reducing variation
Commitment Management Service Expectations

MANIFESTO RELEVANCE

WHAT NOESTIMATES IS NOT SAYING
▫︎ You are evil if you estimate
▫︎ All estimates are totally useless
▫︎ Stop doing your successful estimating practice
▫︎ Stop having the conversations to understand/analyze/
break down work
▫︎ Work items must be the same size
▫︎ You must place your full faith and conﬁdence in Monte
Carlo forecasts

WHAT NOESTIMATES IS SAYING
▫︎ Know why you are estimating
▫︎ Discover for yourself how good you are at estimating
(measure)
▫︎ Keep doing the things that help you understand the work
▫︎ Upfront estimates need to be held loosely
▫︎ If you focus on delivering value quickly, you obviate the
need for Iron Triangle considerations

MY NOESTIMATES MANIFESTO
… We have come to value:
Reducing sources of variation over Improving estimating
Data over Intuition*
Probabilistic over Deterministic
MVP scope over Full scope
That is, while there is value in the items on the right, we value
the items on the left more.
*Neil Killick uses “empiricism over guesswork”

DO YOU ASSUME CORRELATION?
Is the initial sizing a good predictor for when you can get your stuﬀ?
In our case, the surprising truth was ”no.”
— Mattias Skarin, Real-World Kanban

WHAT WE ESTIMATE
Code

WHAT WE ESTIMATE
Analyze+Design+Code+Test+Deploy

WHAT ACTUALLY HAPPENS
Analyze Code Test DeployDesign
Wait for approval
Wait for staﬀ
availability
Wait for  
rework
Wait for API
availability
Wait for  
higher-priority work
to ﬁnish
Wait for approval
Wait for  
blocker  
removal

WHAT’S GOING ON?
Value-added time
_________________________
Total elapsed delivery time
Flow eﬃciency =

WHAT’S GOING ON?
Low process eﬃciency (typically 5-15%
in software delivery) means that even if
we nailed the eﬀort estimates … we
would be accurately predicting 5-15% of
elapsed delivery time! 
— Troy Magennis

SOURCES OF VARIATION
How many can you name?

SOURCES OF VARIATION
Waiting for  
availability
Rework
Team  
dependencies
Stages in
team
development
(Tuckman)
Steps/ 
handoﬀs
Selection  
policy
Essential  
complication
Accidental  
complication
System  
dependencies
Work In  
Progress
Technology/ 
domain/ 
product
Specialization
Team  
composition
Multitasking/ 
context  
switching
Collaboration 
policy
Blockers

WHAT CAN YOU DO ABOUT VARIATION
How many remedies 
can you name?

WHAT YOU CAN DO ABOUT VARIATION
▫︎Lower WIP
▫︎ConWIP/System WIP
▫︎Five Focusing Steps
▫︎Blocker clustering
▫︎Reduce workﬂow stages
▫︎Explicit policies
▫︎Cost of Delay scheduling, sequencing and selection
▫︎Reduce “expedites”
Lean-Kanban

▫︎“Agile 101” (simple, decoupled design; thin vertical
slices; pairing)
▫︎Identify/make visible/measure dependencies
▫︎Collaborate/Share work (Dimitar Bakardzhiev)
▫︎Spike and stabilize (Dan North)
▫︎Reduce accidental complexity (Liz Keogh)
Team
Why?

▫︎Determine what actions would be diﬀerent based on
the estimate
▫︎Customer-based ﬁtness criteria
▫︎Budgeting: Team run rate
▫︎Focus conversation on value, not cost
▫︎MVP and product ownership
▫︎Create probabilistic forecast ASAP (as soon as you have
data) – together!
▫︎Service-Delivery Reviews
▫︎Teams: Keep teams together, dedicated (reduces
context-switching, Tuckman stages)
Business

POLICIES TO REDUCE VARIATION
▫︎We will only start new work at about the same rate that
we finish old work.
▫︎We will make every reasonable effort to finish all work
that is started and minimize wasted effort due to
discarded work items
▫︎If work becomes blocked, we will do everything we can
do unblock that work as expeditiously as possible.
▫︎We will closely monitor our policies around the order in
which we pull items through our system so that some
work items do not sit and age unnecessarily.
— Daniel Vacanti, When Will It Be Done?

HUMAN BIAS
▫︎Recency
▫︎Availability
▫︎Gambler’s conceit
▫︎Anchoring
▫︎Halo eﬀect
▫︎Motivational
▫︎Conﬁrmation

FIRST, KNOW WHERE YOU ARE

KEOGH’S “SCALE OF IGNORANCE”
1. Just about everyone in the world has done this.
2. Lots of people have done this, including someone on
our team.
3. Someone in our company has done this, or we have
access to expertise.
4. Someone in the world did this, but not in our
organization (and probably at a competitor).
5. Nobody in the world has ever done this before.

DATA OVER INTUITION

LET’S PRACTICE!

31
PROBABILISTIC OVER
DETERMINISTIC

PROBABILISTIC OVER DETERMINISTIC
Dec 28
— graphic courtesy of Larry Maccherone, (maccherone.com/lumenize)

VACANTI’S VERITY
When making a forecast (predicting the future),
you have to accept that there is more than one
possible outcome. Therefore… A forecast is a
calculation about the future that includes both
a range and a probability of that range
occurring.

2%
20% 21%
3%
16% 16%
9%
7%
2% 2% 2%
Dec 28 Feb 17Nov 30 Jan 20

PROBABILISTIC FORECASTING

PROBABILISTIC FORECASTING: ONE ITEM

BACK TO YOUR SCATTERPLOT

STRAIGHT-LINE (AVERAGE) FORECASTS
Team A Team B
Work item #1 5 days 3 days
Work item #2 1 day 3 days
Work item #3 3 days 3 days
Average 3 days 3 days
3-day delivery time  
expectancy
66% 100%

EXPECTATIONS, NOT COMMITMENTS

SERVICE-DELIVERY EXPECTATIONS
Expectation Performance
Expedite 90% 3 days or less
Standard Urgency 85% 10 days or less
Fixed-Date 90% On-time

TO ESTIMATE OR NOT TO ESTIMATE?
NeverEstimateAnything
AlwaysEstimateEverything
Estimate Default Behavior Don’t estimate
Eﬀort Factors All sources of variation
Intuition Approach Data
Deterministic Forecast Type Probabilistic
Tasks in hours Level of Detail Work items in days
High Eﬀort Spent Low
Estimating better Improvement Aim Reducing variation
Commitment Management Service Expectations

BETTER QUESTIONS TO ASK
@mattphilip
▫︎In what context would estimates bring value, and what
are we willing to do about it when they don’t? – Woody
Zuill
▫︎How much time do we want to invest in this? – Matt
Wynne
▫︎What can you do to maximize value and reduce risk in
planning and delivery? – Vasco Duarte
▫︎Can we build a minimal set of functionality and then
learn what else we must build?
▫︎Would we/you not invest in this work? If not, at what
order-of-magnitude estimate would we/you take that
action?
▫︎What actions would be diﬀerent based on an estimate?

HOW TO GET STARTED (PROJECT IN PROGRESS)
1. Continue to discuss the work to analyze it and break it into thin
vertical slices.
2. Start tracking two pieces of data for each work item: commit
date and delivery date.
3. After accumulating 10 data points, run a Monte Carlo
simulation to provide a probabilistic forecast.
4. Begin answering the “When” question using data, either at the
project level or individual work-item level.
5. Validate whether the probabilistic forecasts match what actually
occurs.
6. If the forecasts are helpful, wean your team oﬀ estimating.
7. Focus improvement eﬀorts on reducing the sources of variation.

HOW TO GET STARTED (NEW PROJECT)
1. Ask the “better questions” above about the use and
importance of estimates.
2. Use reference-class data (similar delivery times from other
projects in similar tech stacks with similar teams) to provide an
initial probabilistic forecast
3. As soon as you accumulate 10 data points, run a Monte Carlo
simulation to provide an updated probabilistic forecast.
4. Continually reforecast when you get more information.

REFERENCES AND FURTHER EXPLORATION
▫︎ noestimatesbook.com (Vasco Duarte)
▫︎ infoq.com/articles/noestimates-monte-carlo (Dimitar Bakardzhiev)
▫︎ priceonomics.com/why-are-projects-always-behind-schedule
▫︎ http://scrumandkanban.co.uk/estimation-meets-cyneﬁn/
▫︎ ronjeﬀries.com
▫︎ Lizkeogh.com
▫︎ https://neilkillick.wordpress.com/
▫︎ http://zuill.us/WoodyZuill/
▫︎ mattphilip.wordpress.com/noestimates-game
▫︎ When Will It Be Done? (Dan Vacanti)
▫︎ focusedobjective.com (Troy Magennis)
▫︎ actionableagile.com (Dan Vacanti)
▫︎ kanbanize.com
▫︎ scrum.org

Questions?
THANK YOU

To Estimate or Not to Estimate, Is that the Question? (2017 Better Software Conference)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to To Estimate or Not to Estimate, Is that the Question? (2017 Better Software Conference)

Similar to To Estimate or Not to Estimate, Is that the Question? (2017 Better Software Conference) (20)

More from Matthew Philip

More from Matthew Philip (17)

Recently uploaded

Recently uploaded (20)

To Estimate or Not to Estimate, Is that the Question? (2017 Better Software Conference)