How #NoEstimates ideas can be applied and help direct the work of testers in software projects

1. #NoEstimates
10 New Principles for
Software Development
(and testing!)
Vasco Duarte
@duarte_vasco
Tweet at:
#TAHelsinki

2. Vasco Duarte
@duarte_vasco
http://SoftwareDevelopmentToday.com
http://bit.ly/vasco_slideshare
Vasco.Duarte@oikosofy.com
http://NoEstimatesBook.com

3. #NoEstimates is for you if…

8. Principle #1
Trust your process, or change your
process

9. NumberofBugs
Timeline
Bug evolution in a non-agile project
Open
Closed
Submit
Development phase Desperately testing and fixing phase
Waterfall

10. Principle #2
Shorten the feedback cycle

11. Can estimates be accurate?

12. Accidental complication

13. Cost/Duration =
Essential Complication
x
Accidental Complication

15. Yet, some people still argue we
can be good at estimating… Let’s
look at the evidence

16. 80% Late
or Failed
Source: Software Estimation by Steve McConnell
Reality: 80% is late or failed

17. Lets break that down
Chaos Report 1995
For the same combined challenged and impaired projects, over one-third also
experienced time overruns of 200 to 300%. The average overrun is 222% of
the original time estimate. For large companies, the average is 230%; for
medium companies, the average is 202%; and for small companies, the
average is 239%.
Time Overruns % of responses
Under 20% 13.9%
21 – 50% 18.3%
51 – 100% 20.0%
101 – 200% 35.5%
201 – 400% 11.2%
Over 400% 1.1%
~68% of
projects 51%
or more late!

18. Let’s continue to break that down
Chaos Report 2009
Average cost overrun: 45%
Average time overrun: 63%
Chaos Report 2011
Average time overrun: 63%

19. Just in case you don’t like the
CHAOS report

20. Source Gartner survey of project failure, 2012
Failure, means total
failure, not just late

21. Of the large systems that are comple
ted, about 66% experience schedule
delays & cost overrun
Source: “Project Management Tools and Software Failures and Successes” by Capers Jones
Crosstalk, the Journal of Defense Software Engineering

22. Traditional projects: 53% failed
or challenged
Source: Project success survey by Scott Ambler

23. Agile project: 40% failed of
challenged
Source: Project success survey by Scott Ambler

24. 17 percent of large IT projects go so
badly that they can threaten the
very existence of the company
Source : McKinsey & Company in conjunction with the University of Oxford
Type of survey : Study on large scale IT Projects
Date : 2012

25. More data coming, sign-up at
NoEstimatesBook.com to receive
this report when available

26. Principle #3
Believe the data, not the estimates

27. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art

28. Principle #4
Use alternatives to Estimate-driven
decision making

29. Comparison of 17 projects ending between 2001
and 2003. (Average: 62%)

30. Testing for value, a story…

31. Principle #5
Test for value first, then test for
functionality

32. Can #NoEstimates work in Real
Companies?

33. http://j.mp/NoEstimates-Book

34. Principle #6
Estimation is waste, reduce it’s
impact on your business

35. Principle #7
Measure progress only with
validated, running software

36. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art

37. Source: Software Estimation by Steve McConnell
“Good”
estimates:
25% of
estimated

38. I AM GOING TO
GO AHEAD AND
ASK YOU TO
DELIVER 10
STORIES NEXT
SPRINT...

39. WTF!!!!!
!#%&!

40. 0
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
velocity
Average=
LCL
UCL
Target
Actual, measured
throughput over 21
sprints

41. The 4 Estimates Dysfunctions
1. Estimate Bargaining
2. Internal Politics
3. Blame Shifting
4. Late Changes
5. Sunk Cost fallacy

42. Principle #8
The system where you work has
predictable outputs, learn to
understand the system

43. Wow! But I have a business to
run!
Is there a way do better than
that?

44. #NoEstimates delivers!

45. Counting Stories vs. Estimated
Story Points
Q: Which ”metric” is more
accurate when compared to
what actually happened in the
project?

46. A long project
24Sprints

47. Which metric predicted most
accurately the output of the
whole project?
a) After only the first 3 Sprints
b) After only the first 5 Sprints

48. Disclaimer...
This is only one project!
Find 21 more at:
http://bit.ly/NoEstimatesProjectsDB

49. After just 3 sprints
# of Stories predictive powerStory Points predictive power
The true output:
349,5 SPs
completed
The predicted
output: 418 SPs
completed
+20%
The true output:
228 Stories
The predicted
output: 220
Stories
-4%!

50. After just 5 sprints
# of Stories predictive powerStory Points predictive power
The true output:
349,5 SPs
completed
The predicted
output: 396 SPs
completed
+13%
The true output:
228 Stories
The predicted
output: 220
Stories
-4%!

51. Q: Which ”metric” is more
accurate when compared to
what actually happened in the
project?

52. But there is more...

53. What difference does a Story
Point make in a project that
used both Story Points and
#NoEstimates?

54. Next you will see the
forecasted release date when
using Story Points (values
1:3:5)

55. 68
71 71 71 71 71 71 72 72 72 73 73
0 3 7 7 9 11 12 13
20 20 22 23
0
10
20
30
40
50
60
70
80
90
100
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on
20th October
2014

56. Next you will see the
forecasted release date when
using Story Points (values
1:2:3)

57. 48
51 51 51 51 51 51 52 52 52 53 53
0 2 5 5 7 8 9 10
15 15 17 18
0
10
20
30
40
50
60
70
80
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on
14th October
2014

58. Next you will see the
forecasted release date when
#NoEstimates
(or, all stories = 1 story point)

59. 28
31 31 31 31 31 31
32 32 32
33 33
0 1 3 3
5 5 6 7
10 10
12 13
0
10
20
30
40
50
60
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on 29th
September 2014

60. Conclusion...

61. All dates within 3 weeks of
each other in a 38 to 42 week
project (a margin of ~8%)

62. Data used with permission
from Bill Hanlon at Microsoft
”At that point, I stopped
thinking that estimating
was important.”
Bill Hanlon:
http://bit.ly/BHanlon

63. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
In this presentation you have seen examples that far outperform what
estimation specialists consider a ”good estimation”. In common they have
one way to look at work: #NoEstimates

64. #NoEstimates testimonial
The deviation between estimated and
actual velocity would have been
approximately 15% lower if we would have
used #NoEstimates.
We have analyzed data from 50 Sprints…
…at no time the story point based
estimation was better than #NoEstimates.

65. Principle #9
Don’t bet your company on poor
track record methods, use methods
with a proven track record
aka: Hope is a bad management
strategy

66. Carmen faces a very difficult situation…

67. The 7 Step Journey To NoEstimates
1. Start using Story Points
2. Stop estimating tasks
3. Limit the calendar duration for
Stories and Features
4. Reduce the number of
allowed estimates (say, 1,2,3
and 5 only)
5. Track your data
6. Use your data
7. Simply count the number of
stories

68. http://j.mp/NoEstimates-Book

69. http://j.mp/NoEstimates-Book

70. http://j.mp/NoEstimates-Book

71. Principle #10
The transformation starts with you…
Vasco Duarte
Vasco.Duarte@oikosofy.com