AWS Community Day CPH - Three problems of Terraform
Large scale agile_svante_lidman
1. Driving Large Scale Agile
Transformation
Svante Lidman
svante@ivarjacobson.com I have changed employer since
I made this presentation. You
can now reach me at
svante.lidman@hansoft.se,
see: www.hansoft.se for what
we do.
2. This presentation
• Patterns and anti-patterns for transforming an organization with
huge legacy in product, process, and culture to being lean/agile
• Based on:
– Experience in the large over the last 18 months
– Experience in the not quite so large over the
previous 25 years
– Hearsay, reading
– Thinking, trying to generalize, evolve, reformulate
• It is the story as I see it, others may have other
stories
3. Outline
• Introduction
• What’s the deal with large scale agile?
• Case study
• Conclusions and advice
4. What’s the challenge with large scale agile?
• Given a large organization, potentially distributed,
potentially large legacy in product and culture,
what do we need to focus on?
• Practices?
• Organization?
• Tooling?
Individuals and interactions over processes and tools
5. What’s software development after all?
• Software development
– A cooperative, Finite, Goal-Seeking Group Game (Alistair Cockburn)
– I would add creative and evolving (Svante)
• Software is intellectual property, hence…
– Developing software is an intellectual achievement, hence...
– ... to create an effective SW development organization it is key to:
• Maximimize the extent that everyone’s intellect is engaged and aligned
• Eliminate disturbances that disallows people from being fully effective
intellectually
6. Outline
• Introduction
• What’s the deal with large scale agile?
• Case study
• Conclusions and advice
7. The case in question
• Product development unit in a large multi-national company:
– Thousands of developers
– Single product line
– Large legacy system
– Distributed, component-based organization - waterfall process
– Highly successful commercially and of fundamental importance to the
bottom-line of the company
– Fierce market competition
• If it ain’t broken why fix it?
8. Pain – a prerequisite for change
• Pain
– Increasingly difficult to get even small features
developed with reasonable lead times
– Quality through sweat and tears
– Development capacity not matching the business
opportunities
• We want to do more!
9. Cause of pain
• Functional organization
– Analysis
– Software development
– System testing
– Waterfall process and an organization that defines
itself in those terms
• Handovers and functional/component breakdown
– Few persons understand the whole product
– Few persons understands the whole process
• Focus on resource utilization by upfront planning Fredrick Winslow Taylor
Insight: We can’t patch on the waterfall anymore
and get what we want, we need to do something
different
10. Building the vision for lean / agile
• Stories that people can
relate to
• A simple message
11. The Vision Summarized
• Change to a setup where feature development is
driven in a program decoupled from release projects.
This is called Streamline Development.
• Implement true cross-functional feature teams with
responsibility for a feature from analysis through end
of feature verification
• Enable agility on the team level (Self-organization,
Stand-ups, Iterations, Retrospectives, TDD,
Continuous
Integration and so on)
12. Key Ideas of the vision
• Focus on flow, customer-to-customer
– It is more interesting to reduce lead-time with constant productivity
than to improve productivity with constant lead-time
– It is more interesting to improve quality with constant productivity than
to increase productivity with constant quality
• This will expose inefficiencies and force:
– Reduction of handovers
– Reduction of delays
– Reduction of overly detailed studies
and gold-plated designs
– Eradication of late and non-repeatable testing
as a way to get quality
• The focus on lead-time will act as a forcing function
to address quality and efficiency
13. Objections to the vision
• We have special needs
– It is a very large code base, a single designer cannot learn all
– Some subsystems are complex and requires substantial experience
– Many different technologies, it is too difficult to learn them all
– Some features are very large and span the whole system a single
team cannot handle such a feature
– Weak code ownership will deteriorate the integrity of the code base
– It is a complicated domain, experts need to do analysis and design
• That stuff doesn’t work
– We tried that 7 years ago and it didn’t work
– Organization X tried that 4 years ago and they dropped it
14. Pilot – first strike
• We wanted select a pilot with the following characteristics:
– Right size for a team of 5-7 to complete in 3-4 months
– Relatively isolated not too much dependencies
– It should have business value but not be too important
• And we failed before the start line!
15. Pilot – second strike
• Feature that we already looked at:
– Too large
– Too complicated
– Too important – essential for release
of new high-end product
– Work already ongoing
• Waterfall planning now showed that
this feature would delay the new
product with 2-3 months
• Hastily we formed a cross functional team that:
– Sat together (as far as possible)
– Worked iteratively and light-weight
– Delivered the feature in a way that did not risk the schedule of the
new product (took us 7 months)
– About same cost as projected for traditional way of work
16. This was the right pilot at the right time
• It had very high business impact and
high risk -> Attention!
• At sprint reviews/demos we could review:
– Logic behind our iterative plan
– Challenges / Issues / Impediments
• Low quality on the main/trunk forcing us to
stay on branch using old baselines
• The difficulty in defining good back log items
• Problems with IT infrastructure
• Culture and practices enforcing waterfall behavior
• Challenges in existing within the big waterfall
• Shortcomings in tooling / automation
• Challenges in getting a team assigned, allocated and seated
• Management put a lot of trust in the team!
17. Advice on pilots
• Don’t run pilots to:
– Prove that things “work”
– Benchmark against current ways-of-working
• Do run pilots to:
– Learn
– Find problems that you need to fix
• The ideal pilot is:
– Critical to business
– Delayed
– Quite large
• Organizational impediments will surface and become hot issues
immediately
• Management will care and chances are you will get focus on
solving some real problems
18. Find a role model
• Find an organization quite like your own – we got lucky!
– Based on the same platform, using same tools
– Similarities in culture
– Similar order of magnitude of the code base
– Very similar starting position in terms of pains,
processes, organization and culture
• This organization had made a radical change:
– Streamline on the high level
– Scrum on the team level
– Large scale continuous integration
– Massive cultural change
19. It takes an engine to go the distance
• Someone from top management needs to step up
• Ideal characteristics
– Highly trusted inside and outside
– Eager to learn
– Courageous
– Inclusive
20. Outline
• Introduction
• What’s the deal with large scale agile?
• Our journey so far
• Conclusions and advice
21. Keep your eye on the ball
• Ideas
– Continuos improvement
– Respect for people
• Practices
– Cross functional teams sitting together
– Working software every 2-4 weeks
– Visual management
– Go see
22. Question old truths
• Development should always be done in
projects
• High resource utilization = effectiveness
• You should write the code ”once”
• Quality comes in a sequence of steps –
developers develop the product, testers test it
• You get what you predict so we need detailed
estimates based on detailed specifications
• To be professional you should cleanly separate
the business side from the IT side
23. Beware of...
• Analysis-paralysis
• Tool mongers and method
gnomes
• Leaders getting detached
• Throwing out your gems
• Parrots
• Masquerading
24. Advice for line managers
• Educate yourself
• Manage, lead, or teach?
• You cannot delegate this agile / lean ”business”
• What is it that really matters?
– People
– Values and principles
– Practical help to remove impediments / accidental
complexity
– Way of working
• Essentially, line management is a support function, it is not part of
the product development flow
