A Keynote I have delivered to EuroSTAR 2014 and Test+Verification Conference. How did we move from yearly releases to weekly releases and how did our testing change.

1. Continuous Delivery and DevOps
Moving from Staged to Pervasive Testing
Intelligent Testing Conference 2015
Rob Lambert
@rob_lambert
www.thesocialtester.co.uk

2. Slow
Boring
Ineffective
testing
I’ve spent a lot of time doing

3. It’s no surprise then that
we didn’t ship very often

4. Test case
management
system of
DOOM

5. Mind the gap

6. A simple calculation

8. Anyone who can
click a mouse
became a
tester….doesn’t that
say something
about the testing?

9. I can’t believe it took so long to test
Slow
Boring
Ineffective
I don’t mind helping out but my word, testing is so boring
We ran 10 million tests and we still had issues!

10. I joined NewVoiceMedia
just before the last giant
release.
We vowed never to do a
year long release again.
Let’s do something different

11. Why stop doing big releases?

12. This is
what
we
shipped
(One
tower)
This is what
our customers
now needed (a
whole bridge)
Photo by Vaidotas Mišeikis - Clifton Suspension Bridge at Dusk - February 19, 2011

13. When your customer’s needs move quickly,
so too must your release process.

15. The ultimate feedback loop

16. How did we make the change?
We created a vision

17. 1. Adopt Agile
2. Prioritise work
3. Bring Dev and Ops together - DevOps
4. Everyone tests. Test all the time.
5. Become “one” with data
The Vision
Weekly Releases

18. 1. Removed testers from the
centre of the universe
Testers are important, but no more than any other member
of the team
Instead we put “Testing” at the centre of our universe

19. 2. Focus on automating checks
BDD
TDD
CI
UI Testing

22. 3. Ate our own product
Dog-fooding
(Pre-
Production)

23. Event logs,
internal
customer
feedback and
regression
tests

24. 4. Tested and monitored after
release
TRUST
Own usage
Monitoring

25. Every team supports
their work in to
production

26. Data from production tells us
how our product is behaving
This is immensely powerful in
guiding your testing

27. 5. Emphasised discovery over
assumptions
Exploratory Testing
What is the system
telling us?

28. Testing that is data informed

29. 6. Learned.
• 5 Whys on everything
• Continuous Improvement
• Training and personal development
“It’s like studying for a degree but
getting paid for it”

30. 7. Dealt with fires
• “If it hurts. Keep
doing it”
• Political
• Squash “Naysaying”
and “Silent
Plotting”

31. Everyone knows how to test.
But it’s not that sort of testing
you need.
You need testing that is relevant.

32. • At least one release each week,
sometimes more
• We have more to improve though
• Feedback from live is guiding our testing
• Feedback from our testing is immediate
Did it work? And is that it now?

33. Thank you. Any Questions.
NVM
www.newvoicemedia.com
How To Remain Relevant – The Book
www.howtoremainrelevant.com
Cultivated Management Seminar Day
www.cultivatedmanagement.com
Blog
www.thesocialtester.co.uk
Twitter
@rob_lambert