2. Richard Cheng
◊ Agile trainer & coach
◊ Excella Agile Center of Excellence Lead
◊ Member of PMI, Scrum Alliance, Agile
Alliance, Agile Leadership Network
◊ CST, PMP, CSM, CSPO, CSP, PMI-ACP
◊ Founder & executive committee
member of Agile Defense Adoption
Proponents Team (ADAPT), the Agile
DoD Task Force
◊ Deep expertise in Federal and
commercial Agile transformations
3. ◊ Building Quality into the process
– Real time identification of quality issues
• Automation
– Using the right tools and techniques
21st Century Engineering Practices
4. Benefits*
Assuming 100 defects in
10,000 lines of code
1. Traditional testing finds
a defect in about 10
hours
2. Manual code inspections
find a defect in 1 hour
3. Automated testing finds
a defect every 6 minutes
þ 36% reduction in defect rate
when integration/regression testing at each code check-in
þ 90% reduction in bugs reaching QA
Major municipal gas utility
þ
þ
þ
þ
þ
95% cut in cost of bugs
Large retail web site
90% cut in defect remediation cost
Global supplier of healthcare equipment
Faster time-to-market
More features and higher quality
Agility in the marketplace
Added new functionality 2 weeks before ship
Confidence in the process
“Oozing Confidence”
*Sources:
• Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.
• Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Japanese Symposium on Software Testing, Tokyo,
Japan.
• Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.
6. Don’t Do It All At Once
Do This First
1. Version
Control
2. Build
Automation
3. Automated
Unit Testing
4. Continuous
Integration
Next Do This
5. Static Code
Analysis
6. Dependency
Management
Then Do This
7. Automated
Integration
Testing
8. Automated
Acceptance
Testing
9. Deployment
Automation
7. First Do This
1. Version Control
– What: Repository to persist and track version of code and
artifacts
– Effort: Low
– Sample Tools:
• Traditional – TFS (MS), Subversion
• Distributed Version Control (branching and merging) - Git,
Mercurial
8. First Do This
2. Build Automation
– What: One click or one command launch of build process
– Effort: Low
– Sample Tools:
• Microsoft – MSBuild, Powershell
• Java/Others – Ant, Gradle, Maven
9. First Do This
3. Automated Unit Testing
– What: An automated stand alone test that test a single
unit of the code.
– Effort: Medium
– Sample Tools:
• Unit test tools: JUnit (Java), NUnit (MS), MS Test (MS)
• Advanced tools: Moq, Fluent Assertions
10. First Do This
4. Continuous Integration
– What: Provides frequent verification and notification of
changes to the code and application
– Effort: Medium
– Sample Tools: Jenkins, Hudson, TFS (MS), TeamCity
11. Continuous Integration
Build #1
compile
unit test
integration test
package
deploy/run
acceptance test
analyze code
Build
Report
Version
Control
change
#1
change
#2
Build
Server
Build #2
compile
unit test
integration test
package
deploy/run
acceptance test
analyze code
Build
ReportEmail
Failed Build
12. Continuous Integration
◊ CI involves:
– Frequent code check-ins
– Regularly scheduled, automated builds
– Automated tests
– Immediate feedback to developers detailing any build
errors or failed tests
◊ Benefit:
– Developers know immediately upon check-in if their code
works and if any other application features were broken as
a result
13. CI Effect on Schedule
Code Phase QA Phase Code Phase QA Phase
Code Phase Code Phase
Code + Test Phase Code + Test Phase
Schedule
Release 1 Release 2
Original Plan
Reality
With CI
14. Benefits of Continuous Integration
◊ Avoids last-minute chaos at release dates
◊ Early warnings of broken code
◊ Early warning of conflicting changes
◊ Immediate testing of all changes
◊ High availability of a "current" build for testing,
demo, or release purposes
15. Don’t Do It All At Once
Do This First
1. Version Control
2. Build
Automation
3. Automated
Unit Testing
4. Continuous
Integration
Next Do This
5. Static Code
Analysis
6. Dependency
Management
Then Do This
7. Automated
Integration
Testing
8. Automated
Acceptance
Testing
9. Deployment
Automation
16. Next Do This
5) Static Code Analysis
– What: Checks for coding standards and code quality
– Effort: Low
– Sample Tools: Visual Studio Code Analysis, FindBugs
(Java), PMD, Cobertura, Sonar, CheckStyle
18. Next Do This
6) Dependency Management
– What: Manages 3rd party components and ensure we have
the latest / correct version these components
– Effort: Low
– Sample Tools: Gradle, NuGet
19. Don’t Do It All At Once
Do This First
1. Version Control
2. Build
Automation
3. Automated
Unit Testing
4. Continuous
Integration
Next Do This
5. Static Code
Analysis
6. Dependency
Management
Then Do This
7. Automated
Integration
Testing
8. Automated
Acceptance
Testing
9. Deployment
Automation
20. Then Do This
7. Automated Integration Testing
– What: Testing interaction between multiple components to
ensure our component dependencies don’t break
– Effort: Medium to High
– Sample Tools: DBUnit (Java), NDBUnit (MS)
21. Then Do This
8. Automated Acceptance Testing
– What: Automated testing to ensure the systems meets
business needs
– Effort: High
– Sample Tools: SpecFlow, Cucumber, FitNess
– For Browser Testing: Selenium, WatiN
22. Automated Tests
◊ Automated tests involves:
– Unit tests: Testing of smallest possible piece of code that can
operate in isolation
– Integration tests: Testing interaction between multiple
components
– Acceptance tests: Testing complete segments of a system to
ensure it meets the business needs
◊ Benefits
– Enables rapid discovery of root cause
– Reduces defect rate
– Saves time in system test
– Ensures testing starts early in cycle
– Provides for a free regression testing suite
23. Testing Quadrant
Functional Tests
Acceptance Tests
Unit Tests
Component Tests
System Tests
Showcases
Exploratory Tests
Usability Tests
Performance Tests
Security Tests
TheTeam
TheProduct
Functionality
The Systemfrom Brian Marick
24. Then Do This
8. Automated Deployment
– What: Push button deployment capabilities
– Effort: High
– Sample Tools: FluentMigrator, Puppet, Octopus
26. Benefits of Automated Deployments
◊ Reduces complexity by using standardized script
◊ Reduces risk by re-running same script
◊ Reduces cost by speeding up deployment
◊ Frees up resources to focus on new features
27. One More Thing - Developers
Developer’s Bill of Rights
1. Every programmer shall have two monitors
2. Every programmer shall have a fast PC
3. Every programmer shall have their choice of mouse
and keyboard
4. Every programmer shall have a comfortable chair
5. Every programmer shall have a fast internet
connection
6. Every programmer shall have {effective} working
conditions
Posted by Jeff Atwood , http://www.codinghorror.com/blog/2006/08/the-
programmers-bill-of-rights.html
28. Useful References
◊ Stephen Ritchie, Pro .NET Best Practices, Apress,
2011.
◊ Paul Duvall, Steve Matyas, and Andrew Glover,
Continuous Integration: Improving Software Quality
and Reducing Risk, Addison-Wesley, 2007.
◊ Jez Humble, Continuous Delivery: Reliable Software
Releases through Build, Test, and Deployment
Automation, Addison-Wesley, 2010
29. Excella Consulting
Experience and Expertise in Agile Solutions
– Coaching
– Training
– Assessments
– Agile Adoption
– Agile Development Teams
– Agile PMO
Training Courses
– Certified ScrumMaster (CSM)
– Certified Scrum Product Owner (CSPO): The Agile Business Analyst
– Advanced Certified Scrum Product Owner (CSPO)
– Certified Scrum Developer (CSD)
– Agile Testing
– Agile Business Intelligence and Data Warehousing
See http://www.excella.com/services/agile-training-services.aspx for more
information
30. Contact Information
Richard K Cheng
richard.cheng@excella.com
703-967-8620
http://www.excella.com
Twitter: @RichardKCheng
Editor's Notes
Best teams and best products where everyone is responsible for processQuality is not just a phaseIts more then just building quality into the processIt’s a shared responsibility for everyone
Whatever technology stack you use, this works inAreas outside Sharepoint, ETL/BI, BizTalk
thousands of committers around the world, spoke and hub models will not workTraditional is more command and control, distributed more peer to peer (collaborative) and spreads the load of administration
First list are test runners, they run testsSecond list, isolation facilities, for test isolation – create testFluent makes tests more readableRequires less coaching and more workshoppingSomeone to help coach you through the initial sets, the CSD is helpful for this
Maintence is lightstanding it up is easy if you have done it, tougher if you have not
FindBugs – potentially buggy codeCheckStyle – potential out of coding standardsPMD – shows a little bit of both
FindBugs – potentially buggy codeCheckStyle – potential out of coding standardsPMD – shows a little bit of bothCoverage report, shows how much of this code is being tested Shows how much code is being covered
ORM against DB as the exampleBiggest challenge revolves around the data and databaseHigh if integrating the components can be expensive OPM – background investigation is expensive because of massive infrastructure
Make the problem statement firstprolonged sprints or need for more testersvelocity is limited by number of testers