Devops Event Slides

1. Hello, Real World.
Building Better
Collaboration Between
Development and Testing
in a DevOps World

2. Organizations
practicing agile
development
Organizations have or
are planning a DevOps
initiative in their
company in the next
12 months
The difference in the
number of code
deployments between
high performing and
low performing IT
organizations
94% 71% 46x
Getting Faster Leads to Higher
Performance
Source: V1 2016 State of Agile, Puppet 2016 State of DevOps

3. INSERT PHOTO ON TOP
Design Code Test Deploy
Design Code Test Deploy
Waterfall
Agile
DevOps
1-4 releases a year
6-24 releases a year
50+ releases a year
Design Code Test Deploy
Continuous Integration
Demands Continuous
Feedback

4. “Functional testing is one of the most
crucial, time-consuming, and expensive
steps in continuous testing — so it’s
necessary to automate this testing, and to
automate it at higher levels than most Agile
teams achieve today.”
– Forrester

5. Jenkins

6. In real world DevOps, the question is not
“Do you have test automation?”…
…rather it is “How are you
approaching your test automation?”

7. How can teams align for quick and
accurate feedback on quality?
Promote
Collaboration
Drive an Agile Approach with
Collaboration Across all Teams
Clarify Up
Front
Integrate
Feedback
Adopt a Test-First
Approach
Integrate Testing
Directly with Builds

8. Promote
Collaboration
Clarify Up
Front
Integrate
Feedback

9. 9
Project
Management
TestingDevelopment Operations
The Old “Silo-ed” Approach

10. 10
Source: https://luis-goncalves.com/content/uploads/2014/08/Organisational-Structure.png
Head of
Development
New Matrix Approach

11. Capabilities Needed to Foster
Collaboration
Capability Why it’s Needed? How QAS Supports
Visibility into Testing For all Teams –
Dev, BA, Operations
Promote organizational quality
culture and allow entire team to
remove testing roadblocks
Embedded traceability and test
results within JIRA
Real Time Status Updates Between
JIRA and Other Systems
Make sure teams are all working on
the same version of the truth,
tougher when doing many
releases/day
Real time integration with JIRA,
Bamboo, Jenkins, and other DevOps
tools
Consolidated and Customizable
Dashboarding – Across Projects
Consolidate the various team level
activities and processes for executive
buy-in and visibility
qTest Insights – cross project
reporting and analytics and
embeddable dashboards

12. Promote
Collaboration
Clarify Up
Front
Integrate
Feedback

13. Moving Testing Up Front
Traditional Development Timeline
Moving Testing Up Front removes the risk of having to make compromises at
the end of the cycle on quality or on-time delivery:
There is risk in this process that any step, typically Code, will run over and either squeeze development,
or push release dates. TDD removes it!
Ends on: Day 1 Day 3 Day 14 Day 20 Day 21

14. Moving Testing Up Front
Traditional Approach
Test-first Approach

15. Benefits of BDD
Organization and Guidance
BDD offers more precise guidance on organizing the conversation between
developers, testers and domain experts
Collaboration
Notations have an everyday language and have a shallower learning curve
compared other automation frameworks
Easy Transition to Automation
Tools targeting a BDD approach generally afford the automatic Non technical and end user
documentation from BDD "specifications"

16. Traditional Testing Testing in BDD
Step
Description
Expected
Result
Actual
Result
Walk up to
machine
Machine is
present
Select Coffee Buy Coffee selected Columbian Coffee
Selected
Enter money Machine takes
money
Machine only took
cash
Press Button Coffee Comes
out
Coffee didn’t come
out…maybe I didn’t
have enough
money?
Grab Coffee Coffee is Hot Couldn’t grab
coffee
Scenario: Buy Last Coffee
Given There are 1 Coffees left
And I have deposited $5
And I previously have $100 in the bank
When I press the coffee button
Then I should be served a coffee
Feature: Serve Coffee
Coffee should not be served until paid
Coffee should not be served until the
button has been pressed
If there is no coffee left, then the
money should be refunded

17. Traditional Testing
Tests Cases Test Scenarios
Requirements Features / Specifications
Separate Teams Embedded Teams
Functional Tests Specifications of Product
Behavior
Step 1, 2, 3 Given…When…Then
Testing in BDD
Static Behavior Living Documentation

18. Capabilities Needed to Drive Test First
Adoption
Capability Why it’s Needed? How QAS Supports
Structured Syntax Support for
Automation Reuse
Structured Syntaxes like
Gherkin allow for automation
to be reused, reducing effort
and maintenance
Built in Gherkin Syntax
validation throughout Scenario
& Pulse
Support for Feature & Scenario
Creation in JIRA
Specifications replace
acceptance criteria and need
to be visible and defined in
JIRA
Enable BA, Dev, and Test to
define features and scenarios
on user story and other objects
Integration of tests into
Version Control Systems
Determine which tests should
fail and which should not when
using red-green—refactor
process
Integration to Github,
BitBucket, and other popular
VCS to allow for direct
versioning & storage of tests

19. Promote
Collaboration
Clarify Up
Front
Integrate
Feedback

20. Typical CI Workflow
Stop the line when tests fail
Feedback – test results, monitoring data, etc.
Dev Unit Test Staging Prod
Commit
Code
Build and
Test
Manual
Automated
Test
Deploy to
Prod and Test
Release
Automated
Trigger
Manual
Trigger
Plan
User
Stories

21. VERSION
CONTROL
BUILD & UNIT TEST
AUTOMATED AND
MANUAL TEST
DEPLOY
Check In
TESTS FAIL
Typical CI Workflow
PLAN
DEV
Trigger
Trigger
FEED BACK
Check In
Trigger
Trigger
TEST PASS
PUSH TO PROD
FEEDBACK

22. Capabilities Needed to Integrate
Feedback Early
Capability Why it’s Needed? How QAS Supports
Exploratory Testing Support
Quick feedback on new features
created as automated tests may be
under development
Explorer recording plugin with
session manager to centralize
feedback
CI/Build Tool Integration
Ingest results of build tests in real
time, link to user stories and defects
in JIRA, consolidate with adhoc
automated and manual tests
Out of the box plugins for Jenkins and
Bamboo
Customizable Workflows and Pipeline
Automation
Reduce manual work and process risk
– repeatable and real-time rules to
connect activities in the pipeline
Rules generation with webhooks to
facilitate promotion and status
changes based on testing progress

23. How is the Leading Enterprise Solution
Adapting to Change?

24. How QASymphony is Investing in the
Future to Support the Industry

25. Key Considerations to Make When
Selecting a New Vendor
Are they here to
Last?
Do they
Understand Where
I am Going?
Can they support
me in getting to
where I want to be?

26. Hello, Real World.
Q&A