Webinar: Going Agile? Don't Forget About Testing

September 25, 2015
Introduction to QASymphony
for [INSERT COMPANY
NAME]
#AgileTransformation
Agile Transformation: People,
Process and Tools to Make
Your Transformation Successful
The audio for this webinar is delivered through your computer. There is no dial-in
number. Make sure your speakers are turned up or use a pair of headphones.
Going Agile?
Don’t Forget About Testing

HOW TO WATCH THIS WEBINAR
• Audio for this webinar is delivered through your computer. Make sure
your speakers are turned up or use a set of headphones.
• If your audio quality is choppy, it could be your internet connection.
• You can customize your webinar viewing experience by increasing,
decreasing or minimizing the size of the widgets on your screen.
• If you have questions, enter them in the widget on the right.
2

• This webinar will be recorded and available on-demand tomorrow.
You will get an email when it is available.
• Join the conversation on Twitter using the hashtag
• Use the Q&A widgets to ask questions during the webinar.
• At the end of the webinar, you will be asked to take a short survey.
HOUSEKEEPING

NEXT WEBINAR

Robust test management platform purpose-built to help agile teams
accelerate software testing
ABOUT QASYMPHONY

ABOUT GALLOP
Analyst
Recognitions
Awards
Thought
Leadership
Certifications
Partnership with
Leading Tools
Vendor
2,000+
resources
worldwide
FY2015-16 turnover of
USD 85mn that grew at
a CAGR of ~58% over
the last 3 years
Currently serving
80 of the Forbes
Global 2000
Companies
Project Cignificance aims
to empower lives with free
and quality education
CSR Initiatives
Independent
Testing Services
Company in North
America
Publicly Listed
& CMMi L5
certified
5 continents, 10+
countries
Global presence
500+
Satisfied
Customers
58%+
Revenue Growth
Gallop
Thought Leadership
Total Quality Management
5 Patents*
Testing IP
400+ Testers in North America
1200+ Testers at the Global Delivery
Center in India
Certifications : ISTQB, HP, IBM, CSQA,
CEH, CISSP and more
Reduced Cost of Quality
*Pending

OTHER WEBINARS & RESOURCES
WWW.QASYMPHONY.COM/RESOURCES

PRESENTERS
Kevin Dunne
VP, Strategy and Business Development
QASymphony
@KevinDunneQA
Sujit Nair
Associate Director with the
Enterprise Solutions Group
Gallop
@Sujit_N

WHY COMPANIES ARE GOING AGILE
There are a number of microeconomic and macroeconomic trends pushing teams
towards adopting agile, some of the most prominent include:
App Economy
Instant feedback and
“needed it
yesterday”mentality
Cloud Adoption
Ability (and expectation)
for frequent releases
Best of Breed
Integrating several tools,
increasing complexity
Startup Competition
Fast moving startups with
funding disrupting
markets
Open Source
Ability to build quickly on
existing components
Digital Disruption
Shift from traditional
channels to mobile & web
platforms

WHAT DOES AGILE LOOK LIKE?
Agile differs from Waterfall in creating shorter, more integrated cycles that generate code more quickly,
allowing teams to react to changes as they come and reduce risk and waste
Source: http://www.kenwayconsulting.com/wp-content/uploads/2012/03/Agile-v-Waterfall.jpg

Making a successful Agile transformation is not as simple as selecting an agile process and
moving forward, it requires thoughtful planning across process, as well as people and tools
WHAT DO WE NEED TO CONSIDER?
Process
● What works well in
my current process?
What is not working
well?
● What do I value -
time to market, cost,
quality, something
else?
● What restrictions are
in place to keep us
from adopting agile?
Which can we move?
People
● What valuable skills
do my current
employees have?
Which are they
lacking?
● Is my organization
equipped to deal with
uncertainty?
● Can my people adopt
new skillsets needed
by agile? Mindsets?
Tools
● What tools are critical
to my process?
Which can be
replaced to assist in
transformation?
● What new tools and
categories of tools
will be required?
● How can I empower
smaller teams to use
and leverage tools?

Thank You
PROCESS

WHAT AGILE MEANS FOR TESTING
Agile development means more frequent development
cycles, often with more regular code delivery.
What does this mean for testing?
1. Testers must be involved and aligned with the entire
development process
2. Testing will happen much more regularly, in smaller
increments
3. Testing will follow development more closely, in tandem
with coding

HOW TO OPTIMIZE TESTING FOR AGILE
Get Testers Involved Early
Embed testers within product or scrum teams, making sure they have a seat at the table in
planning discussions and provide insights around quality and testing considerations
Embrace Frequent, if not Continuous, Testing
As agile promotes frequent code merges, builds, and deployments, employ a process that allows
the most vital testing to be identified and executed efficiently
Decrease the Time from Design to Test
With 2-3 agile sprints, we do not have weeks (or even days) to define and automate test cases,
adopt a process for manual or automated tests that allows tests to closely follow designs

GALLOP & AGILE TESTING
SDLC
Requirements
Gathering
Application
Design
Development Testing
Test Requirement
Analysis
Test Planning
& Strategy
Test Design
Text
Execution
Analyse Plan
Design
BuildTest
Deploy
Analyse Plan
Design
BuildTest
Deploy
People
Development Lead QA Lead
Project Manager
Developers Testers
Functional
Automation
Developer Architect
SCRUM Master
Sprint Team
ProcessTools
Product Owner
Process
Test Management Defect Management Automation
Service Test
High Focus on Compliance Focus on Outcomes
Process Heavy Project Based Delivery Lean Processes Product Based Delivery
Waterfall to Agile Transformation

Quality Assurance to Quality Engineering – The
Transition
Manual
Testing Skills
Specific
Testing Skills
Regression
Automation
Performance
Testing
Manual Test
Cases
Black Box Testing
Prolonged
Cycle Time
UI Testing
Standard Third
Party Tool Sets
Transform to
QE Skill-Set
Optimization by
Cross Training
Early Functional
Automation
Performance
Engineering
Automated Test
Case Generation
White Box Testing
Reduced
execution cycles
Database, Services &
Component Testing
Open Source Tools &
Commercial Tools
Quality
Assurance
(From)
Quality
Engineering
(To)
Courtesy by Lisa Crispin/Brian Marick
Agile Testing Quadrants

GALLOP’s AGILE ASESSMENT FRAMEWORK
Discover and
Document
Analyze
and
Benchmark
Roadmap &
Implementation
Approach
Goals, Constraints,
Timelines, Risks,
Key Focus Areas
Applications,
Technology, Platform,
Automation
Framework,
Automated scripts
Agile Processes,
Metrics, Tools Strategy,
Test Assets, Skills, Org
Map, Functional
Testing, Test
Environments, Test Data
Performance Goals,
Scripts & Reports
Assess Performance
Test Strategy,
Performance
Engineering, Monitoring
and related processes
Evaluate functional
testing assets, maturity
of QA processes &
benchmark based on
agile maturity model
Analyze Tool
Utilization, Licensing,
Integration
opportunities
Review Test
Automation Strategy,
Framework maturity,
test scripts &
coverage
Design the roadmap
based on the target
operating model
and prioritized key
focus areas
Identify & Prioritize
the key focus areas
Review the analysis and
findings with PoC & Key
Stakeholders
Design the Target
Operating Model
Assessment Plan Test Assets Inventory
Target Operating
Model
Agile QA Test
Maturity
Implementation
Roadmap
Interim Test
Assessment Report
Final Test Assessment
Report

Discover and Document
Create Test Assets Inventory and Review with the POC
Elicit the existing processes, tools, test assets related to functional testing, test automation, performance testing and also the team skills to
understand the AS-IS state and provide comprehensive documentation
Understand and Document Test Assets
Understand Organization
Setup and Day in the life
of QA
Understand the
existing processes
and test assets
Collect process
and project
artifacts
S.No TestAssetsInventory Overview AssetType Details
1 Agile/SCRUMGuide
GuidelinesforworkingwithSCRUMmethodology,
techniques
ProcessDocs 1
2 TestManagementTool TestManagementTool Tools 1
3 AutomationTool QTPToolOverview Tools 1
4 PerformanceTesting Performanceengineeringscripts Non-functionalScripts 200
5 ManualTestCasesforApp1 FunctionalTestCasesforApp1 FunctionalTestCase 1240
6 ManualTestCasesforApp2 FunctionalTestCasesforApp2 FunctionalTestCase 673
7 AutomatedTestScripts AutomatedFunctionalTestScriptsforApp1&2 AutomatedTestScripts 350
8 Environments
Environments-Dev,QA,Pre-Production,Production,
Demo,Training
Environments 5
9 TestOrgRoles TeamstructureandR&R RolesList 1

Analyze and Benchmark
Design Target Operating
Model
Overall Agile Test
Maturity
Customized Assessment Framework built using industry standard models such as SAFe, TPI, Agile & DevOps, ITIL, TMMi that helps to
benchmark and design target operating model for transformation.
Conduct Assessment
Map to the Industry Benchmarks and Standards
Analyze the Test Assets
Inventory from the
previous phase
Conduct
interviews with
stakeholders
Analysis of
defects and
artifacts
Conduct gap
analysis

Develop Roadmap
Based on the score, the recommendations are categorized into
Quick wins, and Medium & Long Term initiatives
Design the roadmap aligned to client’s objectives, that helps in prioritizing the initiatives and balances the risks &
impact, along with timelines
FocusAreas Reccomendations Deliverables Transformation Score
Easeof
Implementation
ROI
AgileProcesses DefineKeyAgileProcesses
AgileFramework
SCRUMKit
Deployinwave1 6 2 3
ToolsIntegration IntegrateJIRAwithZenDesk Integrateworkflowsintools Deployinwave2 2 1 2
AgileTestCoE
EstablishaFlexibleOrg.designwith
specializedandgeneralistsrolesinQA
TCoEOrg.Design
TeamsfullyalignedtoTCoE 3 1 3
PerformanceTesting
EstablishPerformanceCapabilityfor
prioritizedapplications
ApplicationPerformanceAnalysisReport
PerformanceScripts
Deployinwave3 6 2 3
Enabler EstablisheffectiveProcessfeedback
ProcessImprovementProcess
ProcessImprovementguidelines
ProcessImprovementtemplate
DeployProcessClinics
IdenitfyProcess
ImplementationChampions
6 2 3
TestManagementTool EstabslishTestManagementTool
ToolFeasibilityAnalysisreport
ToolImplementationGuide
Deployinwave3 4 2 2
TestAutomation EstablishTestAutomationCapability AutomationFeasibilityAnalysis
TestSuite,Scripts,datais
madeready
6 2 3
Analyze the recommendations based on ease of
Implementation and ROI

Case Study
» Client is a leading provider of face to face marketing services
ranging from Pre-Event Planning, On-Site Execution, Extending
the Experience to Audio Visual Solutions with a presence in
North America, Europe and Asia
» Key Requirement: Functional, Regression and Compatibility
Testing of four applications
» Application Environment: SharePoint and .NET Technology
» Testing Types: Functional, Regression and Compatibility Testing
» Delayed receipt of user stories in every sprint
» Lack of clarity in user story requirements
» Parallel sprints, each with a rigid schedule
» Lack of standardized QA processes and best practices
» Unstable Test Environment with configuration and access issues
» Critical defects leaked into post-production after regression
testing
Key Challenges
» Reduced RAI (Requirement Ambiguity Index) from 35% in sprint
1 to 12% in sprint 3 by improving the clarity of user stories
» Increased test coverage by leveraging Gallops Requirements
Testing Framework
» Reduced post-production defects by 40%
» Met aggressive sprint schedules with the help of reusable
process templates
» Introduced industry standard Agile and QA best practices to
ensure that every sprint stays on schedule
Key Benefits
» Leveraged Gallop Requirement Testing Framework to improve
requirements review process and hence the quality of user
stories
» Mandated the use of sprint demos to hasten understanding of
functionality being added in every sprint
» Worked with client team to resolve test environment issues
» Introduced reusable process templates to accelerate and
standardize QA processes across applications and sprints
» Published detailed test execution reports for the compatibility
matrix defined to enable easier and fast sprint sign-off
Solutions Provided
About the Client

Thank You
TOOLS

A NEW BREED OF TOOLS
Automation is undergoing even more dramatic changes than
manual testing, with a new breed to tools:
● Open source: to provide lower cost and allow more
members to contribute automation
● Cross Platform: to accommodate for multi channel
experience across web, mobile, & desktop
● DevOps Capable: reliable, with ability to be run quickly
and in parallel to allow for fast, frequent builds

OPEN SOURCE AUTOMATION
Every category of automation tool is undergoing pressure
from open source options:
Unit Testing
● jUnit
● nUnit/xUnit
UI Testing
● Selenium
● Protractor
Load/Performance
● jMeter
● Gatling
Build
● Jenkins
● Travis
Orchestration
● Puppet
● Chef
Configuration
Management
● Docker
● Selenium Grid

Early Automation Approach in Agile
Write Acceptance Test
Cases
Decision to
Automate
Test
Define Acceptance Test Criteria
Yes
Prototype Rinse & Refine Integrate Testing
Iterative design
Design/Update proto
types for the Business
Components at UI and API
level
Develop/Update the code,
User Interface controls and
Unit test cases
Unit testing, integration
testing and do bug fixes
No
User Story
Perform Manual Testing
Coordinate each other for developing Automation suite
Execute API level
automation scripts on
continuous
integration
environment till
Acceptance criteria is
met
Execute end-to-end
automation scripts on
developed UI
Design Pseudo
codes/Classes for API
Level Automation
Build API Level automation
scripts with negative and
positive scenarios
Execute API level
automation scripts on every
component of developed
code
Design Keywords for UI
Level automation
Build automation scripts
using Keywords for UI
Level automation
Capture UI Objects
properties and integrate
with UI automation
scripts
API Level Automation Testing
UI Level Automation Testing
Application Code Development
Perform goal based
Exploratory testing
Adopt Hybrid Risk
Based Testing for Test
optimization

Thank You
PEOPLE

MOVING FROM SILOS TO EMBEDDED TESTERS
Siloed Testers Embedded Testers

A NEW BREED OF TESTER
Testers in the past were able to work independently,
focusing on discrete issues with less collaboration
Agile testers do not have that luxury, they must be much
more:
1. Collaborative, as they might be a single tester working
inside a product team full of developers & analysts
2. Adaptable, as they might be required to wear 4 or 5 hats
in any given day
3. Well versed, as they may be reassigned to many different
projects within one organization

HOW TO DEVELOP TESTERS FOR AGILE
Testers in Agile should focus on learning a broader set of
skills, rather than going deep in one specific area:
Team Skills
● Collaborating in diverse
teams
● Managing conflict across
teams
● Hiring and retaining top
talent in fast growing
companies
● Creating a team culture
of trust and buy in
Technology Skills
● Understanding and
contributing to application
code
● Understanding and
contributing to automation
(functional, unit testing)
● Expanding knowledge of
additional testing types
(performance, security, etc.)
● Expanding knowledge of
build & release pipeline
Managerial Skills
● Motivating teams under
high pressure
● Estimating and planning
in agile organizations
● Managing agile project
on both time and budget

Q&A
Questions?

Thank You
PLEASE TAKE OUR SHORT SURVEY
Thank you

Webinar: Going Agile? Don't Forget About Testing

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (19)

More from QASymphony

More from QASymphony (20)

Recently uploaded

Recently uploaded (20)

Webinar: Going Agile? Don't Forget About Testing