Lavanya Kalaiselvan and Arnab Majumdar delivered a session on "Redefining DevOps for seamless performance testing" at ATAGTR2020 ATAGTR2020 was the 5th Edition of Global Testing Retreat. Lavanya has over decade experience in software industry. She assure quality applications and always delighted about sharing the acquired knowledge. Arnab has nearly 11 years of experience of Software Quality Assurance in IT services primarily working as a Non-Functional Test Analyst. The video recording of the session is now available on the following link: https://youtu.be/PpUrU65i7xw To know more about #ATAGTR2020, please visit: https://gtr.agiletestingalliance.org/

1. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Re-Defining DevOps for
seamless performance testing
Lavanya Kalaiselvan
Arnab Majumdar

2. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
DevOps – Bright Spots
Implementation Challenges
A Case Study - Banking Payments
Implemented Solution
Automated Solutions
Intelligent Performance Analysis
Benefits

3. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
1460
Deploys per Year
106 times
Faster lead time
Less than hour
time to restore
7 times
lower change fail rate
DevOps – Bright Spots
DevOps market size is estimated to
grow from 2.90 billion USD in 2018 to 10.31
billion USD by 2023
Above data from the 2019 State of DevOps report shows how and why DevOps has grown in the IT market in last few years
63%
Improvement in quality of
the software deployments
63%
Growth in release cycles for
new software changes
55%
Improved Collaboration and
Co-operation
38%
Higher quality of code
production

4. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Implementation Challenges
Despite the stats, DevOps implementation has numerous constraints and
arguably, continuous performance testing is one of the key factor
pertaining to it
Higher
Inter-Dependence
Lengthy Test
Execution Cycles
Labour
Intensive
Demands Multi
Platform support
Requires special
expertise
• End to End Performance
testing with integrated
components
• Testing at later stage of
SDLC makes defects
resolutions costly
• Creation of load
testing scripts takes
significant time to
completion
• Test execution of
complex applications
takes up to 50% effort
of the entire testing
life cycle
• Test data mining and
preparation are
tedious tasks
• Testing environment
setup and health
check is mandatory
and time consuming
activity before every
test execution
• Log extraction and
analysis from different
components to present
a holistic view of
performance view
consumes significant
time and effort
• Tedious RCA process
requires extensive log
analysis to pin point
the problematic
component
• Demands technical
expertise to provide
tuning
recommendations

5. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
A Case Study - Banking Payments
High
Value
Payments
Complex
Architecture
with 1000’s
of rule engines
Monthly
release model
along with
regulatory &
Upgrade
activities
20+
application
changes
introduced per
release
Rate of increase
in usage every
quarter
Handles more
than 25K
payment
transactions
and settlements
Single Payment
traverses up to
14App tiers, 5
DB tiers and 28
interfaces
• A payment failure has impacts on different multitudes in a
bank’s business
• Significant surge in usage of online payments in recent
past due to product diversities
• High Value payments processing engine is engineered
across complex set of applications (#6) and interfaces
(#20) for seamless routing of every payment transaction
• Lots of labour-intensive and time consuming activities to
replicate application traffic
• Takes up to 4 hours to identify, isolate and analyse the
root cause of any performance failures

6. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
CreateProjectwithNFR
RequirementsinJira
Update Config file with the
business flow order
Automatic Code deployment
Automatic Code deployment
Execute Job
Runautomationscript
Sendnotificationabout
theJobstatus
Runautomation
script
Create HAR FILE
Custom Templates
Apply Templates
UploadScript
Trigger Job to execute
Performance Center test
Application Monitoring Details
Results
Collation
Report Analyser tool
PushResults
Results pulled for
‘Trend’ analysis
Automated Deep
Dive Report
Automated Report and
Email Alert with Report
initiate Test
preparatory Activities
Deep learning based
defect triage(ML
algorithms)
Custom code
Raise defect in Jira
Implemented DevOps Solution

7. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Automated Solutions for Smart NFT
 Record network traffic from Selenium scripts
 JAVA code snippet to convert record ed traffic Load Runner scripts
 Script enhancement ( parametrization, transaction names, text checks and necessary
correlations) with customised JAVA code
Scripting
 UNIX job
• Environment health check and the payment gateway connectivity
• Upload the batch and SEPA files onto SWIFT
 Z/OS plugin triggers the necessary batch jobs
 PC Rest API’s
• Upload the enhanced scripts and scenario design
Environment
Setup
 Triggered from Jenkins post each load test completion.
 Creates in-depth report on SLA breach(response time, achieved throughput and error
percentage).
 Capture timestamp of response time spikes.
 Provides test status based on end result after detailed analysis.
 Triggers Deep learning based tuning model in-case of a performance anomaly
Result
Analysis

8. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
STEP 1 – Application Hotspot Identification
(Classifier)
 Collation of logs from all the concerned
applications
 Python based log parser to analyse the application
and database logs
 Training deep learning model with the historical
dataset
 Define processing layers to enable the hierarchical
approach
 Keras neural network library with tensor
flow backend to build the Sequential model.
 ‘Dense’ layer type and ReLu activation used
to build the layers.
 Compiled with optimizer ‘Adam’ which sets
the learning rate during the training
 Pass the extracted logs as test data to isolate the
issue (Classification) and to identify the
application under concern
Classifier
Performance Analysis

9. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
STEP 2 – Root Cause Analysis (Regressor) –
Implementation In progress
 Exercise Deep dive analysis extracting the time-consuming
process/methods
 Train Feed forward neural network with multilayer
perceptron's
 Input layer - response time behaviour, throughput and
server utilisation metrics
 Output Layer – Performance score and responsible Hyper
Parameter
 Define processing layers to enable the hierarchical
approach
• Keras with sequential model
• Sigmoid Activation
 Define weightages for functional relevance
Regressor
Logs specific to
performance
Hotspot
Classifier
Performance Analysis
Defect Triage
Extract
process/methods
STEP3 – Defect Triage
 Pass the results through defect triage engine to provide
recommendations based on historical data
 In turn defect details and the recommendation will be used
to raise JIRA defects automatically

10. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)© 2018 Cognizant 10
Benefits
Reduced
Testing Effort
Reduced
Analysis Effort
No Additional
Cost
Increased
Revenue
Less Human
Dependency
Potential
Revenue benefits
is projected up
to
Reduced
dependency on
skilled resources
to carry out
tuning activities
Overall testing
effort per
release is
optimised by
40%
Result analysis
and triage effort
per release is
reduced by
55%
There is no
establishment
cost incurred as
existing tools
used in banking
infrastructure 50%

11. #ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)
Classification: Private
#ATAGTR2020
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media marketing, publishing it on ATA Blog or ATA social medial
channels(Provided due credit is given to me/us)