SlideShare a Scribd company logo
1 of 30
Download to read offline
“Is regression hindering your progression?”
#SMARTest #SmartRegression
T Ashok
Founder & CEO, STAG Software
linkedin.com/in/ashokstag
© 2018, STAG Software, www.stagsoftware.com
2
New/incremental features are the focus for
revenue generation & market expansion
whilst
bug fixes are necessary to ensure that
customers stay.
3
3
Whilst on the path of progression
towards revenue enhancement,
the challenge is:
3
Whilst on the path of progression
towards revenue enhancement,
the challenge is:
“Did I break stuff that were working well”?
3
Whilst on the path of progression
towards revenue enhancement,
the challenge is:
As product grows,
so does regression,
increasing costs &
threatening to
slow down releases.
“Did I break stuff that were working well”?
4
the challenge of CHANGE
4
the challenge of CHANGE
what to RE-TEST?1
how to RE-TEST efficiently?2
what not-to RE-TEST?3
5
Let us put together a simple model of
“WHAT-is-TESTING”
and use this to TEST/(RE)TEST smartly
6
DEPLOYS WELL L8
ATTRIBUTES MET L7
WORKS ON ALL ENVIRONMENTS L6
BUSINESS FLOWS CLEAN L5
FEATURES CLEAN L4
INTERNALS CLEAN L3
UI INTERFACE CLEAN L2
INPUTS CLEAN L1
F1 F2 F3 F4
what to test
test-for-what
Testing is a cartesian product of
what-to-test 

& 

test-for-what
7
DEPLOYS WELL L8
ATTRIBUTES MET L7
WORKS ON ALL ENVIRONMENTS L6
BUSINESS FLOWS CLEAN L5
FEATURES CLEAN L4
INTERNALS CLEAN L3
UI INTERFACE CLEAN L2
INPUTS CLEAN L1
F1 F2 F3 F4
what to test
test-for-what
structural
COMPONENT
technical
FEATURE
business
REQUIREMENT
user
FLOW
WHAT IS THE
EUT?
Testing is a cartesian product of
what-to-test 

& 

test-for-what
8
DEPLOYS WELL L8
ATTRIBUTES MET L7
WORKS ON ALL ENVIRONMENTS L6
BUSINESS FLOWS CLEAN L5
FEATURES CLEAN L4
INTERNALS CLEAN L3
UI INTERFACE CLEAN L2
INPUTS CLEAN L1
F1 F2 F3 F4
what to test
test-for-what
Testing is a cartesian product of
what-to-test 

& 

test-for-what
structural
COMPONENT
technical
FEATURE
business
REQUIREMENT
user
FLOW
WHAT IS THE
EUT?
deploys well?
attributes met?
works on all env?
business flows clean?
features clean?
internals clean?
UI interface clean?
inputs clean?
CLEANLINESS CRITERIA
to TEST FOR?
9
DEPLOYS WELL L8
ATTRIBUTES MET L7
WORKS ON ALL ENVIRONMENTS L6
BUSINESS FLOWS CLEAN L5
FEATURES CLEAN L4
INTERNALS CLEAN L3
UI INTERFACE CLEAN L2
INPUTS CLEAN L1
F1 F2 F3 F4
what to test
test-for-what
Regression testing is then
what-to-(RE)test 

x 

(RE)test-for-what
structural
COMPONENT
technical
FEATURE
business
REQUIREMENT
user
FLOW
WHAT IS THE
EU(RE)T?
10
DEPLOYS WELL L8
ATTRIBUTES MET L7
WORKS ON ALL ENVIRONMENTS L6
BUSINESS FLOWS CLEAN L5
FEATURES CLEAN L4
INTERNALS CLEAN L3
UI INTERFACE CLEAN L2
INPUTS CLEAN L1
F1 F2 F3 F4
what to test
test-for-what
deploys well?
attributes met?
works on all env?
business flows clean?
features clean?
internals clean?
UI interface clean?
inputs clean?
CLEANLINESS CRITERIA
to (RE)TEST FOR?
Regression testing is then
what-to-(RE)test 

x 

(RE)test-for-what
structural
COMPONENT
technical
FEATURE
business
REQUIREMENT
user
FLOW
WHAT IS THE
EU(RE)T?
11
SMART approach to tackling
the challenge of CHANGE
what to RE-TEST?1
what may be the affected entities that need to be (re)tested?
what criteria of these entities are to be (re)tested for?
12
L1
L2
L3
L4
E1
E2
E3
E4
structural component
technical feature
business requirement
user flow
1
Given an entity (say, an
component) that has
been modified
1. Analyze if this could affect
any of its other criteria ?
e.g. performance?
3. Next analyze if this could
affect any other similar
entity (say component) and
what criteria of that entity
4. Finally analyze which of the
larger entity (say feature)
that uses this entity could be
affected and also the
potential affected criteria
13
L1
L2
L3
L4
E1
E2
E3
E4
structural component
technical feature
business requirement
user flow
1
Given an entity (say, an
component) that has
been modified
1. Analyze if this could affect
any of its other criteria ?
e.g. performance?
2. Next analyze if this could
affect any other similar
entity (say component) and
what criteria of that entity
3. Finally analyze which of the
larger entity (say feature)
that uses this entity could be
affected and also the
potential affected criteria
2
14
L1
L2
L3
L4
E1
E2
E3
E4
structural component
technical feature
business requirement
user flow
1
2
3
Given an entity (say, an
component) that has
been modified
1. Analyze if this could affect
any of its other criteria ?
e.g. performance?
2. Next analyze if this could
affect any other similar
entity (say component) and
what criteria of that entity
3. Finally analyze which of the
larger entity (say feature) that
uses this entity could be
affected and also the
potential affected criteria
15
L1
L2
L3
L4
E1
E2
E3
E4
structural component
technical feature
business requirement
user flow
1
2
3
This is the technique
FAULT
PROPAGATION
ANALYSIS
in HBT
(Hypothesis Based Testing)
www.hbtcentral.org
GIVEN THE FOLLOWING
LEVELS OF QUALITY :
L8 DEPLOYS WELL
L7 ATTRIBUTES MET
L6 WORKS ON ALL ENVIRONMENTS
L5 BUSINESS FLOWS CLEAN
L4 FEATURES CLEAN
L3 INTERNALS CLEAN
L2 UI INTERFACE CLEAN
L1 INPUTS CLEAN
To enable focused
purposeful tests that
validates correctness of
inputs, interfaces, going on
to features & flows and
then correctness on all
environments, finally onto
system attributes and
deployment.
AID #1: 

FAULT PROPAGATION ANALYZER

Who is affected? WHAT-to-RETEST
What is affected?-RE-TES T for WHAT?
Analyse fault propagation level-wise
- within an entity (1)
- across entities (2,3)
Given an entity (say, an component)
that has been modified:
1. Analyze if this could any of its
other criteria e.g. performance?
2. Next analyze if this could affect
any other similar entity (say
component) and what criteria of
that entity
3. Finally analyze which of the
larger entity (say feature) that
uses this entity could be affected
and also the potential affected
criteria
17
SMART approach to tackling
the challenge of CHANGE
what to RE-TEST?1
how to RE-TEST efficiently?2
well, automating execution is most useful here, challenge is scripts to be in sync with scenarios
ensure that scenarios are segregated by levels so that scripts are shorter and maintainable
Fault propagation analysis
Given test scenarios/cases for an
entity that does a variety of
validations
Segregate them into quality levels
so that that they can be automated
easily ensuring that scripts are
simple and maintainable.


“LEVELISE” them

L8 DEPLOYS WELL
L7 ATTRIBUTES MET
L6 WORKS ON ALL ENVIRONMENTS
L5 BUSINESS FLOWS CLEAN
L4 FEATURES CLEAN
L3 INTERNALS CLEAN
L2 UI INTERFACE CLEAN
L1 INPUTS CLEAN
GIVEN THE FOLLOWING
LEVELS OF QUALITY :
L8 DEPLOYS WELL
L7 ATTRIBUTES MET
L6 WORKS ON ALL ENVIRONMENTS
L5 BUSINESS FLOWS CLEAN
L4 FEATURES CLEAN
L3 INTERNALS CLEAN
L2 UI INTERFACE CLEAN
L1 INPUTS CLEAN
To enable focused
purposeful tests that
validates correctness of
inputs, interfaces, going on
to features & flows and
then correctness on all
environments, finally onto
system attributes and
deployment.
*TC= Test Case
TC well structured into levels.
=> FOCUSED TEST CASES
very FIT for automation
1
TC at levels L1-L4 mixed up.
=>SYSTEM TEST includes DEVTEST
Potentially long scripts, brittle,
high maintenance
not FIT for automation
L4
L3
L2
L1
2
TC at levels L4-L7 mixed up.
=> TC validates FEATURES, FLOWS and
ATTRIBUTES
Scripts may do too much, fragile
high maintenance
not FIT for automation
L7
L6
L5
L4
3
AID #2: 

AUTOMATION FITNESS ANALYZER

Are your test cases well structured to enable rapid
automation/maintenance?
*TC= Test Case
Segregate them into quality levels
so that that they can be automated
easily ensuring that scripts are
simple and maintainable.
P
20
SMART approach to tackling
the challenge of CHANGE
what to RE-TEST?1
how to RE-TEST efficiently?2
what not-to RE-TEST?3
Automation analysis
Fault propagation analysis
as testing progresses, test cases don’t uncover defects
track test cases that do not yield defects , to not execute?
L8 DEPLOYS WELL
L7 ATTRIBUTES MET
L6 WORKS ON ALL ENVIRONMENTS
L5 BUSINESS FLOWS CLEAN
L4 FEATURES CLEAN
L3 INTERNALS CLEAN
L2 UI INTERFACE CLEAN
L1 INPUTS CLEAN
*TC= Test Case
Track test cases that do NOT yield
defects each cycle
Normalise defect/TC* ratio for
each cycle
Analyse by levels to see yield wrt
time
12.5
25
37.5
50
L6
12.5
25
37.5
50
L5
12.5
25
37.5
50
L4
C1 C2 C3 C4
test cycles
GIVEN THE FOLLOWING
LEVELS OF QUALITY :
L8 DEPLOYS WELL
L7 ATTRIBUTES MET
L6 WORKS ON ALL ENVIRONMENTS
L5 BUSINESS FLOWS CLEAN
L4 FEATURES CLEAN
L3 INTERNALS CLEAN
L2 UI INTERFACE CLEAN
L1 INPUTS CLEAN
To enable focused
purposeful tests that
validates correctness of
inputs, interfaces, going on
to features & flows and
then correctness on all
environments, finally onto
system attributes and
deployment.
*TC= Test Case
AID #3: 

YIELD ANALYZER

How is the test yield over time?
yield = outcome/effort i.e #defects/#TC_Executed
Normalise defect/TC* ratio for each cycle
and analyse by levels to see yield wrt time.
12.5
25
37.5
50
L6
12.5
25
37.5
50
L5
12.5
25
37.5
50
L4
C1 C2 C3 C4
test cycles
Track test cases that do NOT yield
defects each cycle
Normalise defect/TC* ratio for
each cycle
Analyse by levels to see yield wrt
time
P
23
SMART approach to tackling
the challenge of CHANGE
what to RE-TEST?1
how to RE-TEST efficiently?2
what not-to RE-TEST?3
Automation analysis
Fault propagation analysis
Yield analysis
AID #1: 

FAULT PROPAGATION ANALYZER

Who is affected? WHAT-to-RETEST
What is affected?-RE-TES T for WHAT?
Analyse fault propagation level-wise
- within an entity (1)
- across entities (2,3)
AID #3: 

YIELD ANALYZER

How is the test yield over time?
yield = outcome/effort i.e #defects/#TC_Executed
Normalise defect/TC* ratio for each cycle
and analyse by levels to see yield wrt time.
12.5
25
37.5
50
L6
12.5
25
37.5
50
L5
12.5
25
37.5
50
L4
C1 C2 C3 C4
test cycles
TC well structured into levels.
=> FOCUSED TEST CASES
very FIT for automation
1
TC at levels L1-L4 mixed up.
=>SYSTEM TEST includes DEVTEST
Potentially long scripts, brittle,
high maintenance
not FIT for automation
L4
L3
L2
L1
2
TC at levels L4-L7 mixed up.
=> TC validates FEATURES, FLOWS and
ATTRIBUTES
Scripts may do too much, fragile
high maintenance
not FIT for automation
L7
L6
L5
L4
3
AID #2: 

AUTOMATION FITNESS ANALYZER

Are your test cases well structured to enable rapid
automation/maintenance?
*TC= Test Case
25
26
Thank you !
Ash_Thiru

More Related Content

Similar to Is regression hindering your progression?

Automation testing
Automation testingAutomation testing
Automation testingTomy Rhymond
 
Maximizing SAP ABAP Performance
Maximizing SAP ABAP PerformanceMaximizing SAP ABAP Performance
Maximizing SAP ABAP PerformancePeterHBrown
 
VEERAREDDY_TIBCO_MULESOFT_CONSULTANT
VEERAREDDY_TIBCO_MULESOFT_CONSULTANTVEERAREDDY_TIBCO_MULESOFT_CONSULTANT
VEERAREDDY_TIBCO_MULESOFT_CONSULTANTVeera Reddy Bapthu
 
Software testing and_quality_assurance_powerpoint_presentation
Software testing and_quality_assurance_powerpoint_presentationSoftware testing and_quality_assurance_powerpoint_presentation
Software testing and_quality_assurance_powerpoint_presentationvigneshasromio
 
Software Testing interview - Q&A and tips
Software Testing interview - Q&A and tipsSoftware Testing interview - Q&A and tips
Software Testing interview - Q&A and tipsPankaj Dubey
 
Chapter 10 Testing and Quality Assurance1Unders.docx
Chapter 10 Testing and Quality Assurance1Unders.docxChapter 10 Testing and Quality Assurance1Unders.docx
Chapter 10 Testing and Quality Assurance1Unders.docxketurahhazelhurst
 
A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...Vincenzo Ferme
 
Unlocking IT Value Chain with DevOps
Unlocking IT Value Chain with DevOpsUnlocking IT Value Chain with DevOps
Unlocking IT Value Chain with DevOpsBart Driscoll
 
Testing practicies not only in scala
Testing practicies not only in scalaTesting practicies not only in scala
Testing practicies not only in scalaPaweł Panasewicz
 
SE - Lecture 8 - Software Testing State Diagram.pptx
SE - Lecture 8 - Software Testing  State Diagram.pptxSE - Lecture 8 - Software Testing  State Diagram.pptx
SE - Lecture 8 - Software Testing State Diagram.pptxTangZhiSiang
 
Software testing
Software testingSoftware testing
Software testingEng Ibrahem
 
Top 20 software testing interview questions for sdet
Top 20 software testing interview questions for sdetTop 20 software testing interview questions for sdet
Top 20 software testing interview questions for sdetDevLabs Alliance
 
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance
 
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance
 
Solving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous TestingSolving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous TestingPerfecto by Perforce
 
Automation Testing theory notes.pptx
Automation Testing theory notes.pptxAutomation Testing theory notes.pptx
Automation Testing theory notes.pptxNileshBorkar12
 

Similar to Is regression hindering your progression? (20)

Testing
TestingTesting
Testing
 
Automation testing
Automation testingAutomation testing
Automation testing
 
How to test less and accomplish more
How to test less and accomplish moreHow to test less and accomplish more
How to test less and accomplish more
 
Maximizing SAP ABAP Performance
Maximizing SAP ABAP PerformanceMaximizing SAP ABAP Performance
Maximizing SAP ABAP Performance
 
VEERAREDDY_TIBCO_MULESOFT_CONSULTANT
VEERAREDDY_TIBCO_MULESOFT_CONSULTANTVEERAREDDY_TIBCO_MULESOFT_CONSULTANT
VEERAREDDY_TIBCO_MULESOFT_CONSULTANT
 
Software testing and_quality_assurance_powerpoint_presentation
Software testing and_quality_assurance_powerpoint_presentationSoftware testing and_quality_assurance_powerpoint_presentation
Software testing and_quality_assurance_powerpoint_presentation
 
Software Testing interview - Q&A and tips
Software Testing interview - Q&A and tipsSoftware Testing interview - Q&A and tips
Software Testing interview - Q&A and tips
 
Chapter 10 Testing and Quality Assurance1Unders.docx
Chapter 10 Testing and Quality Assurance1Unders.docxChapter 10 Testing and Quality Assurance1Unders.docx
Chapter 10 Testing and Quality Assurance1Unders.docx
 
A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...
 
Unlocking IT Value Chain with DevOps
Unlocking IT Value Chain with DevOpsUnlocking IT Value Chain with DevOps
Unlocking IT Value Chain with DevOps
 
Testing practicies not only in scala
Testing practicies not only in scalaTesting practicies not only in scala
Testing practicies not only in scala
 
SE - Lecture 8 - Software Testing State Diagram.pptx
SE - Lecture 8 - Software Testing  State Diagram.pptxSE - Lecture 8 - Software Testing  State Diagram.pptx
SE - Lecture 8 - Software Testing State Diagram.pptx
 
Improving Defect Yield - a three step approach
Improving Defect Yield - a three step approachImproving Defect Yield - a three step approach
Improving Defect Yield - a three step approach
 
Software testing
Software testingSoftware testing
Software testing
 
Top 20 software testing interview questions for sdet
Top 20 software testing interview questions for sdetTop 20 software testing interview questions for sdet
Top 20 software testing interview questions for sdet
 
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
 
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
DevLabs Alliance Top 20 Software Testing Interview Questions for SDET - by De...
 
Solving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous TestingSolving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous Testing
 
Ooad
OoadOoad
Ooad
 
Automation Testing theory notes.pptx
Automation Testing theory notes.pptxAutomation Testing theory notes.pptx
Automation Testing theory notes.pptx
 

More from STAG Software Private Limited

Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"STAG Software Private Limited
 
Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)STAG Software Private Limited
 
Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)STAG Software Private Limited
 
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...STAG Software Private Limited
 
Think better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachThink better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachSTAG Software Private Limited
 
Test Process Transformation Protects Product Development Investment
Test Process Transformation Protects Product Development InvestmentTest Process Transformation Protects Product Development Investment
Test Process Transformation Protects Product Development InvestmentSTAG Software Private Limited
 
Agile Sutra "Do more by doing less, Prevent rather than detect"
Agile Sutra "Do more by doing less, Prevent rather than detect"Agile Sutra "Do more by doing less, Prevent rather than detect"
Agile Sutra "Do more by doing less, Prevent rather than detect"STAG Software Private Limited
 

More from STAG Software Private Limited (20)

Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"
 
Choked by technical debt?
Choked by technical debt?Choked by technical debt?
Choked by technical debt?
 
Are your quality metrics insightful?
Are your quality metrics insightful?Are your quality metrics insightful?
Are your quality metrics insightful?
 
Weighed down by automation?
Weighed down by automation?Weighed down by automation?
Weighed down by automation?
 
Covid19 and Clean Code Part 2 - Process & Criteria
Covid19 and Clean Code Part 2 - Process & CriteriaCovid19 and Clean Code Part 2 - Process & Criteria
Covid19 and Clean Code Part 2 - Process & Criteria
 
Seven Thinking Tools to Test Rapidly
Seven Thinking Tools to Test RapidlySeven Thinking Tools to Test Rapidly
Seven Thinking Tools to Test Rapidly
 
The Power of Checklist
The Power of ChecklistThe Power of Checklist
The Power of Checklist
 
The power of checklist
The power of checklist The power of checklist
The power of checklist
 
Webinar - 'Test Case Immunity’- Optimize testing
Webinar - 'Test Case Immunity’- Optimize testing Webinar - 'Test Case Immunity’- Optimize testing
Webinar - 'Test Case Immunity’- Optimize testing
 
Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)
 
Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)
 
Language shapes the way you think
Language shapes the way you thinkLanguage shapes the way you think
Language shapes the way you think
 
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
 
Are Your Test Cases Fit For Automation?
Are Your Test Cases Fit For Automation?Are Your Test Cases Fit For Automation?
Are Your Test Cases Fit For Automation?
 
Think better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachThink better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” Approach
 
Regression Suite Optimization
Regression Suite OptimizationRegression Suite Optimization
Regression Suite Optimization
 
Reflect and Change
Reflect and ChangeReflect and Change
Reflect and Change
 
Test Process Transformation Protects Product Development Investment
Test Process Transformation Protects Product Development InvestmentTest Process Transformation Protects Product Development Investment
Test Process Transformation Protects Product Development Investment
 
Agile Sutra "Do more by doing less, Prevent rather than detect"
Agile Sutra "Do more by doing less, Prevent rather than detect"Agile Sutra "Do more by doing less, Prevent rather than detect"
Agile Sutra "Do more by doing less, Prevent rather than detect"
 
Intelligent Automation and Smart Tooling
Intelligent Automation and Smart ToolingIntelligent Automation and Smart Tooling
Intelligent Automation and Smart Tooling
 

Recently uploaded

ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 

Recently uploaded (20)

ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 

Is regression hindering your progression?

  • 1. “Is regression hindering your progression?” #SMARTest #SmartRegression T Ashok Founder & CEO, STAG Software linkedin.com/in/ashokstag © 2018, STAG Software, www.stagsoftware.com
  • 2. 2 New/incremental features are the focus for revenue generation & market expansion whilst bug fixes are necessary to ensure that customers stay.
  • 3. 3
  • 4. 3 Whilst on the path of progression towards revenue enhancement, the challenge is:
  • 5. 3 Whilst on the path of progression towards revenue enhancement, the challenge is: “Did I break stuff that were working well”?
  • 6. 3 Whilst on the path of progression towards revenue enhancement, the challenge is: As product grows, so does regression, increasing costs & threatening to slow down releases. “Did I break stuff that were working well”?
  • 8. 4 the challenge of CHANGE what to RE-TEST?1 how to RE-TEST efficiently?2 what not-to RE-TEST?3
  • 9. 5 Let us put together a simple model of “WHAT-is-TESTING” and use this to TEST/(RE)TEST smartly
  • 10. 6 DEPLOYS WELL L8 ATTRIBUTES MET L7 WORKS ON ALL ENVIRONMENTS L6 BUSINESS FLOWS CLEAN L5 FEATURES CLEAN L4 INTERNALS CLEAN L3 UI INTERFACE CLEAN L2 INPUTS CLEAN L1 F1 F2 F3 F4 what to test test-for-what Testing is a cartesian product of what-to-test & test-for-what
  • 11. 7 DEPLOYS WELL L8 ATTRIBUTES MET L7 WORKS ON ALL ENVIRONMENTS L6 BUSINESS FLOWS CLEAN L5 FEATURES CLEAN L4 INTERNALS CLEAN L3 UI INTERFACE CLEAN L2 INPUTS CLEAN L1 F1 F2 F3 F4 what to test test-for-what structural COMPONENT technical FEATURE business REQUIREMENT user FLOW WHAT IS THE EUT? Testing is a cartesian product of what-to-test & test-for-what
  • 12. 8 DEPLOYS WELL L8 ATTRIBUTES MET L7 WORKS ON ALL ENVIRONMENTS L6 BUSINESS FLOWS CLEAN L5 FEATURES CLEAN L4 INTERNALS CLEAN L3 UI INTERFACE CLEAN L2 INPUTS CLEAN L1 F1 F2 F3 F4 what to test test-for-what Testing is a cartesian product of what-to-test & test-for-what structural COMPONENT technical FEATURE business REQUIREMENT user FLOW WHAT IS THE EUT? deploys well? attributes met? works on all env? business flows clean? features clean? internals clean? UI interface clean? inputs clean? CLEANLINESS CRITERIA to TEST FOR?
  • 13. 9 DEPLOYS WELL L8 ATTRIBUTES MET L7 WORKS ON ALL ENVIRONMENTS L6 BUSINESS FLOWS CLEAN L5 FEATURES CLEAN L4 INTERNALS CLEAN L3 UI INTERFACE CLEAN L2 INPUTS CLEAN L1 F1 F2 F3 F4 what to test test-for-what Regression testing is then what-to-(RE)test x (RE)test-for-what structural COMPONENT technical FEATURE business REQUIREMENT user FLOW WHAT IS THE EU(RE)T?
  • 14. 10 DEPLOYS WELL L8 ATTRIBUTES MET L7 WORKS ON ALL ENVIRONMENTS L6 BUSINESS FLOWS CLEAN L5 FEATURES CLEAN L4 INTERNALS CLEAN L3 UI INTERFACE CLEAN L2 INPUTS CLEAN L1 F1 F2 F3 F4 what to test test-for-what deploys well? attributes met? works on all env? business flows clean? features clean? internals clean? UI interface clean? inputs clean? CLEANLINESS CRITERIA to (RE)TEST FOR? Regression testing is then what-to-(RE)test x (RE)test-for-what structural COMPONENT technical FEATURE business REQUIREMENT user FLOW WHAT IS THE EU(RE)T?
  • 15. 11 SMART approach to tackling the challenge of CHANGE what to RE-TEST?1 what may be the affected entities that need to be (re)tested? what criteria of these entities are to be (re)tested for?
  • 16. 12 L1 L2 L3 L4 E1 E2 E3 E4 structural component technical feature business requirement user flow 1 Given an entity (say, an component) that has been modified 1. Analyze if this could affect any of its other criteria ? e.g. performance? 3. Next analyze if this could affect any other similar entity (say component) and what criteria of that entity 4. Finally analyze which of the larger entity (say feature) that uses this entity could be affected and also the potential affected criteria
  • 17. 13 L1 L2 L3 L4 E1 E2 E3 E4 structural component technical feature business requirement user flow 1 Given an entity (say, an component) that has been modified 1. Analyze if this could affect any of its other criteria ? e.g. performance? 2. Next analyze if this could affect any other similar entity (say component) and what criteria of that entity 3. Finally analyze which of the larger entity (say feature) that uses this entity could be affected and also the potential affected criteria 2
  • 18. 14 L1 L2 L3 L4 E1 E2 E3 E4 structural component technical feature business requirement user flow 1 2 3 Given an entity (say, an component) that has been modified 1. Analyze if this could affect any of its other criteria ? e.g. performance? 2. Next analyze if this could affect any other similar entity (say component) and what criteria of that entity 3. Finally analyze which of the larger entity (say feature) that uses this entity could be affected and also the potential affected criteria
  • 19. 15 L1 L2 L3 L4 E1 E2 E3 E4 structural component technical feature business requirement user flow 1 2 3 This is the technique FAULT PROPAGATION ANALYSIS in HBT (Hypothesis Based Testing) www.hbtcentral.org
  • 20. GIVEN THE FOLLOWING LEVELS OF QUALITY : L8 DEPLOYS WELL L7 ATTRIBUTES MET L6 WORKS ON ALL ENVIRONMENTS L5 BUSINESS FLOWS CLEAN L4 FEATURES CLEAN L3 INTERNALS CLEAN L2 UI INTERFACE CLEAN L1 INPUTS CLEAN To enable focused purposeful tests that validates correctness of inputs, interfaces, going on to features & flows and then correctness on all environments, finally onto system attributes and deployment. AID #1: FAULT PROPAGATION ANALYZER Who is affected? WHAT-to-RETEST What is affected?-RE-TES T for WHAT? Analyse fault propagation level-wise - within an entity (1) - across entities (2,3) Given an entity (say, an component) that has been modified: 1. Analyze if this could any of its other criteria e.g. performance? 2. Next analyze if this could affect any other similar entity (say component) and what criteria of that entity 3. Finally analyze which of the larger entity (say feature) that uses this entity could be affected and also the potential affected criteria
  • 21. 17 SMART approach to tackling the challenge of CHANGE what to RE-TEST?1 how to RE-TEST efficiently?2 well, automating execution is most useful here, challenge is scripts to be in sync with scenarios ensure that scenarios are segregated by levels so that scripts are shorter and maintainable Fault propagation analysis
  • 22. Given test scenarios/cases for an entity that does a variety of validations Segregate them into quality levels so that that they can be automated easily ensuring that scripts are simple and maintainable. 
 “LEVELISE” them L8 DEPLOYS WELL L7 ATTRIBUTES MET L6 WORKS ON ALL ENVIRONMENTS L5 BUSINESS FLOWS CLEAN L4 FEATURES CLEAN L3 INTERNALS CLEAN L2 UI INTERFACE CLEAN L1 INPUTS CLEAN
  • 23. GIVEN THE FOLLOWING LEVELS OF QUALITY : L8 DEPLOYS WELL L7 ATTRIBUTES MET L6 WORKS ON ALL ENVIRONMENTS L5 BUSINESS FLOWS CLEAN L4 FEATURES CLEAN L3 INTERNALS CLEAN L2 UI INTERFACE CLEAN L1 INPUTS CLEAN To enable focused purposeful tests that validates correctness of inputs, interfaces, going on to features & flows and then correctness on all environments, finally onto system attributes and deployment. *TC= Test Case TC well structured into levels. => FOCUSED TEST CASES very FIT for automation 1 TC at levels L1-L4 mixed up. =>SYSTEM TEST includes DEVTEST Potentially long scripts, brittle, high maintenance not FIT for automation L4 L3 L2 L1 2 TC at levels L4-L7 mixed up. => TC validates FEATURES, FLOWS and ATTRIBUTES Scripts may do too much, fragile high maintenance not FIT for automation L7 L6 L5 L4 3 AID #2: AUTOMATION FITNESS ANALYZER Are your test cases well structured to enable rapid automation/maintenance? *TC= Test Case Segregate them into quality levels so that that they can be automated easily ensuring that scripts are simple and maintainable.
  • 24. P 20 SMART approach to tackling the challenge of CHANGE what to RE-TEST?1 how to RE-TEST efficiently?2 what not-to RE-TEST?3 Automation analysis Fault propagation analysis as testing progresses, test cases don’t uncover defects track test cases that do not yield defects , to not execute?
  • 25. L8 DEPLOYS WELL L7 ATTRIBUTES MET L6 WORKS ON ALL ENVIRONMENTS L5 BUSINESS FLOWS CLEAN L4 FEATURES CLEAN L3 INTERNALS CLEAN L2 UI INTERFACE CLEAN L1 INPUTS CLEAN *TC= Test Case Track test cases that do NOT yield defects each cycle Normalise defect/TC* ratio for each cycle Analyse by levels to see yield wrt time 12.5 25 37.5 50 L6 12.5 25 37.5 50 L5 12.5 25 37.5 50 L4 C1 C2 C3 C4 test cycles
  • 26. GIVEN THE FOLLOWING LEVELS OF QUALITY : L8 DEPLOYS WELL L7 ATTRIBUTES MET L6 WORKS ON ALL ENVIRONMENTS L5 BUSINESS FLOWS CLEAN L4 FEATURES CLEAN L3 INTERNALS CLEAN L2 UI INTERFACE CLEAN L1 INPUTS CLEAN To enable focused purposeful tests that validates correctness of inputs, interfaces, going on to features & flows and then correctness on all environments, finally onto system attributes and deployment. *TC= Test Case AID #3: YIELD ANALYZER How is the test yield over time? yield = outcome/effort i.e #defects/#TC_Executed Normalise defect/TC* ratio for each cycle and analyse by levels to see yield wrt time. 12.5 25 37.5 50 L6 12.5 25 37.5 50 L5 12.5 25 37.5 50 L4 C1 C2 C3 C4 test cycles Track test cases that do NOT yield defects each cycle Normalise defect/TC* ratio for each cycle Analyse by levels to see yield wrt time
  • 27. P 23 SMART approach to tackling the challenge of CHANGE what to RE-TEST?1 how to RE-TEST efficiently?2 what not-to RE-TEST?3 Automation analysis Fault propagation analysis Yield analysis
  • 28. AID #1: FAULT PROPAGATION ANALYZER Who is affected? WHAT-to-RETEST What is affected?-RE-TES T for WHAT? Analyse fault propagation level-wise - within an entity (1) - across entities (2,3) AID #3: YIELD ANALYZER How is the test yield over time? yield = outcome/effort i.e #defects/#TC_Executed Normalise defect/TC* ratio for each cycle and analyse by levels to see yield wrt time. 12.5 25 37.5 50 L6 12.5 25 37.5 50 L5 12.5 25 37.5 50 L4 C1 C2 C3 C4 test cycles TC well structured into levels. => FOCUSED TEST CASES very FIT for automation 1 TC at levels L1-L4 mixed up. =>SYSTEM TEST includes DEVTEST Potentially long scripts, brittle, high maintenance not FIT for automation L4 L3 L2 L1 2 TC at levels L4-L7 mixed up. => TC validates FEATURES, FLOWS and ATTRIBUTES Scripts may do too much, fragile high maintenance not FIT for automation L7 L6 L5 L4 3 AID #2: AUTOMATION FITNESS ANALYZER Are your test cases well structured to enable rapid automation/maintenance? *TC= Test Case
  • 29. 25