Aligning Software Testing With Modern Age Development Practices

1. Aligning Software Testing With
Modern Age Development Practices

2. Evolution of Development and Testing Practices
ITERATIVE &
INCREMENTAL
• Focus on product
rather than the process
• Focus on early
involvement in SDLC
• Tight coupling with
development
• Faster time to market
• Build quality in the
product
• Thin line between
developers and testers
‘V’ MODEL WATERFALL
MODEL
AGILE
• Smaller version of
waterfall model
• Generally late
involvement in the
SDLC
• Test Automation is
mainstream
• Focus on smaller
units
• Shift towards gray
box testing
• Parallel involvement
along with the
development in the
SDLC
• Test Automation is
still optional
• Predominantly
black box testing
• Generally late
involvement in
the SDLC
• Test Automation
is optional
• High cost of
quality
2

3. Agile: Management vs. Engineering Methods
MANAGEMENT METHODS ENGINEERING METHODS
XP
TDD, ATDD, BDD
3
SCRUM
Lean
Kanban
DevOps
CI/CD
Continuous Delivery

4. Current Business Trends
Top Digital Transformation Trends
Mobile First
Real-time
Customizations
Usability Security
Consumers spend over
60% of their time on
internet through their
smartphones.
70% of mobile
searches lead to
action on websites
within one hour.
57% of users will
abandon the website if
it takes three or more
seconds to load.
27% percent of senior
executives consider digital
transformation as now
being “a matter of survival.
Survey by CMO.com indicated that majority of the respondents rated current internal IT systems as the third biggest obstacle to
achieving digital transformation.
4

5. Challenges for IT
Lack of vision
Shorter time to market
Omni-channel delivery
Infrastructure limitations
High probability of failure
Lack of skilled resources
Continuously evolving technology
Scalable Tests
Availability of
Skilled Testers
Evolving
Requirements
Frequent
Testing
Less time to
test
Test across
devices
Challenges for QA
5

6. Solution Focus Areas
• 45%-65% defects are introduced
in Requirements, Architecture
and Design phases
• Pull data regularly and analyze
patterns
• Span all the way from design
through operations
Test
Early
All the
way
Metrics
6

7. The Solution – Quality Assurance to Quality Engineering
Shift Left Approach
Quality
Engineering
Principles
Review Req, Architecture and Design
Be a voice of the customer
Provide Actionable Data
Development Testing
Continuous Integration / Continuous Testing
Quality Built In
Complete Test Automation
7

8. Change Agents
8
- Digital Testing - SMAC
- Continuous Testing
- Automate everything
- Focus on NFRs
- Shift left approach to testing
- Quality Assurance to Quality Engineering
- Customer centric approach
• Thinning lines between tester & dev
• Higher involvement of business
People
Process
Technology

9. QA Solution
PROCESS

10. New Development Practices – A Snapshot
• Focus on modules &
units
• Write unit level tests
before development
• Develop features to
satisfy the tests
• Thorough unit testing
• Early feedback
• Incremental in nature
• Focus on overall product
• Write acceptance tests
before development
• Develop to satisfy the
acceptance criteria
• Outcome-based
• Focus on business value
• Early feedback
• Incremental in nature
• Focus on behavior of the
software
• Same visibility to all in the
Team
• Works towards the big
picture – Business value
• Automate right set of tests
• More effective testing efforts
• Tests in easy English/Native
languages
Test Driven
Development
(TDD)
Behaviour Driven
Development
(BDD)
Acceptance Test Driven
Development
(ATDD)
10

11. Behavior Driven Development/Testing - A Deep Dive
• Focus on requirements
• Identifies the different flows /
application behaviors
• Easy to understand the
scenarios
• Much useful for automating
multi channel applications
• Open Source savvy
• Greater ROI
• No longer costly to maintain
Extension of
TDD
Work towards the
big picture
Describes the
tests as expected
behavior
Shifting thinking
“tests” to thinking in
“behavior”
Behavior
based user
stories
Improved
collaboration
between
stakeholders
11

12. Why BDD/BDT?
Communicate
requirements
effectively
Mutually understandable
structure between BA,
QA and Dev
Executable
specifications
English language
like tests
Test faster
Built in quality
Cost
Effective
12

13. QA Solution
TECHNOLOGY

14. Challenges in Digital Testing
• Reliance on cloud for multi-device testing
• High percentage of automation for
regression tests
• Parallel test execution
• Continuous integration
• Reliance on open source tools
• Pixel-based image comparison tests for
internationalization testing
• Performance tests across simulated
network bandwidths
Solutions
Device
Fragmentation
Internationalization
Slower Test Cycles
Different Network
Bandwidths
Immature
Testing Process Identification of
Right Test Tools
14

15. Superior Customer
Experience
Business Performance &
Reliability
Increased Velocity &
Agility
Complex Integrations with
Tools, Services, Data
• Compatibility Testing
• Omni Channel QA
• ETL/Big Data Testing
• Usability Testing
• Accessibility Testing
• Localization Testing
• A/B Testing
• Performance Testing
• Scalability Testing
• Security Testing
• Agile Model
• Flexible Test Automation
Framework
• Easy Integration with Tools
<- Continuous Integration - >
Digital Transformation Testing – Key Technical Focus Areas
15

16. Tools & Technology
Cloud Security
TestingTest Process
16

17. Identifying the Right Test Tool
Tool
Identification
Test Script Execution of
devices
Prepare Object
Recognition Matrix
Develop Proof of Concept
with identified tools
Prepare fact sheets for
evaluation parameter
Recommend suitable tool
and suggest automation
approach
Evaluation Checklist
Technology
Based Parameters
a) .NET CF
b) Instrumentation Required?
c) Native Support
d) Other Technologies Support
Environment
Based Parameter
a) Windows CE
b) Intermac Devices
c) Emulators
Automation
Tool Parameters
Mandatory Parameters
a) Data Driven
b) Object Recognition
c) Ease of Script Development
d) Easier Script Maintenance
e) Total Ownership Cost
f) Pricing ($)
g) Talent Pool
Optional Parameters
a) Collaboration with Other Tools
b) Power of Languages Supported by Tool
c) Report Generation/Ease of Reports
d) UI Synchronization
e) Ease of Script Execution
f) Performance of script execution
17

18. Balancing Manual vs. Test Automation
• Test automation is staggered to one release further
• Delivered build from Sprint ‘n’ is automated & tested in
Sprint ‘n’ + 1
• Improves test design & script stability
• Sufficient time to provide higher coverage
• Improved product quality
• Higher return on investment with lower maintenance
The Sprint +1 Approach to Balance Manual & Test Automation (S+1 Approach)
Week 1
Sprint 1
Week 2
Sprint 2
Week 3
Sprint 3
Week 4
Sprint 4
• Code Development
• Manual Testing
• Automate Scenarios
• Run Regression Tests
• Sign Off
• Code Development
• Manual Testing
• Code Development
• Manual Testing
• Code Development
• Manual Testing
• Automate Scenarios
• Run Regression Tests
• Sign Off
• Automate Scenarios
• Run Regression Tests
• Sign Off
Build ‘n’ Build ‘n’ + 1 Build ‘n’ + 2
Build ‘n’ Build ‘n’ + 1 Build ‘n’ + 2 Build ‘n’ + 3
18

19. QA Solution
PEOPLE

20. • Predominantly manual functional testing
• No necessity to perform automated tests
• Focus on domain knowledge
• Act as the voice of the customer
• QA Engineers will not become obsolete, though
demand may go down
• Predominantly automated tests
• Build, deploy and maintain test automation suites
• Automate not just the application but also the test process
• Participate in architecture and design discussions
• May perform source code reviews and run white box tests
• High demand in market
• Skill gap in market
Improve product quality with higher test
coverage through test automation
Faster time to market with high degree of
automated tests that are scalable
High degree of reusability of test scripts to
save time and effort
BENEFITS of an SDET
Metamorphosis of the QA Engineer
QA Engineer Software Development Engineer in Test
20

21. The Future

22. INTERNET of
THINGS (IoT)
INTELLIGENT
QA
CROWD
SOURCED
TESTING
• Delegate testing to a community of QA experts
• Short turn around time
• Real world multiplatform testing
• Higher test coverage
• Higher sync with real time business scenarios
• IoT is connecting any piece of hardware in the internet e.g.
coffee maker, washing machines, head phones etc.
• There will be 26 billion connected devices by 2020, as per
Gartner.
• Testing will involve hardware and software
• Context based testing
• Improved quality through inference (machine learning &
artificial intelligence)
• Shift from defect detection to defect prediction
• E.g. Analyzing the social media sentiments to understand
what works and what needs fixing
• No way to ensure Non
Disclosure Agreement is upheld
by all testers
• Higher cost than offshore
testers
• Still at a nascent stage
• No integrated mechanism to
test hardware and software
• Still at a nascent stage
• No available industry tools to
run intelligent tests
• Lack of skilled resources
Highlights Challenges
The Next Big Things
22

23. Thank you