Agile Testing and Test Automation

Naveen Kumar Singh
Agile Testing and Test Automation
+91 9810547500
naveenhome@gmail.com
Twitter - @naveenhome
Skype – naveen75home

Capacity Building Workshop by Me…
Scrum Developer Workshop
Product Discovery and Requirement Analysis
Test Driven Development
Behavior Driven Development
Agile Testing and Test Automation
Agile and Scrum Foundation
DevOps Foundation
Continuous Integration and Delivery
Agile Coaching and Facilitation
Agile Technical Practices

Program Overview
• This workshop focuses primarily on agile testing techniques and
processes in addition to the mindset and role of an agile tester.
• Learning outcomes include the ability to distinguish different
types of testing on an agile project.
• Understanding how business, development, and testing
personnel best collaborate on an agile development cadence.

Learning Objective
• What is Agile and available agile frameworks
• What is Agile Testing?
• Agile mindset and culture for continuous testing
• Agile Testing Quadrants and category
• Test Automation Pyramid
• Pair Programming, Pair Testing and Peer review
• Example Driven Development – Spec by Examples, ATDD, BDD
• Feature, story, unit and NFRs testing
• Roles and Responsibilities of Test Engineers and Managers
• Test Strategies and planning
• Agile testing metrics
• Successful delivery
• Test Environment and Infrastructure
• Working on Distributed Team

Risk in Software Development
Schedule slips
Project canceled
System goes sour
Defect rate
Business misunderstood
Business change
False feature rich
Staff turnover

Agile Manifesto
We are uncovering better ways of developing software by doing
it and helping others do it. Through this work we have come to
value
Individuals and
Interactions
Processes and Toolsover
Working Product
Comprehensive
Documentation
over
Customer Collaboration Contract Negotiationover
Responding to Change Following a Planover
That is, while there is value in the items on the right, we value
the items on the left more

Agile Principles
I. Our highest priority is to satisfy the customer through early and continuous delivery of
valuable software.
II. Welcome changing requirements, even late in development. Agile processes harness
change for the customer's competitive advantage
III. Deliver working software frequently, from a couple of weeks to a couple of months, with
a preference to the shorter timescale
IV. Business people and developers must work together daily throughout the project.
V. Build projects around motivated individuals. Give them the environment and support they
need, and trust them to get the job done.
VI. The most efficient and effective method of conveying information to and within a
development team is face-to-face conversation.
VII. Working software is the primary measure of progress.
VIII.Agile processes promote sustainable development. The sponsors, developers, and users
should be able to maintain a constant pace indefinitely.
IX. Continuous attention to technical excellence and good design enhances agility.
X. Simplicity--the art of maximizing the amount of work not done--is essential.
XI. The best architectures, requirements, and designs emerge from self-organizing teams.
XII. At regular intervals, the team reflects on how to become more effective, then tunes and
adjusts its behaviour accordingly.

Agile Methods
Scrum
XP
DSDM
FDD
LeSS
Kanban
AUP
DIY

How to select right process framework?
Source - Wikipedia

Agile Testing
Building quality into the product rather than trying to test it in
later.
How agile testing is different from traditional approach if team
still struggling to answer what testers will do at beginning of
sprint?

Traditional Approach
Specification
Coding
Test cases
Testing
Deploy
Rework

Found a bug during deployment?
Manager/
Customer
Testers Developer
Architect Business
Analyst
Product
Manager

Agile Testing
Let’s understand by a simulation game to understand why
agile testing is important?
Simulation will cover below:-
• Uncovering problem
• Whole Team Approach
• Agile Testing Principles

Whole Team Approach
• Sub-optimization Vs Whole Team
• Care about Cost of Delay
• Calculate Cost of Production
• Remove waste from software development

Waste in Software Development
Defects Overhead processes Waiting
Not-realizing
people's potential
Task switching Inventory
Motion Extra features

Testing Approach
Plan Quality - Test Strategy, Test Data, Infrastructure
Built-In Quality - Specification By Examples, API vs GUI
Testing, Regression Testing, NFR testing, Tools to test and
provide feedback
Validate Quality - Acceptance Test Driven Development, UAT,
CAT

Software Testing Pyramid
Automated UI Test
Integration Test
Unit Test
Manual Test
Anti-Patterns

Inverted Testing Pyramid
Automated
UI Test
Automated API Test
Automated Service Test
Automated Component Test
Automated Unit Test
Exploratory
Test
70%
20%
10%

Brian Marick’s Testing Quadrants
Functional Tests
Story Tests
Prototypes
Simulations
Exploratory Tests
Usability Tests
User Acceptance
Tests (UAT)
Performance Testing
Load Testing
Security testing
NFR Testing
Unit Tests
Component Tests
Integration Tests
Automated
& Manual
Manual
Automated Tools
SupportingTheTeam
CritiqueProduct
Business Facing
Technology Facing

Pair Programing
• Two developers work together to accomplish a single task.
Driver and Navigator
• One developer focus on details of task and other focus
fitment of task in whole project
• Produces code through conversation
• Reinforce good programming habits
• May be uncomfortable at first
• Two brains on one task and two sets of eye produce fewer
bugs
• Investment in continual training

Pair Programing
Team Work
High Morale
Simple Design
Comprehensive Testing
Collective Code Ownership
Good Development Practices

Pair Testing
• Similar to Pair Programming
• Transfer Knowledge
• Save Time
• Improve Communication
• Positive Energy
• Promote Negative Testing

Peer Review
• Peer code review
• Improve coding practices
• Faster feedback
• Encourage refactoring
• Improve collaboration

Building right or right product?
Business Failure
Useless Stuff
Business Success
Technical Debts
Specification By
Example
Build it Right
Build the Right Things

Specifications by Example
Impact Mapping
(Vision)
Story Mapping
(MVP & MMF)
User Story
(Specification)
As A Student I want to Purchase used Books Online So that I can
save money

What are Specifications by Example
 Thin Slices of System Behavior
 That Deliver Business Value
 Described as concrete examples
 That are potentially automatable
 To create executable specifications
 Captured in live documentation

Refining the Specifications
Specifications with examples are acceptance tests. Gojko Adzic
• Be Precise and make sure specs are testable
• Avoid “Scripts” and “Flows”
• Focus on business functionality not design
• Avoid UI details
• Avoid covering every possible combination

Specifications By Collaboration

• Hold Regularly
• Full Team workshop
• Three Amigos workshops
• Product Owner
• Developers
• Testers

Given_________________
When_________________
Then__________________
Gherkin
Given “James” want to upload new prescription
When “James” selected “DoctorVisit.jpeg”
And Click on “Record”
Then Prescription available on search page

Write Specification
Write examples for one specification.
In order to maintain health record
As a patient
I want to enter my doctor visit details

Discussion of Acceptance Criteria
If(user==“new”
{
object = user
}
Else
error
We should encourage
employee to update
ride
Login -> Click on
“New” and enter test
data and then Result
&%$^&

Collaboration of 3 amigos
Technical
Feasibility
Happy Path
Exceptions, Test
Data, Boundary
Conditions
Developer Business Tester

Collaboration of 3 Amigos
As a patient
• New prescription can be uploaded as Image
• Duplicate prescription not allowed
• Photo can be captured by application
• New prescription can be added manually also

Collaboration of 3 Amigos
As a patient
• New prescription can be uploaded as Image
• Books can be removed from shopping cart
• Shopping basket should be empty for new
user
• User should not be able to add same book
twice
Given “James” want to upload new prescription
When “James” selected “DoctorVisit.jpeg”
And Click on “Record”
Then “Prescription Uploaded” displayed

Gherkin
 Feature – Name of feature
 Scenario – Behavior to be developed
 Given – Pre-conditions
 When – Actions to be performed
 Then – Expected Result
 And – Use for multiple Given, When & Then
 But – Describe exception cases
 Scenario Outline – Define multiple scenarios
 Examples – Multiple Scenarios
 Background – Avoid repeated Given

Test First
Write Test
Write Code
Test Again
Refactor Code
Pass
Fail
Fail

Behavior Driven Development
BDD
Given Scenario
Then Expected
Outcome
When Perform
Action

Let’s get started
Hands-on Behavior Driven Development
• Write Feature to describe scenario in Gherkin
• Eclipse, Java, Cucumber, Selenium, Junit
• Create Test Runner class file
• Generate Steps file
• Write code to pass test

BDD - Characteristics
• A testable story (it should be the smallest unit that
fits in an iteration)
• The title should describe an activity
• The narrative should include a role, a feature, and a
benefit
• The scenario title should say what's different
• The scenario should be described in terms of Givens,
Events, and Outcomes
• The givens should define all of, and no more than,
the required context
• The event should describe the feature

BDD – Feature File
Feature: Checkout
In order to calculate price of groceries
As a Store Staff
I should be able to calculate price for groceries during
checkout
Scenario: Checkout a banana
Given The price of a “Banana” is 5
When I checkout 1 “Banana”
Then the total price should be 5

BDD – Test Runner Class
package test.java;
import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
@RunWith(Cucumber.class)
@CucumberOptions(features = "src/test/resources")
public class testrunner {
}

BDD – Run Output
1 Scenarios ([33m1 undefined[0m)
3 Steps ([33m3 undefined[0m)
0m0.000s
You can implement missing steps with the snippets below:
@Given("^The price of a "([^"]*)" is $(d+)$")
public void The_price_of_a_is_$(String arg1, int arg2) throws Throwable {
// Express the Regexp above with the code you wish you had
throw new PendingException();
}
@When("^I checkout (d+) "([^"]*)"$")
public void I_checkout(int arg1, String arg2) throws Throwable {
}
@Then("^the total price should be $(d+)$")
public void the_total_price_should_be_$(int arg1) throws Throwable {
throw new PendingException(); }

BDD – Step File
public class TestSteps {
}
@When("^I checkout (d+) "([^"]*)"$")
}
} }

BDD – Modified Step File
public class TestSteps extends TestCase {
String product;
int rate, qty;
product = arg1;
rate = arg2;
}
@When("^I checkout (d+) "([^"]*)"$")
qty=arg1;
}
if(product.equals("Banana"))
assertEquals(arg1, rate*qty); }}

BDD – Lifecycle
Your Project Features Scenarios Steps
Your System
Automation
Library
Support Code
Step
Definitions
Technology
Facing
Business
Facing

Gherkin – Background
Feature: Feedback when entering invalid credit card
details
In user testing we've seen a lot of people who made
mistakes entering their credit card. We need to be as
helpful as possible here to avoid losing users at this
crucial stage of the transaction.
Background:
Given I have chosen some items to buy
And I am about to enter my credit card details

Gherkin – Background
Scenario: Credit card number too short
When I enter a card number that's only 15 digits long
And all the other details are correct
And I submit the form
Then the form should be redisplayed
And I should see a message advising me of the correct
number of digits

Gherkin – Multiple AND
Scenario: Expiry date invalid
When I enter a card expiry date that's in the past
And all the other details are correct
And I submit the form
Then the form should be redisplayed
And I should see a message telling me the expiry date
must be wrong

Gherkin – Comments
# This feature covers the account transaction and
hardware-driver modules
Feature: Withdraw Cash
In order to buy beer
As an account holder
I want to withdraw cash from the ATM
# Can't figure out how to integrate with magic wand
interface
Scenario: Withdraw too much from an account in
credit
Given I have $50 in my account

Gherkin – Step Definitions
In Cucumber, results are a little more sophisticated
than a simple pass or fail. A scenario that’s been
executed can end up in any of the following states:
• Failed
• Pending
• Undefined
• Skipped
• Passed
These states are designed to help indicate the
progress that you make as you
develop your tests.

Gherkin – Test Runner Setup
Element Purpose Default
dryRun true (Skip execution of glue code) False
strict true (will fail execution if there are undefined or
pending steps)
False
glue where to look for glue code (stepdefs and hooks) {}
features the paths to the feature(s) False
monochrome whether or not to use monochrome output False
format what formatter(s) to use {}
tags hat tags in the features should be executed {}

Gherkin – Test Runner Setup
@CucumberOptions(
dryRun = false,
strict = true,
features = "src/test/features/com/sample",
glue = "com.sample",
tags = { "~@wip", "@executeThis" },
monochrome = true,
format = {"pretty", "html:target/cucumber",
"json:target_json/cucumber.json",
"junit:taget_junit/cucumber.xml“ }
)

Acceptance-TDD
• What is A-TDD?
• How A-TDD is different than BDD
• Inside – out Vs Outside – in
• Tools for ATDD
• How FitNesse is different than Cucumber

Let’s build software using A-TDD
Let’s try FitNesse
• Setup FitNesse
• Write tests on WIKI
• Create Fixture
• Develop system under test
• Execute Test from Wiki

• Unit test
• Why TDD
• How TDD is different than A-TDD
• What is Right-BICEP
• Mocking
• Available tools
• TDD vs A-TDD vs BDD

Unit Testing
• A unit test is to test one unit of work. Generally
one unit means of one requirement for one
method.
• Unit testing functional testing
• Unit testing is not also User Interface testing

Unit Testing - Characteristic
Isolated from the other code
Isolated from the other developers
Targeted
Repeatable
Predictable

Let’s try to develop a component using TDD
focusing Right-BICEP
Expected result must be RIGHT
B – Boundary Condition
I – Inverse Check
C – Cross check
E – Error Condition
P – Performance condition

Boundary Condition
• Totally bogus or inconsistent input values, such
as a file name of "!*W:Xn&Gi/w>g/h#WQ@".
• Badly formatted data, such as an e-mail address
without a top-level domain ("fred@foobar.").
• Empty or missing values (such as 0, 0:0, "", or
null).
• Values far in excess of reasonable expectations,
such as a person's age of 10,000 years.

Boundary Condition
• Duplicates in lists that shouldn't have
duplicates.
• Ordered lists that aren't, and vice-versa. Try
handing a pre-sorted list to a sort algorithm, for
instance - or even a reverse-sorted list.
• Things that arrive out of order, or happen out of
expected order, such as trying to print a
document before logging in, for instance.

Boundary Condition - CORRECT
Conformance – Does the value conform to an
expected format?
Ordering – Is the set of values ordered or
unordered?
Range – Is the value within reasonable range?
Reference – Does the code reference anything
external that isn’t under direct control of code?
Existence – Does the value exist? (not-null etc.)
Cardinality – Are there exactly enough values?
Time (absolute and relative) – everything happening
at right time?

TDD was introduced long back in 1999 as part of
Extreme Programming (XP) by a group of developers.
Basic philosophy is “Before you write code, think
about what it will do”.
Testing code once will not ensure future changes will
not impact to earlier tested code so TDD is something
you write once but run once, twice, three times etc.
TDD is risk averse programming, investing work in
the near term to avoid failures later on.

When TDD used as an application design
methodology, it works best when the business user is
engaged in the process to help the developer define
the logic that is being created. Even possible to define
a set of input and expected output.
TDD ensures quality code from the start. Write only
the code needed to pass the test so method is going to
have less code. Lesser the code lesser opportunity for
defects.

TDD ensures a high degree of conformity of the
business requirement
TDD helps keep unused code out of the system
TDD provides built-In regression testing

Write a
test
Compile
Fix
compilati
on error
Run Test
Write
Code
Run test
Refactor
code

Test Driven Development - Rules
First Law You may not write production code until
you have written a failing unit test.
Second Law You may not write more of a unit test
than is sufficient to fail, and not compiling is failing.
Third Law You may not write more production code
than is sufficient to pass the currently failing test.

Misconceptions
• Writing code 1st then Test is same like TDD
• TDD isn’t useful for designing architecture
• Takes too much time
• Too complex to learn and implement
• TDD is only good for small project

Test Double
Test double is just like replacing on-screen Actor with
stunt actor during film making.
4 ways to achieve test double
• Dummy
• Stub
• Mock
• Fake

Refactoring
• Refactoring is the act of changing the internal
implementation of a class or method with the
aim of making the code more readable and
maintainable.
• Refactoring also reduces the code’s overall
complexity without changing the external
behavior of the class or method.
• Refactoring allows you to continuously change
and improve your code.

Code Smells
Code smells are simply a collection of commonly
known and widely found code-based anti-patterns.

Technical Debts
Also known as design debt or code debt is "a
concept in programming that reflects the extra
development work that arises when code that is easy
to implement in the short run is used instead of
applying the best overall solution".
Technical debt can be compared to monetary debt. If
technical debt is not repaid, it can accumulate
'interest', making it harder to implement changes
later on. - Wikipedia

Extreme Programming Practices
Don’t Repeat Yourself
(DRY)
Keep It Simple, Stupid
(KISS)
You Aren’t Gonna Need
It (YAGNI)

Why To Refactor
Improve Code Readability
Rename Variables/
Methods/Classes
Extract Methods
Extract Classes
Write Tests
Collective Code Ownership
Improve Code
Maintainability
Automated unit testing
Feature envy code smell
Don’t repeat yourself
Inappropriate Intimacy code
smell
Lazy class code smell

When To Refactor
The Rule of Three: The first time you do
something, you just do it. The second time you do
something similar, you wince at the duplication,
but you do the duplicate thing anyway. The third
time you do something similar, you refactor.
When add new function/method
When you need to fix
During code review

Exploratory Testing
• How is different than other testing?
• Concurrent test design and execution
• Balancing it with automated test
• Why to perform exploratory testing?
• Does exploratory testing add value?
• Do we need to plan for exploratory testing?

Non-Functional Requirement
• Need for Non-Functional Requirement Testing
• What are the area needs to cover as part of NFR
• Isn’t it NFR part of customer requirement?
• Different type of NFR testing such performance,
security and compliances etc.

Test Strategies
• Strategy is a thought process not a documents
• Preventive vs Corrective
• Team is responsible not just test engineer
• Developer test vs Tester test
• Business centric vs Technology centric
• Based on – Goal, Risk, Opportunity and Constraints

Test Planning
• Infrastructures
• Tools
• Strategies
• Methodologies
• Customer focused
• Let’s Do It Yourself

Test Metrics
• Test pass/fail rates
• Defects discovery rate
• Regression test results
• Defects density
• Found and fixed cycle time
• Code/test coverage
• Requirement coverage
It’s a trap! Don’t fall for it. Focus on value delivery.

Environment and Infrastructure
Test Environment
Configuration Management
Automated Build
Continuous Integration
Continuous Delivery
Delivery Pipelines
Test Data Management

A software development practice where members of
the integrate their work frequently, usually each
person integrates at least daily- leading to multiple
integrations per day. Each integration is verified by an
automated build to detect integration errors as
quickly as possible. Many teams find that this
approach leads to significantly reduced integration
problems and allows a team to develop cohesive
software more rapidly – Martin Fowler

• All developers run private builds on their own
workstations before committing their code
• Developers commit their code to a version control
repository at least once a day.
• Integration builds occur several times a day on a
separate build machine.
• 100% of tests must pass for every build.

Source Code Management
This is where developers commit their changes.
Purpose of version control repository is to manage
changes to source code and other software assets
(docs, and database scripts).
Popular Version Control – SVN, TFS, CVS, GIT

CI Server
A CI server runs an integration build wherever a
change is committed to the version control
repository. Usually CI server is configured to check
version control repository every few minutes.
The CI servers will pull out latest changes and run
build scripts to produce new build/product.

Continuous Build but why?
Improve Product Quality
Save time and cost
Eliminate redundant task
Minimize “Bad Builds”
Eliminate dependency on individual
Need for continuous integration

Continuous Build
• Manual Build Vs Automated Build
• Automation using build tools vs Script
• Available Build Tools
• Server Type
• On-demand
• Scheduled
• Triggered
• Continuous Build using Maven/MS Build/Rake/Gradle

Build Script
• Build script is a single script or set of scripts to compile,
test, inspect and deploy software.
• Build script can be used without CI system. Ant, NAnt,
MSBuild and Rake etc are example of build tools.
• Team usually build through IDEs.
Continuous Integration is much more than just
compilation of code.

Continuous Testing and Inspection
Many consider CI without automated test is not useful. This is
not true because you can still execute many other types of
testing load, component and security etc.
Automated code inspections can be used to enhance the
quality of the software by enforcing rules.
You can use CI system to run these automatically against a
code base. Tools like FXCOP and Stylecop is popular for
Microsoft platform. SonarQube, CheckStyle, JavaNCSS and
CPD are for Java platform.

Continuous Integration - Practices
• Commit code frequently
• Don’t Commit broken code
• Fix broken build asap
• Write automated tests
• All tests must pass
• Run private build
• Avoid getting broken

Hands-On Exercise
Language – Java 7
Build Tool – Maven
SCM – GIT & GITHUB
TDD – Junit
ATDD – Cucumber and Selenium
Continuous Integration – Jenkins
Code Quality – Cobertura, CheckStyle and PMD

Role of Test Manager
See the whole Go See
Technical
Solutions
Improvement
Services

Collaboration - Distributed
Do they understand
what I am saying?
??

Distributed - 3 Amigos in Action
Given
When
Then
Let’s Write
Examples on
shared Screen

Agile Testing and Test Automation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to Agile Testing and Test Automation

Similar to Agile Testing and Test Automation (20)

More from Naveen Kumar Singh

More from Naveen Kumar Singh (20)

Recently uploaded

Recently uploaded (20)

Agile Testing and Test Automation

Editor's Notes