Tutorial: Implementing Specification-By-Example with Gherkin

CHRISTIAN HASSA (CH@TECHTALK.AT)
TWITTER: @CHRISHASSA
XP 2013 Vienna, June 3rd 2013
Specification-By-Example
with Gherkin

About me
• Managing Partner at TechTalk
• Working as/coaching Product Owners
TechTalk office Vienna/Austria

• Agile consulting and delivery
• Offices: Vienna, Budapest, Zurich
TechTalk Team
About my Team

5
Impact Mapping
Story Mapping
Specification-By-Example
Establishing a shared understanding
Why?
How?
Code
Acceptance
Criterion
Epic
Capability
Impact, Goal
Easier to define upfront Harder to define upfront
Bug
report
Isolated,
formalized
example
Feature
User Story
Example
Reminder
for a
discussion

6
Collecting Acceptance Criteria
“I would try to put a book into the
shopping cart …”
“I would try to remove a book
from the shopping cart…”
“I’d check whether the shopping cart
is empty, when I enter the shop …”
“I would try to add the same book
again to the shopping cart …”
Books can be placed into
shopping cart.
Books can be removed from
shopping cart.
Shopping cart should be empty
when entering the shop.
Adding same book again to
shopping cart should increase
quantity.
As a potential customer
I want to collect books in a shopping cart
So that I can order several books at once.
“Imagine this story is
already implemented:
how would you verify it?”

7
Example:
Mobile Ticket for Wiener Linien

8
Exercise: Smart Alarm
• What would you try out when the following user story was
done?
• Collect and write down the list of discussed acceptance criteria
• Work in groups of 2-3
As a regular traveler
I want to setup a
smart alarm
so that I can be
warned about service
disruptions
I want to setup
multiple smart
alarms
so that I can plan
ahead for different
travel plans I have

10
UI wire frames,
existing UI
rules, key examples
existing artifacts,
samples
Examples for user stories

11
Specification-by-Example
Examples …
• make abstract descriptions
better understandable
However …
• examples are usually not formally
exchanged or documented
Brian Marick
Examples Tests
Requirements
consist of
describe verify
fulfillment of

12
Discussion of acceptance criteria
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order newOrder = new Order(newCustomer);
newOrder.AddBook(
Catalog.Find(“ISBN-0955683610”)
);
Assert.Equals(33.75,
newOrder.Subtotal);
}
Register as “bart_bookworm”
Go to “/catalog/search”
Enter “ISBN-0955683610”
Click “Search”
Click “Add to Cart”
Click “View Cart”
Verify “Subtotal” is “$33.75”
We would like to encourage new users to
buy in our shop.
Therefore we offer 10% discount for their
first order.
Original idea for the illustration: George Dinwiddie
http://blog.gdinwidiee.com

13
… illustrated with formalized examples
Given the user has not ordered yet
When the user adds a book with the price of EUR 37.5
into the shopping cart
Then the shopping cart sub-total is EUR 33.75.

14
Discover hidden assumptions
Actually, this is not quite right:
Books on sale should be excluded.

15
Collaboration: 3 amigos
“Happy
Path”
Technical
feasability
Exceptions,
border cases

16
Abstract acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books can be added to the shopping basket
Books can be removed from the shopping basket
Shopping basket is initially empty
The same book can be added multiple times to the shopping
basket

17
Examples for acceptance criteria
As a shop visitor
Given my shopping basket is empty
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 1 copy of “Harry Potter”

18
As a shop visitor
Examples for acceptance criteria
Given my shopping basket contains 1 copy of “Harry Potter”
Then my shopping basket should contain 2 copies of “Harry Potter”
The same book can be added multiple times to the shopping basket

19
The same book can be added multiple times to the shopping basket
Structure of examples
Title: Describes intention/abstract acceptance criterion
Arrange: Context, initial state of the system
Act: Execution of the feature
Assert: Assertion of observable behaviour
And I should see the warning: “Book already existed in basket”
Triple-A
constraint
“Checks”
Chaining
up steps

21
Feature: Description of feature or user story
Szenariogrundriss: Beschreibung des Akzeptanzkriteriums
Szenario: Beschreibung des Akzeptanzkriteriums
Gherkin Feature Files
Background: context for all scenarios in the feature file
Scenario: Description of acceptance criterion
Angenommen/Wenn/Dann: Automatisierte Szenario Schritte
Given/When/Then: automated scenario steps
Scenario Outline: Description of acceptance criterion
Given/When/Then: automated scenario steps with <place holder>
Examples: table with examples for <place holders>Examples: table with examples for <place holders>
Given: automated scenario steps
@tagname
@tagname
@tagname
@tagname

22
Feature
Feature: Description of feature or user story
#language:en-en
@done
Feature: Shopping cart
As visitor of the web shop
I want to collect books in a shopping cart
so that I can
- prepare a short list of books I want to order
- combine multiple books in one shipment
@Tagname

23
Scenario
Scenario: Description of acceptance criterion
Given/When/Then: automated scenario steps
@Tagname
@inprogress
Scenario: The same book can be added multiple times to shopping cart
Given my shopping cart contains 1 copy of "Harry Potter"
When I add the book "Harry Potter" to the shopping cart
Then my shopping cart contains 2 copies of "Harry Potter"
And a warning is displayed: "You have added the same book again"

24
Scenario Outline
Scenario Outline: Description of acceptance criterion
Examples: table with examples for <place holders>
Given/When/Then: automated scenario steps with <place holder>
Examples: table with examples for <place holders>
@Tagname
@Tagname
Scenario Outline: Simple search
When I search for books by the phrase '<search phrase>'
Then the list of found books should contain only: <books>
Examples:
| search phrase | books | Explanation |
| Domain | 'Domain …' | whole words are matched |
| Analysis Communication | 'Bridging…', 'Analysis …' | multiple words are matched with OR |
Scenario Outline: Candidate profile can only be saved with <mandatory field>
When I enter a valid candidate profile, without filling in '<mandatory field>'
Then I see the following validation error: 'field <mandatory field> missing'
Examples: Main fields
| mandatory field |
| email |
| name |
| first name |
Examples: Extended fields
| mandatory field |
| birthdate |

27
Exercise
• Work in same previous groups of 2-3
• Pick an acceptance criteria from the
previously compiled list
• Outline usage examples with the following
structure
• Given: Context, initial state
• When: utilization of the feature
• Then: assertions of observable behaviour
• Validate that scenario title (acceptance
criterion) still describes intention of
formalized scenario

29
Purpose of the examples
• Shared understanding:
acceptance criteria
• Documentation:
system details
• Regression-tests:
violated assumptions

30
Continuous validation with automation
System
„Step Definitions“ are binding individual steps
to an automatable interface of the application.
Automatable
interface
UI
Automation
Automation does not necessarily have to bind to the UI.
Automatability of system is supported/evolving with development.

31
Demo
http://www.specflow.org
Gherkin automation for .NET
• Visual Studio plugin (VS-Gallery)
• NuGet Package

32
Some more complex exercises
Choose between
• Best-fare option ticket
• Smart Alarm

33
Best-fare option ticket
• Assumptions
• Ticket price is always adjusted during
validation to the best option (considering
all previous trips)
• Feedback about cost of travel and ticket
option (type) after each validation
• Valid in the core zone 100
• Scope of the story
• Different travel scenarios and how the
price adjusts
• Billings resulting from the user’s travel
• Feedback to traveler (cost of travel, status
of ticket) when validating ticket
• Out of scope for this story
• Credit limit of traveler
• Authentication of ticket (validity)
As an ad-hoc traveler
I want my ticket to adjust
to the best available fare
whenever I validate
So that I don’t need to
study upfront which ticket
I need
Ticket Type Price
Single trip 2,10
24 hours 7,10
48 hours 12,40
72 hours 15,40
8 days 35,80
Week 15,00
Month 45,00
Year 365,00

34
Smart Alarm
• Assumptions
• User can specify route to monitor for
disruptions
• User can specify when disruptions
should trigger the alarm earlier
(minutes before alarm)
• Scope of the story
• Different possibilities of disruptions
provided by GTFS and their impact on
activating the alarm
• Out of scope for this story
• Configuring routes (the user is
selecting an already defined route)
• Setting up alarms (the user has
already setup an alarm)
As a regular traveler
I want to have a smart
alarm that activates
earlier when there are
traffic disruptions on my
route
So that I can travel earlier
to be still on time for my
appointment

35
Exercise
Choose between
• Best-fare option ticket
• Smart Alarm
Collect and discuss scenarios
• What would you try out?
• Collect examples
• Limit scope of the story

36
Exercise
Refine scenarios and formalize
into Gherkin
• Start with a “warm-up” scenario
• Increase complexity with additional
scenarios
• Reconsider structure of scenarios as you
refine them
• Validate scenario title/description after
refinement

38
Test automation becomes expensive
when …
• trying to automate
manual tests
• making tests
unreadable when
automating them
• automating after
completing
development
structure
readability
point in time

39
Structure
Manual tests
Asserts Multiple combined
features
Structure ACT-ASSERT-
ACT-ASSERT-
ACT-ASSERT-
…
Dependent features
Long test path with
high chance to break
Cause and impact of
error hard to trace
Automated Check
Single aspect of a
single feature
ARRANGE –
ACT –
ASSERT
Independent features
Short test path with
lower chance to break
Cause and impact of
error easy to relate

40
Test automation pyramid
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many
few
hard
easy
Automatability

41
// Go to web page 'http://localhost:40001/' using new browser instance
BrowserWindow localhostBrowser = BrowserWindow.Launch(
new System.Uri(this.RecordedMethod1Params.Url));
// Click 'Register found item' link
Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9));
// Click 'Save' button
Mouse.Click(uISpeichernButton, new Point(44, 14));
int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));
// Click 'Register found item' link
Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7));
// Click 'Save' button
Mouse.Click(uISpeichernButton, new Point(34, 11));
int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));
Assert.IsTrue(fundNr1 + 1 == fundNr2);
// Click 'Close' button
Mouse.Click(uICloseButton, new Point(26, 11));
Readability

42
A readable test case
Scenario: New found items should receive a
consecutive number for the current year
Given the previous found item of the
current year had the number 145
When I register a new found item
Then the last found item of the
current year should have the number 146

43
When to test (point in time)
Acceptance criteria
(ATDD, BDD)
Unit Tests
(TDD)
business view
technical view
Exploratory tests
Workflow tests
Performance, Scalability,
Usability,Security, …
definingtheproduct
criticizingtheproduct
New dimension: defining the product
Synergy: Specification of requirements and tests
Agile Testing Quadrants: Brian Marick

44
Manual Testing is always necessary!
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many
few
harder
easier
Automatability
Manual
Check
after
Story
Done
Main
success
pathes
Undiscovered
acceptance
criteria
No/(few)
manual
regression
checks
Few pathes
are enough
More time
for exploration

50
Transparency for stakeholders
In Progress

51
Current sprint report: all sprint scenarios

52
Starting with first scenario (AC)

53
Finishing the first scenario (AC)

54
Progressing scenario after scenario

55

56

57
Implementing user stories in parallel

58
First user story ready for testing

59
Manual testing can start even earlier

60
Work done already can break again

61
Work done already can break again

62
See what is temporarily not working

65
Living documentation
Drill into Details
(Gherkin scenarios)
Overview
(Story Map)

66
User Stories vs. Features
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Living Documentation
Feature 1
AccCrit 1
AccCrit 2
Feature n
AccCrit 4
AccCrit m
User Story n
AccCrit 5
AccCrit m
AccCrit 3
AccCrit 5
„Done“
• Future options of the system
• Organized/refined according to
priority, value, effort, risk, ...
• Next possible increments of
the product (units of work)
• Current state of the system
• Organized/refined for
functional overview
• Versioned and maintained
together with source code

68
Collaboration over specification
Documentation after conversation
• Collaborative discovery
• Should trigger new questions
• Infinite number of examples
• Readability and ability to automate

69
Collaboration smells
• Silos and too formal hand-overs
• Formalization cannot follow implementation
• Scenarios replace communication
• Scenarios inhibit consideration of solutions
• Scenarios inhibit discovery of new things
• Duty after development is done
• Unreadable scenarios without value
• High cost for creation and maintenance
• Technical problems with automation

70
Level of automation
Controller
Business Layer
Data Layer
Model
View
Browser
automation
Trigger behaviour
through controller
Assert behaviour
on model, db, ..
Setup pre-conditions
through service
interfaces
Out-of-process
In-process

71
Test execution performance
• Grouping tests
• Current WIP
• Completed work
• Database
• In-memory
• Templates for setup
• Parallel execution
• Smart execution order

72
Internal vs. external DSL
Example Source:
Liz Keogh
https://github.com/lunivore/tictactoe-java/blob/master/
scenarios/com/lunivore/tictactoe/scenarios/
Three_in_a_row_wins.java

73
Non-functional acceptance criteria
Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms
Given 1000 users are hitting the homepage
simultaneously
Then each user should get a response within 2ms
Matt Wynne
http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing

75
Gherkin automation tools
www.cukes.info
www.behat.org
Ruby, Java, JavaScript, C++
www.specflow.org
.NET, Mono, Silverlight, WP7
PHP

77
Books
Gojko Adzic
Bridiging the
Communication Gap
Gojko Adzic
Specification by
Example
Elisabeth Hendrickson
Explore IT!

78
Christian Hassa: ch@techtalk.at - @chrishassa

Tutorial: Implementing Specification-By-Example with Gherkin

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

More from Christian Hassa

More from Christian Hassa (20)

Recently uploaded

Recently uploaded (20)

Tutorial: Implementing Specification-By-Example with Gherkin