SlideShare a Scribd company logo
1 of 68
Acceptance Testing

In the land of the startup



                         Joseph Wilk
What kind of land is this?
What kind of land is this?
Do startups need
acceptance tests?

    Build           Measure



            Learn
Context
      is king
Toolset




Rails, MongoDB, Mysql,       Cucumber, Rspec
RabbitMQ, Memcache
Cucumber
•   Behaviour Driven    Feature: Filling Cucumbers with
    Development         rocket fuel
    framework
                        Scenario: Rocket fuel
•   Plaintext             Given I have trained monkeys
                          When I click the launch button
•   Promotes              Then the monkeys should not die
    communication

                       Given /^I have trained monkeys$/ do
                         @test_pilots << Monkeys.new
                       end
Why Acceptance?

• Developer centric tool
• Behaviour Driven
  Development

• Conversations
Where acceptance
               tests are born


      Product                Developer
      Owner




   User
interaction                       QA
Where acceptance
               tests are born

                   Just In Time
      Product
                      Talking     Developer
      Owner




   User
interaction                            QA
Non-technical
people writing
  Gherkin?
Feature: Ajax pagination for upcoming events
  In order to reduce friction finding concerts I want to go to
  As a logged in user
  I want an Ajax button which loads more concerts

 Scenario: Events 60 days in the futures
   Given ...
   When ...
   Then ...

 Scenario: No events outside of the next 60 days
   Given ...
   When ...
   Then ...

 Scenario: No events
   Given ...
   When ...
   Then ...
Are we done yet?

“A story isn't done until it is being
used by real users
in production and has been validated to be a
useful part of a product.”
                                Joshua Kerievsky
Learnings
      Good and bad
Asynchronous Testing is
        messy
Scenario: Upload image
  ...
  Given all the messages have been processed
  ...
Feature Overload

254    Feature files
1257   Scenarios/Examples
10807 Steps
Slicing Features
@media, @public
Feature: Visitor views artist’s media

@feed, @admin
Feature: Logged in user views views their activity feed


 Tagging                    $ cucumber --tags @media

 Filenames                  $ cucumber visitor_*

 Subsystems                 $ cucumber features/admin/*

 Profiles                    $ cucumber --profile admin
Acceptance tests
          vs
      Unit tests
The Test Build

254 Features, 1257 scenarios, 10807 steps
The Test Build

254 Features, 1257 scenarios, 10807 steps

   1 build server ~ 4 hours
The Test Build

  254 Features, 1257 scenarios, 10807 steps

      1 build server ~ 4 hours
19 EC2 build servers ~ 13:40 minutes
The Test Build

  254 Features, 1257 scenarios, 10807 steps

      1 build server ~ 4 hours
19 EC2 build servers ~ 13:40 minutes

            $2068.99/$3000
Divide and Concuquer
Divide and Concuquer
Divide and Concuquer
Divide and Concuquer
Divide and Concuquer
Divide and Concuquer

Scenario: Related artists
 Given an artist “SYGC”
 And “M.Bison” is related to “SYGC”
 When I visit the artist
 Then I should see within Similar
      Artists a link to “M.Bison”
Divide and conquer


   Rails


Scenario: Related artists
 Given an artist “SYGC”
 And “M.Bison” is related to “SYGC”
 When I visit the artist
 Then I should see within Similar
      Artists a link to “M.Bison”
Divide and conquer

                                      Related
   Rails                              Artists
                                      Service

Scenario: Related artists
 Given an artist “SYGC”
 And “M.Bison” is related to “SYGC”
 When I visit the artist
 Then I should see within Similar
      Artists a link to “M.Bison”
Divide and conquer

                 related(artist)
                                      Related
   Rails                              Artists
                                      Service

Scenario: Related artists
 Given an artist “SYGC”
 And “M.Bison” is related to “SYGC”
 When I visit the artist
 Then I should see within Similar
      Artists a link to “M.Bison”
Divide and conquer

                 related(artist)
                                      Related
   Rails             HTML             Artists
                                      Service

Scenario: Related artists
 Given an artist “SYGC”
 And “M.Bison” is related to “SYGC”
 When I visit the artist
 Then I should see within Similar
      Artists a link to “M.Bison”
Divide and conquer

          related(artist)
                                        Related
Rails         HTML                      Artists
                                        Service

                            Scenario: Related artists
                             Given an artist “SYGC”
                             And “M.Bison” is related to “SYGC”
                             When I visit the artist
                             Then I should see within Similar
                                  Artists a link to “M.Bison”
Divide and conquer

            related(artist)
                                                   Related
Rails               HTML                           Artists
                                                   Service
        unit test          unit test
                                       Scenario: Related artists
                                        Given an artist “SYGC”
                                        And “M.Bison” is related to “SYGC”
                                        When I visit the artist
                                        Then I should see within Similar
                                             Artists a link to “M.Bison”
Tests that
never fail
Tests that
never fail
Tests which      Tests that
regularly fail   never fail
Tests which      Tests that
regularly fail   never fail




Daily Build
Tests which       Tests that
regularly fail    never fail




Daily Build      Nightly Build
Tests which                   Tests that
      regularly fail                never fail




       Daily Build                Nightly Build

$cucumber --tags ~@nightly   $cucumber --tags @nightly
Flickering Confidence
Flickering Confidence
               FAIL
Flickering Confidence
               FAIL

               FAIL
Flickering Confidence
               FAIL

               FAIL

               PASS
Flickering Confidence
                     FAIL

                     FAIL

                     PASS




  Main   @flicker   Flicker
Pairwise
Scenario: Testing Cucumber against different environments
  Given I have a rails app <Rails version>
  And I’m using Ruby <Ruby version>
  And I am using the <Test Framework>
  And I am using the driver <Driver>
  Then Cucumber should install and play nicely
  Examples:
  | Rails version | Ruby version | test framework | Driver     |
  |           3.0 |        1.8.7 | Rspec          | Webrat     |
  |           3.0 |        1.8.7 | Testunit       | Capybara   |
  |           3.0 |        1.8.7 | Rspec          | Capybara   |
  |           3.0 |        1.8.7 | Testunit       | Webrat     |
  |           3.0 |          1.9 | Rspec          | Webrat     |
  |           3.0 |          1.9 | Testunit       | Webrat     |
  |         2.3.1 |          1.9 | Rspec          | Webrat     |
                             ....
| Rails version | Ruby version | test framework   |   Driver     |
|         2.3.1 |        1.8.7 | Rspec            |   Webrat     |
|         2.3.1 |        1.8.7 | Rspec            |   Capybara   |
|         2.3.1 |        1.8.7 | Testunit         |   Webrat     |
|         2.3.1 |        1.8.7 | Testunit         |   Capybara   |
|         2.3.1 |        1.9   | Rspec            |   Capybara   |
|         2.3.1 |        1.9   | Rspec            |   Webrat     |
|         2.3.1 |        1.9   | Testunit         |   Capybara   |
|         2.3.1 |        1.9   | Testunit         |   Webrat |

                           ....




“most faults are caused by interactions of at most two
factors”
Run Just Enough Tests
                       Autotest
 #   *   Test files must be stored in test/
 #   *   Test files names must start with test_
 #   *   Test class names must start with Test
 #   *   Implementation files must be stored in lib/
 #   *   Implementation files must match up with a
 #        test file named
 #        test_.*implementation.rb


                       Cucover

 @analyzer = Rcov::CodeCoverageAnalyzer.new
The Art of

         War Plaintext
                   ,                                               ,
                  @"===,                                  ,_____cctI
                  "?AAAAAAAAAAAAAAAA,,,,,,,,,,,,,,,,,,,,,;LLLLLLLLLL
       ~",,,       1""""""""""""###OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
        '"EEEEE, !'"***"~~~~~~"OOOIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
           ,EEEEE)>"'''???????"WWW!MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
             "E.,)+="WWW~~~~~~#"OOO1OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
         ,~:#")LLL!"+++???????"$$$1==========##/
          &LLLLLLLLL;;;;;;;;;;;;,,,/
          1#LLLLLLLLLLLLLLLLLLLLLL!
        ,!###LLLLLL"'EEEE,'"LLLLL!
        !######LLL" "EEE" "LLLL"
       !#########L!    "EEJ. "LL!
      !##########1       "JJ*,l"
    !############"!         ,l"
    1##########" 1"~~,~~"
   !##########"
  !###########!
  !###########1
 !############!
 1#############
!"#############"
!##############!
1##########"'                               A magnum, By calendron
1#####"'
"""""
Don’t force
              Feature: Title
                In order to <value>
                As a <role>

 structure      I want <feature>
Avoid Noise!
               Given I am able to login
               Given I am able to login
Avoid       Given I login
                Given I authenticate

Inconsistency   Given I gain access
Given   I go to the login page


 Balance
              And I   fill in "username" with "cuke"
              And I   fill in "password" with "cuker"
              And I   click "login"

Abstraction   Given I'm logged in
Use Language
               Building blocks
Given /I’m logged in/ do
  User.create!(:user => 'josephwilk', :password => "pass")
  Given 'I fill in "password" with "josephwilk"'
  Given 'I fill in "password" with "pass"'
  Given 'I click "login"'
end
Continuous Integration(WIP)
    Fail
  Work in    Pass
             Done
  Pending
  progress

    add       list    $ cucumber --wip
   movies    movies
                                 --tags @WIP
                                 features/
    edit
   movies


   delete
   movies
Limiting Tags in Flow
   Work in
                    Done
   progress

 add add add add
movies movies
   movies movies           $ cucumber --tags @WIP:3
                                      features/
 add add add edit
movies movies
   movies movies


 add add add
           delete
movies movies
   movies movies
Further reading

•   http://cukes.info

•   http://wiki.github.com/
    aslakhellesoy/cucumber

•   http://www.cuke4ninja.com/

•   http://blog.josephwilk.net
Thanks!


               Joseph Wilk
              @josephwilk
http://blog.josephwilk.net

More Related Content

Similar to Acceptance startup fini

MonoRails - GoGaRuCo 2012
MonoRails - GoGaRuCo 2012MonoRails - GoGaRuCo 2012
MonoRails - GoGaRuCo 2012
jackdanger
 
Web Development using Ruby on Rails
Web Development using Ruby on RailsWeb Development using Ruby on Rails
Web Development using Ruby on Rails
Avi Kedar
 
PTW Rails Bootcamp
PTW Rails BootcampPTW Rails Bootcamp
PTW Rails Bootcamp
Mat Schaffer
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launched
Mat Schaffer
 

Similar to Acceptance startup fini (20)

Rocket Fuelled Cucumbers
Rocket Fuelled CucumbersRocket Fuelled Cucumbers
Rocket Fuelled Cucumbers
 
Calabash automated test
Calabash automated testCalabash automated test
Calabash automated test
 
MonoRails - GoGaRuCo 2012
MonoRails - GoGaRuCo 2012MonoRails - GoGaRuCo 2012
MonoRails - GoGaRuCo 2012
 
Behavior driven development with calabash for android
Behavior driven development with calabash for androidBehavior driven development with calabash for android
Behavior driven development with calabash for android
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
 
Rapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoop
 
Acceptance Test-driven Development with Cucumber-jvm
Acceptance Test-driven Development with Cucumber-jvmAcceptance Test-driven Development with Cucumber-jvm
Acceptance Test-driven Development with Cucumber-jvm
 
TorqueBox
TorqueBoxTorqueBox
TorqueBox
 
Web Development using Ruby on Rails
Web Development using Ruby on RailsWeb Development using Ruby on Rails
Web Development using Ruby on Rails
 
Isomorphic App Development with Ruby and Volt - Rubyconf2014
Isomorphic App Development with Ruby and Volt - Rubyconf2014Isomorphic App Development with Ruby and Volt - Rubyconf2014
Isomorphic App Development with Ruby and Volt - Rubyconf2014
 
Cucumber
CucumberCucumber
Cucumber
 
PTW Rails Bootcamp
PTW Rails BootcampPTW Rails Bootcamp
PTW Rails Bootcamp
 
Wwcode2
Wwcode2Wwcode2
Wwcode2
 
David Bilík: Anko – modern way to build your layouts?
David Bilík: Anko – modern way to build your layouts?David Bilík: Anko – modern way to build your layouts?
David Bilík: Anko – modern way to build your layouts?
 
RxSwift to Combine
RxSwift to CombineRxSwift to Combine
RxSwift to Combine
 
Arquillian: Effective tests from the client to the server
Arquillian: Effective tests from the client to the serverArquillian: Effective tests from the client to the server
Arquillian: Effective tests from the client to the server
 
Realtime collaboration with Clojure - EuroClojure - Barcelona, 2015
Realtime collaboration with Clojure - EuroClojure - Barcelona, 2015Realtime collaboration with Clojure - EuroClojure - Barcelona, 2015
Realtime collaboration with Clojure - EuroClojure - Barcelona, 2015
 
Cucumber
CucumberCucumber
Cucumber
 
Cross-browser testing in the real world
Cross-browser testing in the real worldCross-browser testing in the real world
Cross-browser testing in the real world
 
wwc start-launched
wwc start-launchedwwc start-launched
wwc start-launched
 

More from Skills Matter

Oscar reiken jr on our success at manheim
Oscar reiken jr on our success at manheimOscar reiken jr on our success at manheim
Oscar reiken jr on our success at manheim
Skills Matter
 
Russ miles-cloudfoundry-deep-dive
Russ miles-cloudfoundry-deep-diveRuss miles-cloudfoundry-deep-dive
Russ miles-cloudfoundry-deep-dive
Skills Matter
 
I went to_a_communications_workshop_and_they_t
I went to_a_communications_workshop_and_they_tI went to_a_communications_workshop_and_they_t
I went to_a_communications_workshop_and_they_t
Skills Matter
 

More from Skills Matter (20)

5 things cucumber is bad at by Richard Lawrence
5 things cucumber is bad at by Richard Lawrence5 things cucumber is bad at by Richard Lawrence
5 things cucumber is bad at by Richard Lawrence
 
Patterns for slick database applications
Patterns for slick database applicationsPatterns for slick database applications
Patterns for slick database applications
 
Scala e xchange 2013 haoyi li on metascala a tiny diy jvm
Scala e xchange 2013 haoyi li on metascala a tiny diy jvmScala e xchange 2013 haoyi li on metascala a tiny diy jvm
Scala e xchange 2013 haoyi li on metascala a tiny diy jvm
 
Oscar reiken jr on our success at manheim
Oscar reiken jr on our success at manheimOscar reiken jr on our success at manheim
Oscar reiken jr on our success at manheim
 
Progressive f# tutorials nyc dmitry mozorov & jack pappas on code quotations ...
Progressive f# tutorials nyc dmitry mozorov & jack pappas on code quotations ...Progressive f# tutorials nyc dmitry mozorov & jack pappas on code quotations ...
Progressive f# tutorials nyc dmitry mozorov & jack pappas on code quotations ...
 
Cukeup nyc ian dees on elixir, erlang, and cucumberl
Cukeup nyc ian dees on elixir, erlang, and cucumberlCukeup nyc ian dees on elixir, erlang, and cucumberl
Cukeup nyc ian dees on elixir, erlang, and cucumberl
 
Cukeup nyc peter bell on getting started with cucumber.js
Cukeup nyc peter bell on getting started with cucumber.jsCukeup nyc peter bell on getting started with cucumber.js
Cukeup nyc peter bell on getting started with cucumber.js
 
Agile testing & bdd e xchange nyc 2013 jeffrey davidson & lav pathak & sam ho...
Agile testing & bdd e xchange nyc 2013 jeffrey davidson & lav pathak & sam ho...Agile testing & bdd e xchange nyc 2013 jeffrey davidson & lav pathak & sam ho...
Agile testing & bdd e xchange nyc 2013 jeffrey davidson & lav pathak & sam ho...
 
Progressive f# tutorials nyc rachel reese & phil trelford on try f# from zero...
Progressive f# tutorials nyc rachel reese & phil trelford on try f# from zero...Progressive f# tutorials nyc rachel reese & phil trelford on try f# from zero...
Progressive f# tutorials nyc rachel reese & phil trelford on try f# from zero...
 
Progressive f# tutorials nyc don syme on keynote f# in the open source world
Progressive f# tutorials nyc don syme on keynote f# in the open source worldProgressive f# tutorials nyc don syme on keynote f# in the open source world
Progressive f# tutorials nyc don syme on keynote f# in the open source world
 
Agile testing & bdd e xchange nyc 2013 gojko adzic on bond villain guide to s...
Agile testing & bdd e xchange nyc 2013 gojko adzic on bond villain guide to s...Agile testing & bdd e xchange nyc 2013 gojko adzic on bond villain guide to s...
Agile testing & bdd e xchange nyc 2013 gojko adzic on bond villain guide to s...
 
Dmitry mozorov on code quotations code as-data for f#
Dmitry mozorov on code quotations code as-data for f#Dmitry mozorov on code quotations code as-data for f#
Dmitry mozorov on code quotations code as-data for f#
 
A poet's guide_to_acceptance_testing
A poet's guide_to_acceptance_testingA poet's guide_to_acceptance_testing
A poet's guide_to_acceptance_testing
 
Russ miles-cloudfoundry-deep-dive
Russ miles-cloudfoundry-deep-diveRuss miles-cloudfoundry-deep-dive
Russ miles-cloudfoundry-deep-dive
 
Serendipity-neo4j
Serendipity-neo4jSerendipity-neo4j
Serendipity-neo4j
 
Simon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelismSimon Peyton Jones: Managing parallelism
Simon Peyton Jones: Managing parallelism
 
Plug 20110217
Plug   20110217Plug   20110217
Plug 20110217
 
Lug presentation
Lug presentationLug presentation
Lug presentation
 
I went to_a_communications_workshop_and_they_t
I went to_a_communications_workshop_and_they_tI went to_a_communications_workshop_and_they_t
I went to_a_communications_workshop_and_they_t
 
Plug saiku
Plug   saikuPlug   saiku
Plug saiku
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Recently uploaded (20)

ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Acceptance startup fini

  • 1. Acceptance Testing In the land of the startup Joseph Wilk
  • 2. What kind of land is this?
  • 3. What kind of land is this?
  • 4. Do startups need acceptance tests? Build Measure Learn
  • 5. Context is king
  • 6.
  • 7.
  • 8. Toolset Rails, MongoDB, Mysql, Cucumber, Rspec RabbitMQ, Memcache
  • 9. Cucumber • Behaviour Driven Feature: Filling Cucumbers with Development rocket fuel framework Scenario: Rocket fuel • Plaintext Given I have trained monkeys When I click the launch button • Promotes Then the monkeys should not die communication Given /^I have trained monkeys$/ do @test_pilots << Monkeys.new end
  • 10. Why Acceptance? • Developer centric tool • Behaviour Driven Development • Conversations
  • 11. Where acceptance tests are born Product Developer Owner User interaction QA
  • 12. Where acceptance tests are born Just In Time Product Talking Developer Owner User interaction QA
  • 13.
  • 14.
  • 16.
  • 17. Feature: Ajax pagination for upcoming events In order to reduce friction finding concerts I want to go to As a logged in user I want an Ajax button which loads more concerts Scenario: Events 60 days in the futures Given ... When ... Then ... Scenario: No events outside of the next 60 days Given ... When ... Then ... Scenario: No events Given ... When ... Then ...
  • 18. Are we done yet? “A story isn't done until it is being used by real users in production and has been validated to be a useful part of a product.” Joshua Kerievsky
  • 19. Learnings Good and bad
  • 20. Asynchronous Testing is messy Scenario: Upload image ... Given all the messages have been processed ...
  • 21. Feature Overload 254 Feature files 1257 Scenarios/Examples 10807 Steps
  • 22. Slicing Features @media, @public Feature: Visitor views artist’s media @feed, @admin Feature: Logged in user views views their activity feed Tagging $ cucumber --tags @media Filenames $ cucumber visitor_* Subsystems $ cucumber features/admin/* Profiles $ cucumber --profile admin
  • 23. Acceptance tests vs Unit tests
  • 24. The Test Build 254 Features, 1257 scenarios, 10807 steps
  • 25. The Test Build 254 Features, 1257 scenarios, 10807 steps 1 build server ~ 4 hours
  • 26. The Test Build 254 Features, 1257 scenarios, 10807 steps 1 build server ~ 4 hours 19 EC2 build servers ~ 13:40 minutes
  • 27. The Test Build 254 Features, 1257 scenarios, 10807 steps 1 build server ~ 4 hours 19 EC2 build servers ~ 13:40 minutes $2068.99/$3000
  • 33. Divide and Concuquer Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 34. Divide and conquer Rails Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 35. Divide and conquer Related Rails Artists Service Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 36. Divide and conquer related(artist) Related Rails Artists Service Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 37. Divide and conquer related(artist) Related Rails HTML Artists Service Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 38. Divide and conquer related(artist) Related Rails HTML Artists Service Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 39. Divide and conquer related(artist) Related Rails HTML Artists Service unit test unit test Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
  • 40.
  • 41.
  • 42.
  • 43.
  • 46. Tests which Tests that regularly fail never fail
  • 47. Tests which Tests that regularly fail never fail Daily Build
  • 48. Tests which Tests that regularly fail never fail Daily Build Nightly Build
  • 49. Tests which Tests that regularly fail never fail Daily Build Nightly Build $cucumber --tags ~@nightly $cucumber --tags @nightly
  • 53. Flickering Confidence FAIL FAIL PASS
  • 54. Flickering Confidence FAIL FAIL PASS Main @flicker Flicker
  • 55. Pairwise Scenario: Testing Cucumber against different environments Given I have a rails app <Rails version> And I’m using Ruby <Ruby version> And I am using the <Test Framework> And I am using the driver <Driver> Then Cucumber should install and play nicely Examples: | Rails version | Ruby version | test framework | Driver | | 3.0 | 1.8.7 | Rspec | Webrat | | 3.0 | 1.8.7 | Testunit | Capybara | | 3.0 | 1.8.7 | Rspec | Capybara | | 3.0 | 1.8.7 | Testunit | Webrat | | 3.0 | 1.9 | Rspec | Webrat | | 3.0 | 1.9 | Testunit | Webrat | | 2.3.1 | 1.9 | Rspec | Webrat | ....
  • 56. | Rails version | Ruby version | test framework | Driver | | 2.3.1 | 1.8.7 | Rspec | Webrat | | 2.3.1 | 1.8.7 | Rspec | Capybara | | 2.3.1 | 1.8.7 | Testunit | Webrat | | 2.3.1 | 1.8.7 | Testunit | Capybara | | 2.3.1 | 1.9 | Rspec | Capybara | | 2.3.1 | 1.9 | Rspec | Webrat | | 2.3.1 | 1.9 | Testunit | Capybara | | 2.3.1 | 1.9 | Testunit | Webrat | .... “most faults are caused by interactions of at most two factors”
  • 57. Run Just Enough Tests Autotest # * Test files must be stored in test/ # * Test files names must start with test_ # * Test class names must start with Test # * Implementation files must be stored in lib/ # * Implementation files must match up with a # test file named # test_.*implementation.rb Cucover @analyzer = Rcov::CodeCoverageAnalyzer.new
  • 58.
  • 59. The Art of War Plaintext , , @"===, ,_____cctI "?AAAAAAAAAAAAAAAA,,,,,,,,,,,,,,,,,,,,,;LLLLLLLLLL ~",,, 1""""""""""""###OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO '"EEEEE, !'"***"~~~~~~"OOOIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII ,EEEEE)>"'''???????"WWW!MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM "E.,)+="WWW~~~~~~#"OOO1OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO ,~:#")LLL!"+++???????"$$$1==========##/ &LLLLLLLLL;;;;;;;;;;;;,,,/ 1#LLLLLLLLLLLLLLLLLLLLLL! ,!###LLLLLL"'EEEE,'"LLLLL! !######LLL" "EEE" "LLLL" !#########L! "EEJ. "LL! !##########1 "JJ*,l" !############"! ,l" 1##########" 1"~~,~~" !##########" !###########! !###########1 !############! 1############# !"#############" !##############! 1##########"' A magnum, By calendron 1#####"' """""
  • 60. Don’t force Feature: Title In order to <value> As a <role> structure I want <feature>
  • 61. Avoid Noise! Given I am able to login Given I am able to login
  • 62. Avoid Given I login Given I authenticate Inconsistency Given I gain access
  • 63. Given I go to the login page Balance And I fill in "username" with "cuke" And I fill in "password" with "cuker" And I click "login" Abstraction Given I'm logged in
  • 64. Use Language Building blocks Given /I’m logged in/ do User.create!(:user => 'josephwilk', :password => "pass") Given 'I fill in "password" with "josephwilk"' Given 'I fill in "password" with "pass"' Given 'I click "login"' end
  • 65. Continuous Integration(WIP) Fail Work in Pass Done Pending progress add list $ cucumber --wip movies movies --tags @WIP features/ edit movies delete movies
  • 66. Limiting Tags in Flow Work in Done progress add add add add movies movies movies movies $ cucumber --tags @WIP:3 features/ add add add edit movies movies movies movies add add add delete movies movies movies movies
  • 67. Further reading • http://cukes.info • http://wiki.github.com/ aslakhellesoy/cucumber • http://www.cuke4ninja.com/ • http://blog.josephwilk.net
  • 68. Thanks! Joseph Wilk @josephwilk http://blog.josephwilk.net