Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Introduction to Software Testing

Slides for the according talk at WordCamp Berlin, 2015

  • Login to see the comments

An Introduction to Software Testing

  1. An Introduction to Software Testing A talk by @thorstenfrommen
  2. Foreword Testing is an incredibly complex subject. You won't become fu| |—f| edged testers by sitting here. This is no tutorial for writing tests for your individual WordPress plugin/ theme. Instead, you will be briefly introduced to several relevant topics of software testing. If you have more questions after this talk than you have now, I reached my goal. Sorry. (V)
  3. Why Test?
  4. Reasons for Testing 0 Software contains defects. They hide. o Defects can cause software failures. 0 Failures can cost money, and even be mortal! 0 Testing assures the quality of the software. 0 Testing accelerates software development. 0 And much, much more.
  5. ”But we debug an ywa y! ?,, Debugging is NOT testing!
  6. Infamous Software Failures
  7. Ariane—5 0 Inertial navigation software taken from Ariane—4. Untested. o All other systems thoroughly tested component—by—component. o Ariane—5 had a different trajectory than Ariane 4. 0 Converting 64-bit f| oating—point data into 16-bit unsigned integer values. —> Arithmetic overflow. 0 There was an exception handler for that. It had been disabled. 0 Not even 40 seconds after launch, Ariane—5 literally se| f—destructed. Successfully.
  8. /‘J . g fz, ,, -‘ t ‘/1 g Inverted Flight r A developer of the US Air Force improved a program for an unmanned rocket: , : r. when crossing the equator, flip the coordinates's leading sign. ,1-’ r The program therefore needed less memory. c The rocket also made a 180—degrees roll. So what? r The program later was used for the autopilot of an F-18 fighterjet. I c When crossing the equator, the pilot certainly was somewhat suprised.
  9. Heartbleed «r Heartbeat Extension of TLS is used to keep Datagram TLS sessions open. ~: Simple request and response scheme: r "Send me back the following (padded) string which is n bytes long. ,, An attackerjust had to request a long string, while telling it is short. -: Other party responded with short string, then leaked potentially confidential data. if it
  10. Do Software Testing. Right. «r Testing has to be planned. -' Testing costs easily up to 40 % of a project's budget. c Testing has to be performed in a reasonable way. 6 Testing shall be independent and objective. Testing has to be managed. Software testing is a fundamental part of professional software development!
  11. What is Testing?
  12. Testing The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. —| STOB®
  13. is a Test This Fulfil me .3 33
  14. Different Perspectives, Different Objectives. Perspective Objective Developer Find defects. QA Evaluate quality. Tester Check functionality against requirements. Customer Verify usability and applicability.
  15. Requirements? Slpecification? Requirements Specification?
  16. Requirements and Specification The requirements document is the input to a development phase. The specification document is the ouput of a development phase. The specification document resulting from development phase X serves as requirements document for development phase X+1. Requirements and specification are the same, depending on the usage and point of view. The product is 100 % specification.
  17. Vallidlation and Verification
  18. Validation The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. -IEEE Std 610.12—1990 ’$4re we building the right product? ,,
  19. Verification The process of evaluating a system or component to determine whether the system ofa given development phase satisfies the conditions imposed at the start of that phase. -IEEE Std 610.12—1990 ’$4re we building the product righ t? ,,
  20. Static Testing
  21. Static Testing Verification only. v‘ No execution of code. -: Find defects. r. Static analysis (automated). Reviews (manual examination, tool support).
  22. Static Analysis 1* Analyze code (e. g., control flow, data flow), and generated output. -: Typical use cases for static analysis: r syntax checking; n code smell detection; c coding standards compliance. ~? Typical defects found by static analysis: r syntax violations; C unreachable code; r overly complicated constructs.
  23. Reviews r Different types of reviews: r informal review (no formal process; inexpensive); r walkthrough (train colleagues and users; gain understanding); n technical review (documented, defined process; discuss); r. inspection (formal process; gain metrics). -r Success factors: r clear predefined objectives; r defects found are welcomed and expressed objectively; r application of suitable review techniques; r management supports review process; r emphasis on learning and process improvement.
  24. Dynamic Testing
  25. l‘. Dynamic Testing «: Primarily verification. -t Execution of code. Find failures. Different testing methods: b| ack—box testing; r white-box testing. I‘. I‘. I‘. I‘. ? Different test levels: unit (component, module) testing; integration testing; system testing; acceptance testing.
  26. Dynamic Testing Methods
  27. Black-box Testing «' Specification—based testing. 6 Test functionality by observing external behavior. I: No knowledge of internals (required). -: Different black-box testing techniques: r equivalence partitioning; r boundary value analysis; c decision table testing; F7
  28. White-box Testing ~ Structure—based testing. it Close examination of procedural level of detail. . Knowledge of internals required. ; Different white-box testing techniques: r statement testing; r decision testing; r. (multiple) condition testing; [7
  29. Test Levels
  30. Unit Testing I: Test individual units in isolation. <2 Find defects in software components (e. g., functions, classes). «r Done by developers. c In general, white-box tests.
  31. Integration Testing Test communication/ interaction of units (i. e., their interfaces). r: Maybe separate unit integration and system integration tests. Done by developers. Primarily white-box tests, but also b| ack—box tests.
  32. System Testing Test complete, integrated system. 7 Evaluate system compliance with specified requirements. Stress, performance, usability etc. testing. . Done by (external) testers. .. In general, black-box tests. Additional white-box tests possible.
  33. Acceptance Testing Test complete, integrated system. Evaluate system compliance with specified acceptance criteria. v: May be performed at various times during development. c Done by customers/ users. Only black-box tests.
  34. It's just a model that provides an easy-to—grasp overview.
  35. Function Level -. ;a Architecture i l Software l Implementation
  36. Requirements and Specification Customer Level Requirements Function Specification Level Software Architecture Implementation
  37. Requirements and Specification Customer Level Function Requirements Level Software _ Specification Architecture Implementation
  38. Requirements and Specification Customer Level Function Level Software _ Requirements Architecture Specification Implementation
  39. Requirements and Specification Customer Level Function Level Software Architecture Requirements Implementation Specification
  40. 100 % Specification Customer Level Function Level Software Architecture Implementation 100 % Specification
  41. Static Testing Customer Level Function Level Software Architecture Static Testing Implementation
  42. Customer Level Dynamic Testing Acceptance Testing Function Level Integration Testing Software Architecture Unit Testing Dynamic Testing ( . ... ... ... ... ... ... ... ... .. . . Implementation
  43. Software Testing in the Context. of word Press
  44. Static Testing -:2 PHP_CodeSniffer: c detects violations of coding standards in PHP, JavaScript and CSS files. 6 PHP Mess Detector: F detects potential problems (e. g., possible bugs, unused stuff) in PHP code. <2 JSHint: r detects errors and potential problems in JavaScript code. JSCS: c code style linter for programmatically enforcing your JavaScript style guide. ~' CSSLint: c detects syntax violations and problematic constructs in CSS files. Pretty much all that comes with a decent IDE (e. g., Phpstorm).
  45. Dynamic Testing PHPUHIIZ r unit testing framework for PHP. «r phpspec: r spec-level BDD framework for PHP. ~: ' Behat: r story—| eve| BDD framework for PHP. if QUnit: c unit testing framework for JavaScript. . ;—. Jasmine: r BDD testing framework for JavaScript.
  46. Dynamic Testing c Mockery: r PHP mock object framework for use in unit testing. 6 Prophecy: r object mocking framework for PHP. r Phake: r PHP mocking framework. C WP_Mock: r API mocking framework for unit testing within WordPress. <‘ Brain Monkey: r mocking utility for PHP functions and WordPress plugin API.
  47. Continuous Integration
  48. Continuous Integration I Automated activities: r execute static code analysis; c execute unit tests, and check code coverage; F deploy to test environment; I“. execute integration tests; c Benefits: r earlier detection and analysis of conflicting changes; regular feedback on whether the code is working; no big-bang integration; reduces repetitive manual testing activities; F1 F1 F1 71
  49. Travis CI r Hosted continuous integration server. Single requirement: GitHub account. vi Getting started: r sign in with your GitHub account; r accept GitHub access permissions confirmation; r set up . travis . yml file for your repositories; c enable Travis CI builds for individual repositories; c develop continuously integrated software. .. / ** @see travis-ci. org */
  50. ISTQB® Certified Tester Most widespread qualification scheme in the world. Syllabi contents and glossary de facto industry reference. Test Management Improving the Testing Process 1-es‘ 5""‘<‘9'< W"-'9°'"“"‘ Automation Security Testing Opcmiiomil Tusl lilipleiliciilinq 1031 Engineering , Fim, M,.0. Z0. 5, Mnmiqonwnt Pmcmc. inupmwnwni ll/ aiiagirig the rim Team Assessing Test F’ioLenes Test Manager Test Analyst Tecxggigigest Agile Model Tester Based Testing II‘. II M 0 Lu k) Z < > D <( ISTQB Glossary Foundation FOUNDAT ON / ** @see www. istqb. org */
  51. Lessoris Learned 0 Software testing is important! So make it a topic. I Use (the right) tools, such as a decent IDE. 0 Do code reviews. 0 Write unit tests. continue with integration tests. I Automate!
  52. Thanks! @thorstenfrommen slides. tfrommen. de/ testing/