Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Automatic
Test Case Generation
for Train Control Systems
Adnan Causevic
TOCSYC Industrial Workshop, 2014-11-13, Västerås
2
Today’s talk is about…
●  Software Testing Laboratory @ MDH
●  Research:
●  Test Automation
●  Quality of Tests
●  Developers Testing
●  AGENTS -...
●  ATAC research project
●  2011-2014
●  Funded by ITEA2/Vinnova
●  Advanced Test Automation for Complex Software-
Intensi...
●  Train Control Management System (TCMS)
●  a high capacity, infrastructure backbone
●  the center of the distributed sys...
●  Vehicle
●  Vehicle Integration testing
●  System
●  System Integration testing
●  Software
●  Modules integration testi...
●  IEC 61131-3
●  Function Block Diagram
●  Graphical programming language
●  Based on element composition
●  Various pred...
●  Coverage
●  Strict measurements mandated by safety standards
●  Tooling
●  No tool support to measure coverage on FBD
●...
9
CompleteTest
10
Research approach
FBD  Program
Transformation 1 Annotation2
UPPAAL  timed  
automata
...
Reachability  
properties
∃	
 ...
●  Time required to generate tests
●  satisfying the DC, CC and MC/DC logic coverage criteria
11
Case Study at BT
●  For 34 of the 157 programs, the tool did not terminate
after running for a substantial period of time.
●  Cut-off time ...
●  Should a developer generate the tests from scratch?
●  Should a developer provide manual tests first?
●  How many (or h...
●  Internal behavior of each function block has to be
modeled
●  Currently, blocks from the standard language are modeled
...
●  Complementing model-checking with:
●  Static analysis of FBD’s
●  Search-based software testing
●  Trying some new appr...
●  Adnan Causevic
adnan.causevic@mdh.se
●  Eduard Enoiu
eduard.paul.enoiu@mdh.se
●  www.completetest.org
16
Questions?
Upcoming SlideShare
Loading in …5
×

Automatic Test Case Generation

1,349 views

Published on

Automatic Test Case Generation for Function Block Diagrams

Published in: Software
  • Login to see the comments

Automatic Test Case Generation

  1. 1. Automatic Test Case Generation for Train Control Systems Adnan Causevic TOCSYC Industrial Workshop, 2014-11-13, Västerås
  2. 2. 2 Today’s talk is about…
  3. 3. ●  Software Testing Laboratory @ MDH ●  Research: ●  Test Automation ●  Quality of Tests ●  Developers Testing ●  AGENTS - KKS funded project ●  Bombardier Transportation ●  Maximatecc ●  Involved in the TOCSYC research environment ●  Intelligent Embedded Systems Master Program Coordinator 3 About me
  4. 4. ●  ATAC research project ●  2011-2014 ●  Funded by ITEA2/Vinnova ●  Advanced Test Automation for Complex Software- Intensive System 4 Cooperation with Bombardier “The project has brought test automation on the agenda of European industry.” - ITEA3 Final Review Report
  5. 5. ●  Train Control Management System (TCMS) ●  a high capacity, infrastructure backbone ●  the center of the distributed system ●  Functions controlled by TCMS include ●  collecting line voltage, ●  controlling the train engines, ●  opening and closing the train doors, ●  upload of diagnostic data. 5 About Bombardier
  6. 6. ●  Vehicle ●  Vehicle Integration testing ●  System ●  System Integration testing ●  Software ●  Modules integration testing ●  Modules testing ●  “Get to the driver’s perspective as soon as possible” 6 About Bombardier … and their testing Responsibility of a Developer!
  7. 7. ●  IEC 61131-3 ●  Function Block Diagram ●  Graphical programming language ●  Based on element composition ●  Various predefined elements 7 Development Environment
  8. 8. ●  Coverage ●  Strict measurements mandated by safety standards ●  Tooling ●  No tool support to measure coverage on FBD ●  Time ●  Creating tests just to reach the coverage leaves very little time for functional testing 8 Current challenges … with module testing
  9. 9. 9 CompleteTest
  10. 10. 10 Research approach FBD  Program Transformation 1 Annotation2 UPPAAL  timed   automata ... Reachability   properties ∃  ♢  β  , ∃  ♢  pi  ,   ∃  ♢  c0  and  c1 Test  traces Test  cases (Step)  (1) (Time)  (20s) (Inputs)  (0  1  23) (Outputs)  (1  2  543,9) UPPAAL   model  checker Test  Generation 3 Logic-­‐‑based   coverage  criteria DC,  CC,   MC/DC ”Automated Test Generation using Model-Checking: An Industrial Evaluation”, Eduard Paul Enoiu, Adnan Causevic, Thomas J. Ostrand, Elaine J. Weyuker, Daniel Sundmark, Paul Pettersson. International Journal on Software Tools for Technology Transfer, 2014, Springer.
  11. 11. ●  Time required to generate tests ●  satisfying the DC, CC and MC/DC logic coverage criteria 11 Case Study at BT
  12. 12. ●  For 34 of the 157 programs, the tool did not terminate after running for a substantial period of time. ●  Cut-off time was set to 10 minutes. ●  There is an indication that as the number of decisions increases, the performance deteriorates and the cost of using the tool may become prohibitive. ●  But, if you have so many decisions, how would you test it manually? 12 Case Study at BT (2)
  13. 13. ●  Should a developer generate the tests from scratch? ●  Should a developer provide manual tests first? ●  How many (or how much in terms of coverage)? ●  When should a developer use the Validate Test button? ●  Should developers use this tool at all or should we automate that as well (nightly build, etc.) ? 13 Best Practices?
  14. 14. ●  Internal behavior of each function block has to be modeled ●  Currently, blocks from the standard language are modeled ●  and a few custom made by Bombardier which are often used ●  State-space explosion ●  a known problem of model-checking ●  large number of Boolean or Integer input values ●  but, again, how would you do it manually ? 14 Limitations
  15. 15. ●  Complementing model-checking with: ●  Static analysis of FBD’s ●  Search-based software testing ●  Trying some new approaches: ●  Different model-checkers, SMT solvers, … ●  Measuring effectiveness of generated tests ●  In terms of how good they are in finding faults ●  Experimenting with FBD programs from other organizations 15 On-going and Future work
  16. 16. ●  Adnan Causevic adnan.causevic@mdh.se ●  Eduard Enoiu eduard.paul.enoiu@mdh.se ●  www.completetest.org 16 Questions?

×