Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automated system testing for a learning management system

Präsentation Masterarbeit, TU Graz, November 2019

  • Login to see the comments

  • Be the first to like this

Automated system testing for a learning management system

  1. 1. WISSEN TECHNIK LEIDENSCHAFT Automated system testing for a learning management system 31.10.2019 Lukas Krisper
  2. 2. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 2
  3. 3. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 3
  4. 4. A shift towards Agile Software Development • Over the last two decades … • Emerge of agile methodologies in software development • Acceleration of software delivery and feedback cycles • Automation of manual processes • Quality Assurance? • Only 20% of IT organisations have automated more than half of their test cases (State of Testing Report, 2018) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 4 Motivation
  5. 5. New Releases in Summer 2019 • TeachCenter 3.0 • Learning Management System (LMS) • Migration of whole LMS to new Moodle Version (3.5) • New GUI for all users • Set of Unit-Tests existed • Sparely documented manual tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 5 Motivation
  6. 6. Research Questions • (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 6 Motivation
  7. 7. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 7
  8. 8. Focus of Automation Activities • Software Quality Characteristic*: Functional Suitability • Testing Level: System Testing • Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 8 Test Automation *According to ISO-25010
  9. 9. Software Quality Characteristic: Functional Suitability Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 9 Test Automation • Does the software product do what it is supposed to do? • Functional testing as basis of all testing activities • A software product that does not provide functional suitability is rather worthless • E.g. calculator that calculates very fast and has a nice UI, but the calculations are wrong 2 + 2 = 5
  10. 10. Testing Level: System Testing Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 10 Test Automation • Software product is considered in its entirety • Test environment has close resemblance to production environment • Tests conducted from user’s perspective • Typically using the systems GUI for interaction • Automated system tests as second line of defense • If tests fail at this level → malfunctioning system + wrong or missing test at lower testing level
  11. 11. Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 11 Test Automation • Is already existing functionality affected by the changes applied to a software? • The first type of tests to be considered when automating tests • Quickly run a set of tests after applying changes: • Extensions to a feature • Bug fixes • Refactorings • … • Especially useful when changes happen late in the project and time for manual testing is short
  12. 12. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 12
  13. 13. What Test Cases should be automated? • Important consideration! • Interview with first level support of TeachCenter 2.0 • Research in literature on usages of LMS • Results: • Knowledge on most important use cases for students and teachers E.g. reading the course’s contents, communicating with other participants, submitting material to the course, … • > 30 test cases combined to different test suites Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 13 Test Cases for a LMS
  14. 14. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 14
  15. 15. Real Data vs Synthetic Data • Real Data • Transfer data from production environment to test environment • Data has to be edited (according to laws on data protection/privacy*) • Important: Keep consistency in data set! • Synthetic Data • Newly created data for the purpose of testing • Always GDPR compliant Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 15 Test Data for a LMS * e.g. General Data Protection Regulation (GDPR)
  16. 16. Data for a LMS • A LMS typically contains data on • Courses (course descriptions, learning material, …) • Data on people involved in the courses (names, email-addresses, …) • Data from interaction of people with the courses (submissions, …) • TeachCenter 3.0 is tested with both synthetic and real data • Data used for automated tests is labeled as such Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 16 Test Data for a LMS
  17. 17. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 17
  18. 18. Overview on Infrastructure Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 18 Implementation and Infrastructure
  19. 19. Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Suite 1 Test Case 1 Test Case n Test Case 2 … Overview on Implementation Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 19 Implementation and Infrastructure Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Data Manager Test Driver Utility e.g. initialise headless Chrome browser with cleared cache e.g. get PDF files for submissions, references to test data stored in JSON files, … Page Object 1Page Object 1Page Object 1 TestNG Page Object 1Page Object 1 Helper Classes e.g. taking screenshots on errors See following slides
  20. 20. PageObject Design Pattern Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 20 Implementation and Infrastructure • Components of a website are modelled into reusable objects → “Page Objects” • Page Objects offer functionality to interact with website (click links, enter input fields, …) • Tests use Page Objects to interact with website (perform action → assert result of action)
  21. 21. Identification of Objects in UI Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 21 Implementation and Infrastructure • Tests should be executable regardless … • screen resolution • window sizes • language settings • Identification of objects using stable features (IDs or XPath expressions) DOM Element <button type= "button" id= "btn1" data-role= "next "> Next </button > ID btn1 XPath expression //button[@data-role=’next’]
  22. 22. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 22
  23. 23. (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • Regression Tests for „Functional Suitabilty“ • System Testing Level • Large software system = many integrated components that interact with each other → “high” testing level to ensure stability and detect side effects • Continuous Integration Environment (Automation Server) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 23 Conclusion
  24. 24. (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • Identification of use cases • Interview with 1st level support + literature research • Prioritisation of use cases • Implementation of test cases based on use case • Execution of suites of test cases according to prioritisation of use cases • Multiple times a day ~ once a week Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 24 Conclusion
  25. 25. (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? • Synthetic and real test data • 1 configured course with various activities and resources • 1 student‘s account (enrolled to course) • 1 teacher‘s account (enrolled to course) • 1 PDF file • Test data is persisted on test system (references in JSON-files) … • Or provided to the test cases on demand • Separation of test data and automated test cases • No hardcoded test data Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 25 Conclusion
  26. 26. Thank you for your attention! Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 26 Conclusion

×