Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evaluating the Usability of a Study Support Mobile App in Higher Ed

291 views

Published on

Mlearn conference 2019 delft

Published in: Education
  • Be the first to comment

Evaluating the Usability of a Study Support Mobile App in Higher Ed

  1. 1. Evaluating the Usability of a Study Support Mobile App in Higher Education Mohammad Khalil, Jacqueline Wong, Martine Baars, Farshida Zafar, & Barbara Wasson mLearn Conference, Delft 18.09.2019
  2. 2. Background & Motivation ◉ Self-study is an important factor in student success (Wijnen et al., 2017) ◉ Self-study entails self-regulated learning (SRL) and use of effective learning strategies ◉ Self-testing can be use for self-monitoring as part of SRL (Baars, 2014) 2 Jacqueline Wong, 18.09.19, Delft
  3. 3. Background & Motivation ◉ Smartphones account for 75% of mobile devices used in educational context (Wu et al., 2012) ◉ Growing number of apps used for learning ◉ Few studies examined the usability of self-study apps! ◉ Research on usability for self-study apps can help to: ○ Improve apps and enhance user experience ○ Carry out finer research ○ Support teacher instructions based on the difficulties students are facing ○ Implement support on app use for self-regulated learning, and in turn, improve learning performance 3
  4. 4. “ 4 Usability is the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” -ISO FDIS 9241-210 Jacqueline Wong, 18.09.19, Delft
  5. 5. What is the EUR App? Jacqueline Wong, 18.09.19, Delft
  6. 6. 6 Eur App ● App developed in 2016 ● Made by the students for the students! ● Led by a lecturer in the law faculty - Farshida Zafar ● Formative assessment tool with a focus on gamification
  7. 7. 1. Teacher Perspective 7 ● Teachers have control over the content. ● Teachers can drop in 250 multiple choice questions divided over categories ● Teachers can reference to a literature or video for feedback ● Quick changes can be done ‘on the fly’ ● Simple descriptive analytics (successful students and percentage of answers) Jacqueline Wong, 18.09.19, Delft
  8. 8. 2. Student-face design 8 ● Four game modalities (practice, compete, test, and battle) ● Availability of instant feedback in practice mode ● The app focuses on knowledge testing and entertaining ● Gamification elements are available (i.e., leaderboard, time restrictions...etc.) Jacqueline Wong, 18.09.19, Delft
  9. 9. 9 Eur App - Screenshots Jacqueline Wong, 18.09.19, Delft
  10. 10. 10 Jacqueline Wong, 18.09.19, Delft
  11. 11. The Research Study Jacqueline Wong, 18.09.19, Delft
  12. 12. 12 Research Questions ◉ How often did the students use the app and which mode was mostly used for studying in the course? ◉ What were the students’ perceived usability and enjoyment of the app? Jacqueline Wong, 18.09.19, Delft
  13. 13. Sample Profile ◉ Survey sent to 800 students ◉ Year 1 Bachelor students from Erasmus University ◉ Course “Introduction to constitutional and administrative law” ◉ Response rate was 10.12% ◉ Female : Male (65% : 36%) 13
  14. 14. Criteria used for evaluation 14 ◉ Learnability: How easy is the application to learn and start working on. ◉ Efficiency: How efficient is the system in terms of productivity ◉ Memorability: How memorable is the application when a user leaves it for a while ◉ Errors: The application should have a low error profile without catastrophic bugs ◉ Satisfaction: How pleasant is the application to use and are the users satisfied of its functionalities? ◉ Enjoyment: Closely related to motivation by which we examine whether students were enjoyed using the application. ◉“It was easy for me to learn how to use the app and navigate through it” ◉I was able to use the app without frustration. ◉I remembered how to use the app when I leave it for a while ◉The app gave error messages when I made a mistake ◉I was satisfied with the app as a study tool ◉I enjoyed using the practice mode of the game app Nielsen (1994) and Mun & Hwang (2003)
  15. 15. How often did the students use the app and which mode was mostly used for studying in the course? 15 RQ1
  16. 16. What were the students’ perceived usability and enjoyment of the app? 16 Criteria Cronbach’s a Mean Learnability (4 items) .702 5.59 Efficiency (3 items) .716 5.43 Memorability (2 items) 4.31 Errors (2 items) 4.80 Satisfaction (4 items) .818 5.75 Enjoyment (5 items) .707 5.26 RQ2
  17. 17. Results Mode usage The practice and test modes were the most used ones. The battle and compete modes were not as highly used. App usage Around 50% of the students used the app once a week to practice. 60% of the students used it to assess their knowledge Highest usability criteria Satisfaction (e.g., app is user-friendly) was the highest criterion of the usability testing Lowest usability criterions Memorability (e.g., remembering how the app works, app recalling same page…) was the lowest criterion of the usability testing Other facts Efficiency and enjoyment of the app were perceived on average by the students 17
  18. 18. Study key outcomes ◉ Students used the app as more of a diagnostic tool rather than a study tool ◉ Perhaps, students prefer modalities that provide feedback rather than models with gamification ◉ According to Nielsen (1994), the application can be considered “Stable” ◉ The gamification models of the app should be improved 18
  19. 19. Any questions ? Thanks! 19 Mohammad Khalil mohammad.khalil@uib.no Jacqueline Wong wong@essb.eur.nl Martine Baars Farshida Zafar Barbara Wasson
  20. 20. Any questions ? You can find me at: ◉ wong@essb.eur.nl ◉ mohammad.khalil@uib.no Thanks! 20 Mohammad Khalil, Jacqueline Wong, Martine Baars, Farshida Zafar, & Barbara Wasson You can find me at: ◉ wong@essb.eur.nl ◉ mohammad.khalil@uib.no

×