Evaluating the Student Activity Meter: Two Case Studies.

1. EVALUATING THE STUDENT ACTIVITY METER: TWO CASE STUDIES. Sten Govaerts, Katrien Verbert & Erik Duval http://www.role-project.eu 1 Thursday 8 December 11

2. OVERVIEW • Problem statement • Objectives • Design & Implementation • Evaluations • Conclusion & Future work 2 Thursday 8 December 11

3. PROBLEM 3 Thursday 8 December 11

4. PROBLEM 3 Thursday 8 December 11

5. OBJECTIVES • self-monitoring for learners • awareness for teachers • time tracking • learningresource use and recommendations 4 Thursday 8 December 11

6. STUDENT ACTIVITY METER (SAM): DEMO. 5 Thursday 8 December 11

7. THREE EVALUATIONS • with CS students • ICWL 2010 paper: “Visualizing Activities for Self-reﬂection and Awareness” • with CGIAR courses and teachers • with Learning and Knowledge Analytics course participants 6 Thursday 8 December 11

8. 2010 PAPER • usability and user satisfaction evaluation with 12 CS students • in general, people understand the visualizations well! • solved some issues and requests since Thursday 8 December 11

12. METHODOLOGY. • design-based research: give students & teachers a tool and evaluate • usability studies • paper mockups • surveys 8 Thursday 8 December 11

13. CASE STUDY • 20 persons: 3 CGIAR, 2 Law, 8 CS teachers & 7 CS TA’s. • An online survey about usefulness, teacher issues and how the tool can resolve these. • on average: 40 mins are spent using SAM. 9 Thursday 8 December 11

14. CGIAR CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ? Being aware of what students are doing ✔ ✔ Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ✔ Knowing how and when online tools have been used ? ✔ Finding the students who are not doing well ✔ ? Finding the best students ? ? Knowing how much time students spent ? ✔ Knowing if external learning resources are used ✔ ? 10 Thursday 8 December 11

15. CGIAR CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ? Being aware of what students are doing ✔ ✔ Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ✔ Knowing how and when online tools have been used ? ✔ Finding the students who are not doing well ✔ ? Finding the best students ? ? Knowing how much time students spent ? ✔ Knowing if external learning resources are used ✔ ? 10 Thursday 8 December 11

16. CGIAR CASE STUDY Bar chart ✔ Parallel coordinates ✔ Line chart ✔ 11 Thursday 8 December 11

17. CGIAR CASE STUDY 12 Thursday 8 December 11

18. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 12 Thursday 8 December 11

19. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 12 Thursday 8 December 11

20. CASE STUDY • open course on learning and knowledge analytics, http://bit.ly/ dWYVbX • 12 visual analytics enthousiasts + experts (who also teach) • almost identical survey to CGIAR case. 13 Thursday 8 December 11

21. CASE STUDY • open course on learning and knowledge analytics, http://bit.ly/ dWYVbX • 12 visual analytics enthousiasts + experts (who also teach) • almost identical survey to CGIAR case. 13 Thursday 8 December 11

22. LAK CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ✔ Being aware of what students are doing ✔ ? Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ? Knowing how and when online tools have been used ✗ ? Finding the students who are not doing well ✔ ? Finding the best students ? ✗ Knowing how much time students spent ? ✔ Knowing if external learning resources are used ? ? 14 Thursday 8 December 11

23. LAK CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ✔ Being aware of what students are doing ✔ ? Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ? Knowing how and when online tools have been used ✗ ? Finding the students who are not doing well ✔ ? Finding the best students ? ✗ Knowing how much time students spent ? ✔ Knowing if external learning resources are used ? ? 14 Thursday 8 December 11

24. LAK CASE STUDY Bar chart ? Parallel coordinates ? Line chart ? 15 Thursday 8 December 11

25. LAK CASE STUDY 16 Thursday 8 December 11

26. LAK CASE STUDY more metrics verify the status of the classroom activity chronological course dwell time self-reﬂection to measure progress and increase motivation ﬁnd students experiencing problems and low more data engagement 16 Thursday 8 December 11

27. LAK CASE STUDY more metrics verify the status of the classroom activity chronological course dwell time self-reﬂection to measure progress and increase motivation ﬁnd students experiencing problems and low more data engagement 16 Thursday 8 December 11

28. IDEA’S FROM EXPERTS the used resource types detailed information per student detailed information of 2 students detailed usage stats of resources stats or vis. on content creation 17 Thursday 8 December 11

29. IDEA’S FROM EXPERTS 2 the used resource types 5 detailed information per student 4 detailed information of 2 students 3 detailed usage stats of resources 1 stats or vis. on content creation 17 Thursday 8 December 11

30. CONTINUED WORK... 18 Thursday 8 December 11

31. CONTINUED WORK... 18 Thursday 8 December 11

32. FUTURE WORK • extend SAM with feedback of experts • evaluate the usefulness in a larger group of students • evaluate with teachers during and after a course. 19 Thursday 8 December 11

33. YOUR DATA IN SAM ?! ask me! sten.govaerts@cs.kuleuven.be 20 Thursday 8 December 11

34. 2 slides will appear on http://www.slideshare.net/stengovaerts slides will appear on http://www.slideshare.net/stengovaerts 21 Thursday 8 December 11

Evaluating the Student Activity Meter: Two Case Studies.

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Evaluating the Student Activity Meter: Two Case Studies.

Similar to Evaluating the Student Activity Meter: Two Case Studies. (20)

More from Sten Govaerts

More from Sten Govaerts (20)

Recently uploaded

Recently uploaded (20)

Evaluating the Student Activity Meter: Two Case Studies.