Visual Learning Pulse - Final Thesis presentation
The final presentation of the master thesis project Visual Learning Pulse: Flow Prediction and Feedback in Self-Regulated Learning, a project collaboration between the Department of Data Science and Knowledge Engineering of the Univeristy of Maastricht and the Welten Institute of the Open University in the Netherlands. TITLE: Visual Learning Pulse: Flow Prediction and Feedback in Self-regulated Learning ABSTRACT: Visual Learning Pulse is a Master thesis research project developed in cooperation with the Welten Institute, the Research Centre for Learning, Teaching and Technology at the Open University of the Netherlands, and partially nanced by the European project Learning Analytics Community Exchange (LACE). Visual Learning Pulse explores whether physiological and physical data such as heart rate, step count and weather data if correlated with learning activity data can be used to predict learning success in self-regulated learning settings. To verify this hypothesis an experiment was opportunely designed, consisting of three phases, lasting six weeks and involving nine participants, each of them wearing a Fitbit HR wrist band and having their application usage recorded during their learning and working activities throughout the day. An ad-hoc infrastructure for longitudinal and multi-modal data was designed and implemented. The data from di erent sources were stored using the Experience API (xAPI) data standard in a cloud distributed database called Learning Record Store. The participants (doctoral students at the Open Universiteit) - were asked to rate their learning experience through an Activity Rating Tool indicating their perceived level of productivity, stress, challenge and abilities. These self-reported performance indicators were used as training labels for the two algorithms employed for prediction of time series data: the Vector Autoregression and Linear Mixed E ect Model. A major task of the thesis consisted of developing the software application to pre-process, perform the analysis and generate the predictions on real time, in order to provide timely feedback to the users about their learning performances. Although not showing high overall accuracy, the prediction models were successfully learnt and used in production: in the third phase of the experiment, two visualisations mechanisms were used, the Learner Dashboard and the Feedback Cubes. In addition, a conceptual paper of Visual Learning Pulse, illustrating setup and overall the rationale was presented at the Learning Analytics & Knowledge conference 2016 in Edinburgh, Scotland and was included in CEUR workshop proceedings.
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Visual Learning Pulse - Final Thesis presentation
Similar to Visual Learning Pulse - Final Thesis presentation (20)
More from Daniele Di Mitri
More from Daniele Di Mitri (19)
Recently uploaded
Recently uploaded (20)
Visual Learning Pulse - Final Thesis presentation
- 1. UM supervisors Kurt Driessens Pietro Bonizzi OU supervisors Hendrik Drachsler Maren Scheffel Maastricht, 29th June 2016 Daniele DI MITRI presents MSc Thesis in Artificial Intelligence
- 2. Visual Learning Pulse – Final thesis presentation 2 What was done - visual 21/09/2015 Internship starts 21/12/2015 Internship ends Design pre-test Experim ent Implement 29/06/2016 Thesis ends Report 01/02/2016 Thesis starts Paper submitted to LAK conference Analysis,literature 18-25/06/16JTEL summerschool 25-29/04/16 LAK conference Coding 31/03/16 Announcing Presentation Trainingphase Validation phase Exploitation phase Reporting 17/05/16 11/04/16 30/05/16 8 months of work
- 3. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation What was done - numbers 3 1 publication 2 conferences 2 software apps 6 presentations 9 blog posts 20+ meetings 1800 lines of code
- 4. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Learning Analytics & Knowledge Conference 2016 4 Di Mitri, Scheffel, Drachsler, Börner, Ternier 2016 - Learning Pulse : using Wearable Biosensors and Learning Analytics to Investigate and Predict Learning Success in Self - regulated Learning.
- 5. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 5 Background, meaning, vision 5
- 6. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data deluge in Education 6
- 7. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 7 Data-driven approach Picture from tincanapi.com
- 8. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Self-regulated learners need support 8 Self-Regulated Learning → no guidance → no feedback → no support
- 9. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Related work 9 Signals, Purdue University Student success, University North Dakota S3, Desire To Learn
- 10. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Dimensions of Learning 10
- 11. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Machine Learning with Human Learning 11 y = f(X) Learning performance Predictive Model Input space
- 12. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 12 Own Approach 12
- 13. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 13 RESEARCH QUESTION Can we predict learning success out of physiological, activity and weather data? 13
- 14. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 14 Participants ● 9 PhD students at Welten institute ● Different disciplines ● Different OS
- 15. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Experiment Timeline 15 11th to 29th April 2016 1st phase: “Training” Participants rate their activity 17nd to 27th May 2016 2nd phase: “Validation” Participants rate their activity + Feedback visualization 30th May to 3th June 2016 3rd phase: “Exploitation” Individual and group Feedback visualization
- 16. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 16 Input space Context Body Activities Body: physiological (heart-rate) and physical responses (steps) - from Fitbit HR Activities: applications used during learning from RescueTime Context: weather data from OpenWeatherMap
- 17. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 17 Hypothesis space: the Flow Csikszentmihalyi, 1972 Theoretical Empirical
- 18. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Activity Rating Tool 18 Productivity How productive was last activity? Stress How stressful was last activity? Challenge How challenging was last activity? Abilities How prepared did you feel for the activity? FLOW Participants rate hourly, from 7AM to 7PM A scalable web app! Client: Bootstrap + Jquery Sever: GoogleApp + Python “Very easy to use!”
- 19. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 19 VLP Data Model
- 20. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 20 Scheffel, M., Ternier, S., & Drachsler, H. (2016). The Dutch xAPI Specification for Learning Activities http://bit.ly/DutchXAPIreg Experience API Data storing format for the Learning Record Store
- 21. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 21 The Data journey
- 22. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation A complicated Architecture 22
- 23. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data collection 23 ● PULL data from the 3rd party APIs ● Make the xAPI triples ● PUSH data in the LRS ● It’s scalable! ● No collisions ● It’s fast ● It’s Interoperable Learning Pulse Server + Learning Record Store
- 24. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Data Processing application 24 Script in Python on a VM which processes data in real time
- 25. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 25 Transformed dataset ● Time Series: tabular representation ● 5 minutes intervals ● Enough samples now! ● Easier view for Machine Learning ● Signal resampling needed 8728 observations X 29 attributes
- 26. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 26 (Issue 1) Extract features from TS Heart Rate Variability and Heart Rate Entropy… didn’t work SOLUTION ● Mean of the signal ● Maximum ● Minimum ● Standard Deviation ● Average change Heart-ratesignalfor15mins
- 27. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 27 (Issue 2) Reduce sparsity Rule based grouping of applications Subjects can be compared Applications used are too sparse Let’s create application categories
- 28. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 28 (Issue 3) Ladder effect Trade-off: number of samples vs How much bother people NO SOLUTION
- 29. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 29 (Issue 4) Dependency constraint Independence constraint Knowing one value of et for one observation does not help us to guess value of et+1 yt = α + βX t + et cov(et ,et+1 ) = 0 FIXED Effect RANDOM Effect SOLUTION follows...
- 30. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Approach 1) Vector Auto Regression 30 x0 x1 x2 x... an t0 x x x ... x t1 x x x ... x t... ... ... ... ... ... tp x x x ... x tp+1 ? ? ? ? ? tp+2 ? ? ? ? ? PAST PRESENT FUTURE Time intervals PREPROCESS Timeseries were LOGged LIMITATIONS ● Participants need to be treated separately ● Doesn’t work with categorical data ● Doesn’t work with random effects
- 31. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Approach 2) Mixed Effect Linear Model 31 x0 x1 x2 ... xn-1 xn g y t0 x x x ... x x 1 y t1 x x x ... x x 1 y t2 x x x ... x x 2 y t... ... ... ... ... ... ... 2 y tp- 1 x x x x x x 3 y tp ? ? --- --- --- --- x ? Random EffectsFixed Effects Group Tried both Python and R implementations Used R-squared for goodness-test LIMITATIONS ● Poor results ● Convergence time ● Mono-output ● Algebra errors
- 32. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 32 Issue: high inter-subject variability i.e. Participants have rated very differently
- 33. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Visualisations 33 33
- 34. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 34 Learner Dashboard 9 personal Dashboard + 1 public
- 35. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 35 *Börner, Tabuenca, Storm, Happe, and Specht. 2015Feedback Cubes
- 36. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Conclusions 36 36
- 37. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 37 Limitations ● Low accuracy of prediction RQ-answer: YES but prediction accuracy can be improved. ● Real-time issues Fitbit synchronisations, Virtual Machine performance ● 3rd party API constraints ● No great solution for sparse data (manual grouping)
- 38. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 38 Achievements ● Real-time system works ● Data collection was seamless ● Good dataset for experiments (will be open sourced) ● Useful insights IoT in Learning ● Reusable architecture
- 39. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation Future ideas 39 39
- 40. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 40 Modelling sparse data (idea) a1 a2 a3 a4 a5 a6 Ft Ft+1 Ft+2 Hidden Flow values Ft+3 Random sampling a7 a8 Visible applications Hidden Markov Chains + Random sampling
- 41. Open Universiteit Welten Institute Visual Learning Pulse – Final thesis presentation 41 Thank you! Q&A “Life can only be understood backwards, but it must be lived forwards” - Kierkegaard