Can you predict a degree from five weeks' worth of VLE data? Learning analytics research from the field
•Download as PPTX, PDF•
0 likes•4,188 views
Speakers: Jason Bailey, learning technologies adviser, University of Brighton Katie Piatt, e-learning manager, University of Brighton The University of Brighton are in their second year of student dashboard delivery and are also modelling historical data as part of their learning analytics work in order to predict student outcomes.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Can you predict a degree from five weeks' worth of VLE data? Learning analytics research from the field
Similar to Can you predict a degree from five weeks' worth of VLE data? Learning analytics research from the field (20)
More from Jisc
More from Jisc (20)
Recently uploaded
Recently uploaded (20)
Can you predict a degree from five weeks' worth of VLE data? Learning analytics research from the field
- 1. Can You Predict a Degree from 5 Weeks Worth of VLE data? Learning Analytics Research from the field 6 March 2018 | ICC, Birmingham
- 2. Teaching and learning excellence in a digital age Can you predict a degree from five weeks’ worth of VLE data? Learning analytics research from the field Jason Bailey, Learning technologist advisor, University of Brighton Katie Platt, elearning manager, University of Brighton
- 3. >Dashboard and emojis (How are you today) >Regression modelling – activity and demographic data to model module score >Regression modelling – effect of using increasing number of weeks of activity data In this session
- 4. >Brighton Business School students >Data capture of VLE Activity >VLE ”clicks” per module are counted as Virtual Attendance >Students / Staff dashboard >View VLE Grades >View Seminar Attendance >View VLE time online >View “How are you doing” The Project
- 6. > VLE (Blackboard Learn) log of activity data including: > Timestamp of activity > Event details - course id (module id) and content id > Calculating total time per module per day proved unhelpful (session time out) > Any activity per module (1 click or many) is counted as virtual attendance for a day > Activity count gives the total number of daily visits to a module (semester or year) How we measure activity
- 7. >We are attempting to model and predict a student’s final module score using activity count and demographic data
- 8. >Given 5 or 10 students studying 1 module >With different Activity counts e.g 0, 10, 20, 30, 50, 100 >Plot a graph of their expected final grade for a module (unit of study) >Here’s an empty graph…. What do you think?
- 12. >The line of best fit can be estimated by eye for two dimensional cases e.g. age and height >For multi-dimensional we used regression methods
- 13. > GLM: General Linearised Model > Like simple regression allowing for categorical (demographic) data We also used > KRR: Kernel Ridge Regression > Includes a penalty coefficient to reduce overfitting > Works with co-variate data > RF: Random Forest Regression > A black box approach > Computes ensemble average model for random sets of fields & records > Reduce overfitting Regression methods
- 14. Modelling module score using activity count and demographic data
- 16. How we compared the models Root Mean Square (RMS) error
- 18. Comparing models using less activity count data How much activity count data do we need
- 22. >Increasing number of weeks of activity count data reduces the RMS error
- 24. >Activity Count is providing most of the model fit
- 26. >Look at better measure of activity count >Targeting of clicks >Better representation median/mean/changes versus total >Obtaining previous education and experience has proven difficult >Inclusion of other engagement measures >First assessments >Discussions and posts Future work
- 27. Future work –using first assessment
- 28. Except where otherwise noted, this work is licensed under CC-BY-NC-ND. Jason Bailey Learning Technologies Adviser j.bailey@brighton.ac.uk We are eLearning Team University of Brighton blogs.brighton.ac.uk/elearningteam/ brighton.ac.uk Katie Piatt eLearning Manager k.piatt@brighton.ac.uk
- 29. Any questions?
Editor's Notes
- Start
- KP Hello and welcome…. In this session we’re presenting our work using regression to model module score using VLE Activity data and demographic data. We’ll show a typical result of the modelling results and some work to look at how many weeks worth of VLE data we can use
- The work presented here is part of an ongoing pilot project to present dashboard data to students in the business school The data is captured from our VLE overnight, processed and presented to students. The following information is presented VLE Grades Seminar Attendance VLE time online “How are you doing” The previous slide included our “how are you doing” emoji widget. Students could select an emoji and provide optional information. They could chose whether or not to be contacted.
- Typical usage of the emojis and dashbaord each week are given. Note the drop off at Xmas and the peaks at times of grades being posted.
- We also wanted to see if we could model and predict the module score –final end of year/semester grade Using VLE activity data and demogtaphic data The activity we concentrated on using is analogous to virtual attendance for the module Any activity per module (1 click or many) is counted as virtual attendance for a day
- Over to you. Just for a minute think about how activity on the VLE might influence a student’s module grade.
- So maybe something like this. Not including demographic data
- Given that data you might try and fit a line of best fit something like this. This line can be drawn “by eye” but could also be calculated using regression
- For 2D cases we could find a line of best fit by eye but we’re also including demographic data
- In this presentation we’re just showing the results of GLM model but we also looked at KRR and RF
- ML189
- When we plot module score against the total activiy count over 40 weeks then we see this. Lots of scatter but the model predicts quite well Black dots are Red Dots are
- When we used the different models we needed a way to compare the modelled and real data. We used RMS
- The RMS error is calculated by taking the square root of the mean of the square of the errors
- When we then look at how increasing weekly activity count data effects the RMS errors we see this.
- In this plot I’m showing the zero origin but I’m going to zoom in as that looks quite flat
- Same module but you start to see a reduction in the RMS error from around 5 weeks. Adding more data reduces the error further. There’s a trade off between a reduction in error and how long you wait before attempting model the final score.
- For a different module we see.
- A significant reduction in error over time and some plateaus around Xmas for example
- Quote: https://stats.stackexchange.com/questions/162465/in-a-random-forest-is-larger-incmse-better-or-worse
- So that’s what we’re doing and this is the further work we want to do. I think the activity count is a bit crude. 1 click for one second in a module is given the same weighting as 100 clicks Time on the module had too many 18 hour gaps Previous education We also think and research has shown that earlier 1st Assessment might be a good indicator overall performance See…
- This chart shows 1st Assessment score around Xmas and compares to a student’s final end of year score for all modules. The average Each dot is one student 1st Assessment and Final Grade
- We are….
- Questions