by Paul Bailey

1. Curriculum analytics: Using data from student
learning analytics
December 2018
Paul Bailey, Research and Development, Jisc
Niall Sclater, Consultant

2. Jisc Curriculum Analytics2
https://www.jisc.ac.uk/learning-analytics

3. What is curriculum analytics?
“learning analytics is the measurement, collection,
analysis and reporting of data about learners and their
contexts, for purposes of understanding and optimising
learning and the environments in which it occurs”
SoLAR – Society for Learning Analytics Research

4. Towards Education 4.0 (Davies Model revisited)
Jisc Futures4

5. User stories
Jisc Curriculum Analytics5
Outputs from workshop Nov 2018
•63 user stories
•Institutional goals
•Improving benchmarks (e.g. NSS, TEF)
•Quality assurance
•Resource usage
•Course/module review
•Data-informed course and module design
•Course reporting dashboard
•Improving assessment
•Online learning optimisation
•Feedback driven course design

6. Jisc Curriculum Analytics6

7. What data do we get from learning analytics?
What additional data do we need?

8. Building on the LA
Architecture
Jisc Curriculum Analytics8
Data Collection
Data
Storage
and Analysis
Presentation
and Action
Learning
Data Hub
Student Records
Engagement Data
Staff dashboards in
Data
Explorer/Curriculum
design tools
Course and Module Information
Lecture Activity
Learning Outcomes
Student feedback
Data Processing
Tools
Data Estate
Institutional BI Tools

9. Jisc Curriculum Analytics9
Course
Course Instance
Module
Module Instance
Sessions i.e.
Lectures, labs, etc.Assessment
Assessment Instance
Learning outcomes
Learning Design
Activities
Additional course
information
Additional module
information
Feedback data
Existing UDD New Data
Calculations
Lecture capture data
Derived course and
module data
Student activity data
metrics
Student Activity Data
– VLE, attendance, …

10. What data do we need?
Jisc Curriculum Analytics10
Information Sources Issues
Student information Student record systems Sensitivity of data
Student course information Student record system Structure and quality
Course and module information Curriculum management system,
KIS data, course handbook
Not fit for purpose
Behaviour data Learning activity software,
attendance data
Open APIs, aggregation, patchy,
social apps
Assessment data Student records system, VLE Timely data, centrally available
Learning design data Lectures and module handbooks Rarely available, no culture, staff
resistance
Feedback data Course evaluation survey tools Electronic? Identifiable?
Frequency of feedback

11. Next steps for curriculum analytics
Jisc Curriculum Analytics11
•Curriculum data gathering pilot (April - Sept 2019)
•~12 pilot institutions with access to learning analytics data
•Identify more detail use cases for curriculum analytics
•Create metrics that can give meaningful insights
•Define a minimum viable data set and how to gather it
•Explore changes in practice that will facilitate curriculum analytics

12. Overview of Learning Design and Curriculum
Objects

13. Emerging types of analytics in education
Jisc Learning Analytics Network13
Educational analytics
Learning
Curriculum
Welfare
Intelligent campus
Employability & apprenticeship
Institutional

14. What makes curriculum analytics different?

15. Potential application of curriculum analytics
Jisc Learning Analytics Network15
Potential applications
Identify modules which appear to result in better learning and/or greater
student satisfaction
Understand which aspects of the curriculum result in better learning and
build this knowledge into future curriculum development
Understand how sequencing of exams affects student performance

16. Morris, Finegan & Wu
Jisc Curriculum Analytics16
Looked at VLE logs from 354 students
Completers more engaged than those who withdrew
3 indicators significant:
•No of discussion posts viewed
•Number of content pages viewed
•Time spent viewing forums
Morris, L. V., Finnegan, C. & Wu, S.-S., 2005, Tracking student behavior, persistence, and achievement in online
courses, The Internet and Higher Education, Volume 8, pp. 221-231.

17. 17
https://www.flickr.com/photos/masamunecyrus/
Tim Hardy, University of Maryland Baltimore
County
Introduced adapative release feature
Analytics showed 20% improvement in student
performance
Students performed better in next courses too
Fritz, J., 2013, Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course
Designs, Louisville, CO: EDUCAUSE Center for Applied Research, pp. 6-7.

18. Jisc Curriculum Analytics18
Quan Nguyen, Bart Rienties et al, OU

20. Multiple uses for data gathered for curriculum
analytics
Jisc Learning Analytics Network20
Examples
Real-time adjustments to teaching by the lecturer
during the lecture
Subsequent enhancements to the curriculum to
provide more explanatory material the next time the
lecture is delivered
Correlations with grade data to ascertain whether such
teaching methods appear to be effective

21. Curriculum analytics is the use of data
to help understand and enhance the
curriculum.

22. Potential users of curriculum analytics
Jisc Learning Analytics Network22
• to see which aspects of their modules are proving more effective than
others
Lecturers
• and others responsible for overseeing and reporting on module and
course performance
Associate deans
• and others who can identify and promote good practice in module
development
Learning technologists
• who wish to assess the relative success of different schools or faculties
and develop policy to ensure that good practice is embedded
Senior management

23. “We should only provide learning content and
activities where we have ways of measuring
their impact on student learning.”

24. Lecture
Module code Politics101
Number 7 [out of 10]
Title Marxism
Learning outcome(s)
addressed
Understand Karl Marx’s main
theories
Lecturer Dave Wilson
Date & time 27/01/2019 10:00
Location Renfield 210
Jisc Learning Analytics Network24

25. Curriculum objects
Jisc Learning Analytics Network25
Lecture
Descriptive data
Module code
Lecture number
Learning objective(s) addressed
Lecturer
Date & Time
Location
Individual usage data
Student attended (yes/no)
Stakeholder
Lecturer
Course director
Aggregated usage data
Number of attendees
%age of cohort who attended
Observation
Attendance lower than
expected
Question
Is low attendance because
students are studying for
exams?
Intervention
Stress importance of this
lecture with next cohort
Correlated data
Attendance at this lecture +
attendance at others for this
module
Analytics
Trend in
attendance
over time
Observation
Attendance dropping
more than expected as
module progresses
Question
Are lectures compelling
enough?
Are there timetabling conflicts
with other modules?
Intervention
Review student feedback
and adapt curriculum
accordingly

26. A curriculum object describes an aspect of
the curriculum, the data and the analytics that
can be used to enhance it

27. Activity: Writing Curriculum Objects

28. Activity
Jisc Curriculum Analytics28
Outcome
A prioritised list of curriculum objects to be developed
•In groups – what could be a curriculum object – 10 mins
•In pairs – chose one and activity writing curriculum objects – 2
mins
•10 mins in groups to discuss
•20 mins feeding back from groups (maybe some sort of
collation activity)
•10 mins prioritising/voting

29. jisc.ac.uk
Except where otherwise noted, this work
is licensed under CC-BY-NC-ND
Contact us
Contact
Paul Bailey
Senior Co-Design Manager,
paul.bailey@jisc.ac.uk