Jisc learning analytics update-nov2016

The Open University, 2nd November 2016
8th UK Learning Analytics Network Meeting

Programme
Jisc Learning Analytics 2016
10:25 – 11:15 Update on Jisc’s learning analytics programme
11:15 – 11:30 Tea / coffee
11:30 – 12:30 Learning design meets learning analytics, Dr Bart Rienties, Open University
12:30 – 13:30 Lunch
13:30 – 14:15 Parallel session 1: Legal issues for learning analytics, Andrew Cormack, Jisc
Parallel session 2: Addressing the challenges , Il-Hyun Jo, Ewha Womans
University
14:15 – 15:00 Parallel session 1:The potential of blockchain , Prof John Domingue,
Knowledge Media Institute, OU
The design and deployment of a learning analytics dashboard, David Evans,
NorthWarwickshire & Hinckley College
15:00 – 15:15 Tea / coffee – Juniper/Medlar Room,The Hub
15:15 – 15:55 The Learning Analytics Community Exchange, Dr Doug Clow, Institute for
Educational Technology, OU

Paul Bailey, Senior Codesign Manager, Research and Development
Jisc learning analytics service
http://www.slideshare.net/paul.bailey/

Effective Learning Analytics Challenge
Rationale
»Organisations wanted help to get started and have access to standard
tools and technologies to monitor and intervene
Priorities identified
»Code of Practice on legal and ethical issues
»Develop basic learning analytics service with app for students
»Provide a network to share knowledge and experience
Timescale
»2015-16—test and develop the tools and metrics
»2016-17—transition to service
»Sep 2017—launch, measure impact: retention and achievement

Jisc’s Learning Analytics Project
Three core strands:
Learning
Analytics Service
Toolkit Community
Jisc Learning Analytics

Learning Analytics Sophistication Model

Analytics – the bigger picture
https://docs.google.com/presentation/d/1AdBkYHO3hqEJ7W2McYIsAKzF4EgFNYJM9X
GfDOTRYek/edit?usp=sharing
Michael Webb

Descriptive
Analytics
what
happened?
Diagnostic
Analytics
why did it
happen?
Predictive
Analytics
what will
happen?
Prescriptive
Analytics
what should I
do?
Automated
Decision
making
It's done
Analytics maturity

Descriptive Analytics
what happened? How do I compare?
Prescriptive Analytics
what should I do?
Predictive
what will happen?
Automated
it’s done
Data
Diagnostic Analytics
why did it happen?
Ordered Data
Sector
Transformation
Awareness
Experimentation
Organisation
support
Organisational
transformation
Analytics without a national approach

Sector
Transformation
Awareness
Experimentation
Organisation
support
Organisational
transformation
Predictive Analytics
what will happen?
what should I do?
Automated
it’s done
Data
why did it happen?
Ordered Data
Standardised Data
Analytics with a national approach

Sector
Transformation
Awareness
Experimentation
Organisation
support
Organisational
transformation
Predictive Analytics
what will happen?
what should I do?
Automated
it’s done
Data
why did it happen?
Ordered Data
Standardised Data
Adaptive learning etc.
Recommendation engines
etc.
Predictive models,
Intervention management etc
Data exploration tools,
processes etc
Dashboards,
Benchmarking etc.
Data Warehouse, data
stores
Data connectors
Analytics with a national approach

Descriptive
Analytics
Predictive
Analytics
Prescriptive
Analytics
AutomatedDiagnostic
Analytics
Standardised
Data
Learning
Records
Warehouse
xAPI Plugins
Data
transformation
tools
Data and API
Standards
Jisc
Services
Other
Provider
Services
Basic
dashboards
Student App
Analytics Labs
Benchmarking
services
College
Analytics
Basic predictive
modelling and
intervention
management
Procurement
frameworks
Integration
tools
Services for
researchers
Pilot projects
Services for
researchers
Pilot projects
Institutional
Dashboards
Data
visualisation
tools
Data
exploration
tools
Advanced
predictive
modelling
Integrated
intervention
management
??? ???

- Sector Data used in mashups:
- NSS
- SCONUL
- LiDP
- HESA
- Open Access Reporting/Deposit,
- JUSP / IRUS
- IRUS
- IMD
- Altmetrics
- H index
- Impact Factor
- REF metrics
- Jisc Collections bands & Subscription
data
Library Labs: 6 teams,
33 participants drawn
from Libraries

Library Analytics
Library Labs
- BUT also analytics on institutional
data:
- e-resource usage by type &
department
- e-resource cost benchmarking
- EZProxy logs
- Loans
- Gate entries
- Acquisitions
- Counter reports
- Capita Decisions
- Journal Citation Reports

Library Analytics
Library Labs
Birkbeck,University of London
Sheffield Hallam University
University of Edinburgh
University of Warwick
The University of Manchester
University of Salford
Liverpool John Moores University
Newcastle University
Southampton Solent University
Anglia Ruskin University Library
University of South Wales
University of Nottingham
Brunel University London
Kingston University
Teesside University
Bodleain Libraries, University of Oxford
University of Wolverhampton
University of Leicester
University of Reading
Manchester Metropolitan University
University of Bath
De Montfort University

Library Analytics
- Mashing up Library data was difficult – SCONUL is not HESA
- Many different internal systems, comparative analytics difficult
- Proof of concept dashboards stimulating institutions (traffic lights)
- More interest and contributions to recipes at http://github.com/jiscdev/xapi-lib
- New verbs! Eduroam, presence
- Data Sharing Agreements and an experimental area in the Heidi Lab
- Scope for more librarians alongside planners on Jisc’s beta BI project

Community: Project Blog,
mailing list and network events
Blog: http://analytics.jiscinvolve.org
– over 30 blog posts
Mailing: analytics@jiscmail.ac.uk –
422 members (182 organisations)
8th Network Meeting ~600+
participants

http://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics
Code of Practice
http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Learning Analytics Service Architecture
Library Analytics Service

Learning analytics products and tools
Learning records warehouse – active
Data Explorer – basic visualisations
Student Unified Data Definition –
version 1.2.7 and examples major SRS
and validation too
VLE – xAPI recipe and plugins for
Blackboard and Moodle
Attendance tracking – xAPI recipe
(being piloted soon)
Student App – release 1 Dec 2016
Tribal Student Insights (10)
Open Learning Analytics Processor (4)
Further learning analytics product
pilots (tbc)

UDDValidatorTool
• Customer-side UDD validation (web-based, secure access)
• UDD data preparation tool for institutions
• Jisc will load the historical data (once validated)
• Covers current & future UDD - 1.2.7, 1.2.x, 1.3.0 etc
• Links directly to UDDGitHub site (dynamic updates)
• Agile approach to software functionality/ release
• V1.0 - hard validation (UDD structure, optional/ mandatory fields, field contents)
• Relational entities – integrity checks
• Soft validation - data quality and concentration/ coverage (working withTribal/ Unicon Marist)
• Focus on key fields for predictive modelling purposes, student app
• Gives control & flexibility to our members – rapidly quick data validation (Azure Cloud)

Implementations
Profile Aims Tools No Data Sources
Teaching and
research led
Universities
Student
retention
and success
Tribal student
insight/data
warehouse
7 VLE (Moodle and
Blackboard), student
records and attendance
Teaching and
research led
Universities
Success and
engagement
Student app 4 VLE (Moodle and
records
Teaching led
Universities
Student
retention
Open source
processors/data
warehouse
4 VLE (Moodle and
FE Colleges Student
retention
Tribal student
insight
2 VLE (Moodle), student

Getting on-board…
https://analytics.jiscinvolve.org/wp/on-boarding/

On-boarding Process
Stage 1: Orientation
Stage 2: Discovery
Stage 3: Culture and Organisation Setup
Stage 4: Data Integration
Stage 5: Implementation Planning
https://analytics.jiscinvolve.org/wp/on-boarding/

Stage 1: Orientation
Stage 1. Orientation
1. Sign up to the analytics mailing list
Evidence required:
A list of people in your institution signed up to the mailing list

2. Review the learning analytics blog post and relevant reports
Evidence required:
Notes on useful articles and posts you have found

3. Attend a Jisc webinar, network meeting or workshop
Evidence required:
Notes from attending a recent event


Stage 2: Discovery Readiness
Stage 2. Discovery
4. Decide on institutional aims for learning analytics
Evidence required: A prioritised list of your aims for learning analytics

5. Strategic alignment, senior management approval and you have a nominated
project lead
Evidence Required: Named sponsor from the senior management team, Named project lead and contact details, Named technical lead and contact
leaded, A list of members of your working/management group

6. Undertake the readiness assessment
Evidence required :A completed readiness assessment questionnaire with your commentary on the answers

7. Arrange a verification meeting with Jisc to discuss the outcomes and possible next
steps
Evidence required: Date of meeting, documentation to share and a list of people attending


Discovery readiness
Topic ID Question Commentary Response Score
Leadership 1 The institutional senior management
team is committed to using data to
make decisions
Please provide a commentary on you
response to each question where
appropriate
0 - Hardly or not at all
1 - To some extent
2 - To a great extent
Leadership 2 Our vice-chancellor / principal has
encouraged the institution to
investigate the potential of learning
analytics
1 - To some extent
Leadership 3 There is a named institutional
champion / lead for learning analytics
0 - No
2 - Yes
Vision 4 We have identified the key
performance indicators that we wish to
improve with the use of data
1 - To some extent
A supported review of institutional readiness
https://analytics.jiscinvolve.org/wp/on-boarding/step-6-readiness-assessment/

Stage 3: Culture and Organisation Setup
Stage 3. Culture and Organisation Setup
8. Start to address readiness recommendations
Evidence required: Action plan to address readiness recommendations

9. Legal and ethical policy considerations in hand
Evidence required: List of institutional policies relevant to learning analytics; Plan to update/create policies to cover
learning analytics

10. Decision on learning analytics products to pilot
Evidence required: A documented list of products with an agreed rational for choices

11. Data processing agreement signed
Evidence required: Signed Data Processing Agreement

12. Select student groups for the pilot and engage staff/students
Evidence required: List of student groups/cohorts and numbers of students involved


Stage 4: Data Integration
Stage 4. Data Integration
13. Undertake a data and systems audit 
14. Contact Jisc to start data integration 
15. Install and evaluate the VLE data plugin(s) on a test system at your
institution

16. Extract student data, transform to UDD and validate. 
17. Extract historical VLE (or other activity) data 
18. InstallVLE (or other activity) data plugin(s) on live system, activate for live
data upload to LRW

19.View uploaded LRW data using data explorer to check quality 

Stage 4: Data collection
About the student Activity data
TinCan
(xAPI)ETL

Stage 5: Implementation Planning
Stage 5. Implementation Planning
20: Move to implementation Stage
Evidence required: An implementation plan with agreed timescales


On-boarding Process
Data Visualisation
Dashboards
Ready to
implement
Ready to
implement

On-boarding – get started
Stage 1: Orientation – review/done
Stage 2: Discovery – mostly self-support
Stage 3: Culture and Organisation Setup – Jan 2017
Stage 4: Data Integration – slots from early 2017
Stage 5: Implementation Planning - slots from
early 2017

Further exploration…
https://www.jisc.ac.uk/rd/get-involved

Co-design challenges 2017
Explore our co-design challenges
Help steer our innovation work by exploring the next big ideas for technology in education and
research.

Data
driven
learning
gains
Next
generation
research
environment
Digital skills
for
research
Should we gather more data on
students, staff and buildings that
would allow us to deliver better
experiences?
We think it is time for a new type
of learning environment, but what
would this look like?
We think it is time for a new type
of learning environment, but what
would this look like?
What would a truly digital
apprenticeship look like?
Can we make better use of data to
improve learning, teaching and
student outcomes?
How do we equip researchers and
related staff with the skills they need
for the future of research?
The
intelligent
campus
The digital
apprentice
Next
generation
learning
environment

1
Discuss
emerging
challenges
2
Prioritise
ideas
3
Announce
successful
ideas
4
Report
progress
Identify
ideas
31st Oct – 24th Nov 4th Jan– 30th Jan 6th Feb Apr/May
Release 6 challenge
areas and invite Jisc
members and other
experts to discuss
Audience: managers,
consumers, some
leaders, other experts
Present ideas for
activities Jisc could
do and ask members
which they support
Audience: managers,
consumers, some
leaders
Release 6 challenge
members and other
experts to discuss
Audience: everyone
who followed the
challenge
Release 6 challenge
members and other
experts to discuss
Audience: everyone
who followed the
challenge

Contacts
Paul Bailey paul.bailey@jisc.ac.uk
Further Information:
http://www.analytics.jiscinvolve.org
Join: analytics@jiscmail.ac.uk

Jisc learning analytics update-nov2016

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to Jisc learning analytics update-nov2016

Similar to Jisc learning analytics update-nov2016 (20)

More from Paul Bailey

More from Paul Bailey (13)

Recently uploaded

Recently uploaded (20)

Jisc learning analytics update-nov2016

Editor's Notes