How can universities scale up learning analytics beyond small-scale pilots to seriously use data to improve student learning? This interactive workshop was designed to help you think this through for your institution. Universities are hard to change. Having good data and analytics is a good start, but is only one part of success. This session will provide tools and frameworks to help you analyse what else is needed, building on experiences of successful large-scale learning analytics activity at the Open University and the University of Technology, Sydney, and from the pan-European Learning Analytics Community Exchange project. Slides for a talk at Bett, London, 20 January 2016.

1. Scaling Up Learning Analytics
Dr Doug Clow
Institute of Educational Technology, The Open University, UK
@dougclow
dougclow.org
doug.clow@open.ac.uk
#laceproject

2. 2
2
CC-BY – You are free to:
copy, share, adapt, or re-mix;
photograph, film, or broadcast;
blog, live-blog, or post video of
this presentation provided that:
You attribute the work to its author and respect the rights and
licences associated with its components.
#laceproject

3. The LACE project
3
K12 Workplace
HEI Community-building through events &
communication channels/social media
(cross-disciplinary HEI, K12, Workplace)
 Technology transfer & best practice
 Organized 22 events, and contributed to 33
(tutorials, workshops, conferences, etc.
Bett, 20 January 2016
European support action aimed at integrating communities working on LA from schools,
workplace and universities

4. 4
LACE Reception & Annual Meeting
5.30–6.30pm
Sundown bar
Sign up for an invitation!

5. 5English oak Quercus robur
CC BY-SA Jean-Pol GRANDMONT https://commons.wikimedia.org/wiki/File:Quercus_robur_JPG_(d1).jpg

6. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

7. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

8. Learning analytics …
… the measurement, collection, analysis and
reporting of data about learners and their
contexts, for purposes of understanding and
optimizing learning and the environments in which
it occurs.
8
solaresearch.org

9. Cloud Chamber at the German Electron Synchrotron DESY
Photo public domain: http://commons.wikimedia.org/wiki/File:DESYNebelkammer.jpg
- Erik Duval
http://erikduval.wordpress.com/2012/01/30/learning-
analytics-and-educational-data-mining/
“collecting traces
that learners leave
behind and using
those traces to
improve learning”

10. Clow, LAK12, 2012

11. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

12. The Open University (OU)
12Jennie Lee Building, The Open University
Photo CC BY-NC Bea de los Arcos https://flic.kr/p/pR2xWF
To use and apply
information strategically
to retain students and
enable them to progress
and achieve their study
goals.

13. University of Technology, Sydney (UTS)
13Dr Chau Chak Wing Building, UTS, by Frank Gehry
Photo CC BY-SA Brickworks Building Products https://flic.kr/p/rt7Rxx
A university where staff
and students understand
data and, regardless of its
volume and diversity, can
use and reuse it, store and
curate it, apply and
develop the analytical
tools to interpret it.

14. What’s your goal for scaling up
learning analytics?
What’s your policy objective?
Scaling up learning analytics: Discuss with your neighbour

15. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

16. The Rapid Outcomes Modelling Approach (ROMA)
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in context:
overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.
Adapted from: Young, J., & Mendizabal, E. (2009). Helping researchers become policy entrepreneurs: How to develop
engagement strategies for evidence‐based policy‐making. ODI Briefing Papers. London, UK: ODI.
Define (and
redefine) your
policy
objectives

17. Political context (UTS)
17
Lead: Deputy VC & VP T&L
Pilot projects
Advanced Analytics Institute
Connected Intelligence Centre
UTS Building 11
Photo CC BY extramaster https://flic.kr/p/xXJLbx

18. Political context & Key stakeholders (OU)
18
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

19. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

20. Apply the
framework
to your
situation
Scaling up learning analytics: Discuss with your neighbour
Define (and
redefine) your
objectives

21. 1. What is learning analytics?
2. What is your goal?
3. ROMA framework
4. Apply the framework
5. Next steps
Scaling up learning analytics

22. resources

23. 23
• Current state of play in UK HE & FE
• Code of practice for learning analytics
• Network meeting, London, soon
http://analytics.jiscinvolve.org/
Effective Learning Analytics

24. • LAK conferences
• LASI workshops
• Flare local meetings
• Storm PhD training
• Journal of LA
• … and more!
www.solaresearch.org

25. www.laceproject.eu
• Blog bit.ly/lace-blog
• Newsletter bit.ly/lace-newsletter
• Learning Analytics Review
bit.ly/lace-review-papers
• Become an Associate Partner!

26. 26
LACE Reception & Annual Meeting
5.30–6.30pm
Sundown bar
Sign up for an invitation!

27. What’s your next step?
Scaling up learning analytics: Discuss with your neighbour

28. 28
Organisations are built to resist change
• Rock and the river?
• Big lever
Vernal Falls, Yosemite
Photo CC BY Mary PK Burns https://flic.kr/p/6zUzqh

29. Thanks to:
People:
• LACE at the OU: Rebecca Ferguson, Andrew Brasher,
Bart Rientes, Simon Cross, Linda Norwood Michelle
Bailey, Rebecca Wilson, Evaghn De Souza, Natalie
Eggleston, Oliver Millard, Gary Elliot-Citigottis.
• LACE project partners: CETIS (Bolton), OUNL,
Skolverket, HIOA, Kennisnet, ITS, ATiT.
• The learning analytics community, including SoLAR,
IEDMS, those I’ve met at LAK and LASI
• Bett and venue staff
Funders:
• LACE: European Commission 619424-FP7-ICT-2013-11

30. “Scaling Up Learning Analytics” by Doug Clow,
Institute of Educational Technology, The Open University,
was presented at Bett, London, on 20 January 2016.
@dougclow
dougclow.org
doug.clow@open.ac.uk
This work was undertaken as part of the LACE Project, supported by the European Commission Seventh
Framework Programme, grant 619424.
These slides are provided under the Creative Commons Attribution Licence:
http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.
www.laceproject.eu
@laceproject
30

31. cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/

32. Policy objectives
OU Strategic Analytics Investment Programme
32
Vision
To use and apply information
strategically to retain students and
enable them to progress and achieve
their study goals.
This vision requires
• Discursive changes
to the communication of data and
analytics
• Procedural changes
in how learners are supported
• Behavioural changes
associated with sustainable change
in learner support.
Define (and
redefine)
your policy
objectives

33. Political context
Mapping people and processes
33
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

34. Key stakeholders
OU Strategic Analytics Investment Programme
34
Define
(and redefine)
your policy
objectives
A community of stakeholders working
in different areas:
• Intervention and Evaluation
• Data Usability
• Ethics Framework
• Predictive Modelling
• Learning Experience Data
• Professional Data
• Student Tools
Key stakeholders are
• University administrators
• Students
• Educators

35. Desired behaviour changes
OU Strategic Analytics Investment Programme
35
Define
(and redefine)
your policy
objectives
Vision
To use and apply information
strategically to retain students and
enable them to progress and achieve
their study goals.
Desired behaviour changes
• Staff will use and apply information
strategically
• Students will extend their learning
journeys
• Students will complete their
learning journeys
• Students will set learning goals
• Students will work effectively
towards study goals

36. Engagement strategy
OU Strategic Analytics Investment Programme
36
Define
(and redefine)
your policy
objectives
• Data in action is provided to
stakeholders through a live portal,
enabling them to understand learner
behaviour and make adjustments and
interventions that will have an
immediate positive impact.
• Data on action is a more reflective
process that takes place after an
adjustment or intervention.
• Data for action takes advantage of
predictive modelling and innovation in
order to isolate particular variables and
make changes based on a variety of
analysis tools.

37. Internal capacity to effect change
OU Strategic Analytics Investment Programme
37
Define
(and redefine)
your policy
objectives
Includes
• Recruitment
• Capacity building
• Developing an ethical framework
for the use of learning analytics.

38. Monitoring
OU Strategic Analytics Investment Programme
38
Tynan, B. & Buckingham Shum, S. (2013). Designing systemic learning analytics at the Open University.
http://www.slideshare.net/sbs/designing‐systemic‐learning‐analytics‐at‐the‐open‐university

39. Policy objectives
University of Technology, Sydney, Australia
39
Vision
A university where staff and students understand data and, regardless of its
volume and diversity, can use and reuse it, store and curate it, apply and develop
the analytical tools to interpret it.
Teaching and learning
Ensure that all stakeholders have the capacity to understand and interpret
contemporary data‐rich environments.
Research
Enable researchers to think and act differently when designing their research
methodologies and practices.
Administration
Identify opportunities to obtain, generate, visualize, and
communicate data and analyses that can improve
core business outcomes.
University
Mine existing institutional data to identify areas that can provide
direct evidence or assistance to staff and students.

40. Political context
University of Technology, Sydney, Australia
40
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S., & Dawson, S.. (2015). Setting learning analytics in context:
overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3), 120-144.
• Project initiated and led by Deputy Vice‐Chancellor and
Vice‐President (Teaching and Learning)
• Pilot projects were completed successfully to secure ongoing
funding.
• Critical to the success of the initial pilot projects was the
existence of an Advanced Analytics Institute with
internationally regarded researchers in big data, data sciences
and analytics sciences.
• This enabled the establishment of a Connected Intelligence
Centre.

41. Key stakeholders
University of Technology, Sydney, Australia
41
Define
(and redefine)
your policy
objectives
• 190 staff attended a one‐day ‘Data
Intensive University Forum’, thus
beginning a university‐wide
conversation.
• Working party included Deputy
Vice‐Chancellors, a senior member
of library staff, and representatives
of all faculties and administrative
areas with relevant expertise
• Stakeholder buy‐in and ongoing
participation in the project have
been critical to its success.

42. Desired behaviour changes
University of Technology, Sydney, Australia
42
Define
(and redefine)
your policy
objectives
• Provide information that can be used
to decrease student attrition
• Provide a more detailed understanding
of factors affecting low pass rates in
subjects with very high failure rates
over time
• Provide students with more
information about their own study and
engagement patterns
• Enable a more fine‐grained
understanding of the influences of a
range of possible interventions on pass
rates and completions
• Provide valuable input to learning
futures projects

43. Engagement strategy
University of Technology, Sydney, Australia
43
Define
(and redefine)
your policy
objectives
• Give attention to institutional culture,
ensuring engagement and buy‐in from
key stakeholders through good
communication and governance
• Invest in pilot projects of significant
concern to the university and reporting
of outcomes
• Invest in infrastructure: tools,
applications, services
• Invest in expertise: recruitment of
critical staff
• Provide leadership and engage
institutional leaders

44. Internal capacity to effect change
University of Technology, Sydney, Australia
44
Define
(and redefine)
your policy
objectives
• Students and staff must be sufficiently
numerate and equipped to make use of
the analyses that analytics projects
produce.
• A subject has been developed and
trialled with staff
• The course develops students’ ability to
engage with complex, extended
arguments underpinned by numerical
data as a key to participation as
informed citizens in issues of
significance to our culture and society
• The course available as an elective and
will become compulsory

45. Monitoring
University of Technology, Sydney, Australia
45
• UTS has been engaged in a variety
of learning analytics projects to
assess scale and impact
• For example, the Outreach
Programme rings as many
commencing undergraduate
students as possible. Early results
consistently show a significant
decrease in attrition in the group of
students contacted.
Define
(and redefine)
your policy
objectives