Interactive session run at Open U, CALRG2011 Conference, 14 June 2011

1. CALRG 2011
Social Learning Analytics
Simon Buckingham Shum & Rebecca Ferguson
Knowledge Media Institute & Institute of Educational Technology
The Open University, Milton Keynes, UK
@sbskmi / @R3beccaF
1

2. How we’re going to do this...
10mins Imagine.../background/critical questions
– Simon
15mins A taxonomy of Social Learning Analytics
– Rebecca
10mins SLA: more than just a bunch of techniques
– Simon
15mins Open discussion...
2

3. Coming soon to a future near you?...
Analytics Report
Application from Ali Bloggs to study Z0001
This applicant has a high risk profile:
1. No academic study for last 15 years
2. Low socio-economic background
3. English as a second language
4. Weak ICT skills
5. His responses to the learning styles survey indicate a loner,
rather than a collaborative learner, known to be a
disadvantage on this course
[click to view the 3 other risk factors]
Without a Grade 3 tutor (advanced skills in 1-1 support), based
on the last 5 years data there is a 37% chance of dropping out
by Week 6.
[ACCEPT] [REJECT]
3

4. Coming soon to a future near you?...
“Hi Ann,
In the last 2 weeks, it looks like you’ve been really stretching
yourself. You seem to have been working on your critical
thinking, with that challenge to Mike’s assumption, and the
evidence-based claim about nuclear waste in your blog.
Check out Donna Winter, who seems to have very different views
to yours on global warming. How would you assess her position?
In your next video conference tutorial, try to improve on the last
three, in which you seem to have contributed only once each
time.”
4

5. Coming soon to a future near you?...
“Did you know that two other people you know have used
the Smith & Jones 2009 framework graphic?
Finally, you seem to have really become a pivotal member
of the Local-Global Climate Network. Good work: only a
month ago you were on the edge!
Why not reflect in your blog on how these groups are
helping you in your long term goal to Work for the UN in
Africa?”
5

6. L. Johnson, R. Smith, H. Willis, A. Levine, and K. Haywood, The 2011 Horizon Report (Austin, TX: The New
Media Consortium, 2011), http://www.nmc.org/pdf/2011-Horizon-Report.pdf 6

7. Learning analytics
“Learning Analytics is
concerned with the
collection, analysis and
reporting of data about
learning in a range of
contexts, including
informal learning,
academic institutions,
and the workplace.
It informs and provides
input for action to
support and enhance
learning experiences, and
the success of learners.”
2nd Int. Conf. Learning Analytics & Knowledge 2012
dougclow.wordpress.com

8. “Academic Analytics”
• Stage 1—Extraction and reporting of
transaction-level data
• Stage 2—Analysis and monitoring of
operational performance
• Stage 3—What-if decision support “Academic analytics can be
(such as scenario building)
thought of as an engine to
• Stage 4—Predictive modeling and make decisions or guide
simulation actions. That engine consists
of five steps: capture, report,
• Stage 5—Automatic triggers of predict, act, and refine.”
business processes (such as alerts)
Goldstein, P. J. (2005). Academic Analytics: The Uses of Management “Administrative units, such
Information and Technology in Higher Education: Key Findings.
Boulder, Colorado: Educause Center for Applied Research
as admissions and fund
http://net.educause.edu/ir/library/pdf/EKF/EKF0508.pdf raising, remain the most
common users of analytics
in higher education today.”
Campbell, J. P. & Oblinger, D.G. (2007) Academic Analytics.
EDUCAUSE http://connect.educause.edu/Library/Abstract/
AcademicAnalytics/45275
8

9. Academic analytics in English schools
9

10. OU Analytics service: Predictive
modelling
§ Probability models help us to identify patterns of
success that vary between:
§ student groups
§ areas of curriculum
§ study methods
§ Previous OU study data – quantity and results – are the
best predictors of future success
§ The results provide a more robust comparison of
module pass rates and support the OU in identifying
aspects of good performance that can be shared and
aspects where improvement could be realised
OU Student Statistics & Surveys Team, Institute of Educational Technology 10

11. Purdue University Signals
http://www.itap.purdue.edu/studio/signals Purdue's premise: academic success is defined as a
function of aptitude (as measured by standardized test
scores and similar information) and effort (as measured by
participation within the CMS).
Using factor analysis and logistic regression, a model was
programmed to predict student success based on:
• ACT or SAT score
• Overall grade-point average
• CMS usage composite
• CMS assessment composite
• CMS assignment composite
• CMS calendar composite
Campbell et al (2007). Academic Analytics: A New Tool for a New
Era, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40–57.
http://bit.ly/lmxG2x
11

12. critical questions
12

13. Pause for thought...
§ in the discourse of academic analytics, there is little
mention of pedagogy, theory, learning or teaching
§ what models of “learning” currently underpin
analytics? If we can’t log and measure it, it’s invisible...
§ what learning phenomena should analytics track to
equip learners for the complexities of C21?
§ classification schemes are the mechanisms by which
we choose not only how to remember, but also
systematically forget (Bowker and Star, 1999)
§ power: who is defining the measures, to what ends,
and who gets to see which results? 13

14. social learning analytics
a taxonomy
14

15. Social Learning Analytics
• Social learning network analysis
• Social learning discourse analysis
• Social learning content analysis
• Social learning dispositions analysis
• Social learning context analysis

16. Social network analytics
• Networked learning uses
ICT to promote connections
• Networks consist of actors
(people and resources) and
the ties between them.
• Ties can be classified by
their frequency, quality or
importance

17. SNAPP
• Trace the growth of course communities
• Identify disconnected students
• Highlight the role of information brokers

18. GEPHI
Tony Hirst
blog.ouseful.info
• Networks with interconnected interests
• Interests that are shared by actors in a network
• Role of information brokers in sharing resources,
• Roles played by resources in connecting networks

19. Social network analysis
and social learning
• Identify and support types of interaction
that promote the learning process
• Identify interventions that are likely to
increase the potential of a network to
support the learning of its actors

20. Social learning discourse analytics
• Educational success and failure may be explained
by the quality of educational dialogue, rather than
simply in terms of the capability of individual
students or the skill of their teachers
• The ways in which learners engage in dialogue
are indicators of how they engage with other
learners’ ideas, how they compare those ideas
with their personal understanding, and how they
account for their own point of view

21. Cohere
• Annotations or
discussion as a
network of
rhetorical moves
• Users must reflect
on, and make
explicit, the nature
of their contribution
Simon Buckingham Shum, Anna De Liddo

22. Exploratory dialogue
Rebecca Ferguson

23. Open Mentor
Denise Whitelock
Analyse, visualise and compare quality of feedback

24. Content analytics
Automated methods to examine, index
and filter online media assets, with the
intention of guiding learners through
the ocean of available resources

25. LOCOanalyst
Provides feedback for content authors and teachers that can help
them to improve their online courses (Jovanovic et al., 2008)

26. Visual search
Suzanne
Little
Visual similarity search uses features of images such as colour,
texture and shape in order to find material that is visually related

27. iSpot
Social content analytics draw upon the tags, ratings
and additional data supplied by learners

28. Social learning dispositions analytics
• Learning dispositions provide a way of identifying and
naming the qualities of a good learner.
• They comprise the seven dimensions of ‘learning power’:
changing & learning, critical curiosity, meaning making,
dependence & fragility, creativity, relationships/
interdependence and strategic awareness
• Dynamic assessment of learning power can be used to
reflect back to learners what they say about themselves
in relation to these dimensions
Ruth Deakin Crick, University of Bristol

29. ELLI
• Effective Lifelong Learning Inventory (ELLI) responses produce a learning
profile
• This profile forms the basis for a mentored discussion with the potential
to spark and encourage changes in the learner’s activities, attitude and
approach to learning

30. ELLIment
Thomas Ullmann: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent

31. EnquiryBlogger
Rebecca Ferguson, Simon Buckingham Shum, Ruth Deakin Crick
http://learningemergence.net/tools/enquiryblogger

32. Social learning context analytics
Taking context into account:
• Formal settings
• Informal settings
• Mobile learning
• Synchronous environments
• Asynchronous environments

33. Identifying and using context
My OU Story:
Liam Green Hughes
Stuart Brown
Tony Hirst

34. Affinity groups

36. reflections...
36

37. Tectonic shifts in the learning landscape...
TECH: online,
personalised, real
time, multimedia, mobile... Taken together, these are
profound shifts in power,
FREE/OPEN: expected initially: I’ll
relationships,
pay if it’s good enough
economics...
SOCIAL LEARNING: innovation now
depends on it
VALUES:autonomy, diversity, self-
expression, participation if these reshape
POST-INDUSTRIAL: new institutional
our conception of the
roles in post-industrial education future of learning
system – do they not also
reshape our conception of the
future of learning analytics?
37

38. Tectonic shifts in the learning landscape...
The emerging “2.0”
landscapes for learning,
e.g. social
scholarship and
capital, critical
knowledge work demand
thinking,
new, more meaningful
citizenship,
indicators than
habits of mind,
conventional BI/MIS
resilience,
collaboration
skills, creativity,
emotional
intelligence… 38

39. SLA: it’s not just what they do (taxonomy)
but how we use them (credibility/integrity)
Analytics should step
beyond the C20 business Beyond a tool for
intelligence mindset institutions to track
(cf. C21 “pervasive BI”) learners, these are tools
to place in the hands of
those being tracked
Concerns about the abuse of SLA are about helping
analytics may rest on the old people to grow as
power configuration of an learners through
institutionally wielded personal + collective
instrument, to gather formative feedback
summative data
39

40. Some links...
Learning Analytics blog, resources & open course
http://www.learninganalytics.net
2nd Int. Conf. Learning Analytics, Vancouver, Apr 2012
http://lak12.sites.olt.ubc.ca
KMi Learning Analytics R&D
http://people.kmi.open.ac.uk/sbs/tag/learning-analytics
40