A new algorithm to analyse social semantic appropriation of conceptual knowledge.

1. 1
Learning from Meaningful,
Purposive Interaction
Fridolin Wild · Medieninformatik · Universität Regensburg ·
Knowledge Media Institute · The Open University
Representing and analysing competence
development with network analysis and
natural language processing

2. 2
Outline
 Introduction and overview
 Theoretical foundation
 Precursor algorithms (SNA + LSA)
 Algorithm: Meaningful, Purposive Interaction Analysis
• Mathematical foundation
• Visual analytics using vector maps as projection surfaces
• Implementation
 Application examples for Learning Analytics
 Evaluation: verification and validation
 Summary and Outlook

3. 3
INTRODUCTION
Context, Objectives, Key Contributions

4. 4
Introduction
 Fascination with LSA and Matrix Algebra originated
in Information Retrieval (UR), then shifted to
Technology Enhanced Learning (WU+OU)
 Research on Technology Enhanced Learning has
its place in the canon of Media & Computing (and
Knowledge Media)
 It’s a big and growing global Software Market:
• Adkins (2011, p.6): 9.2% annual growth till 2015
• Docebo (2014,p.8): 7.9% annual growth till 2016
 Drivers of Innovation: open Grand Challenges
to Research and Development in TEL

5. 5
Bridging informal
and formal
Create a unified, seamless
learning landscape with the
help of mobile devices.
learning
analytics
automated feedback using
interaction data to predict
performance.
#6
fostering
engage-
ment
Increasing student motivation to
learn and engaging the
disengaged – using technology.
How can we detect (de-)
motivation? How can make
use intrinsic/extrinsic reward
systems?
#4
New devices for
young children’s
expression of
scientific ideas
Mouse and keyboard are a
blocker to natural mapping and
new modalities of interaction
(touch, gestures) can foster a
more tactile learning.
#1
Learning to
read at home
with digital
technologies
#2
CSCL in teacher training
and professional
development
#3
e-assessment
New forms of assessment of
learning in social TEL
environments
#5
Understanding how toddler apps
can support learning.
early years
technology
dataTEL
Utilising real-time data to
improve teaching and
learning.
#7
#8
networked
learning
ecologies
Interest-driven lifelong learning
in personal learning networks
#9
#10
Fischer,Wild,
Sutherland,Zirn
(2014)
#1

6. 6
Objectives for this work (from GC #5,#6,#8)
Represent: to automatically represent
conceptual development evident from
interaction of learners with more
knowledgeable others and resourceful content
artefacts;
Analyse: to provide the instruments required
for further analysis;
Visualise: to re-represent this back to the
users in order to provide guidance and support
decision- making about and during learning.

7. 7
Key Contributions
(Wild, 2014, p.21)

8. 8
THEORETICAL FOUNDATION
Concept space, Quality requirements

9. 9
Information and Learning
communicatively
successful
cooperatively
successful
[e]=
PmO
purpose
meaning
(Janich, 1998/2003/2006; Hesse et al., 2009;
Hammwoehner, 2005; Wild, 2014, p.27ff)
“learns
‘information’”

10. 10
Information and Learning
(Wild, 2014, p.42)

11. 11
PRECURSOR ALGORITHMS
Foundational examples, Shortcomings

12. 12
The Foundational Example
 Particular unit of company with 9 employees
 All went through trainings recently
 Offered by universities (UR, OU), MOOCs, informal
learning (FaceBook, LinkedIn)
 Now: Christina is off sick
 HR manager to identify worthy replacement
• SNA
• LSA
• MPIA
(Wild, 2014, p.60)

13. 13
(Wild, 2014, p.21,61,63)
Social Network
Analysis (SNA)
Foundational
Example
A =

14. 14
SNA
Paul
Joanna
Maximilian
Peter
Christina
Simon
Ida
Thomas
Alba
Association Matrix

15. 15
(Wild, 2014, pp.99-102)
Latent Semantic Analysis

16. 16
c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5
c1c2c3c4c5m1m2m3m4p1p2p3p4p5
c1c2c3c4c5m1m2m3m4p1p2p3p4p5
original
space
LSA &
‘Similarity’
(Wild, 2014, p.104: cosines)
(Wild, 2014, pp.229)
black = 1, white = 0
c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5
c1c2c3c4c5m1m2m3m4p1p2p3p4p5
c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5
c1c2c3c4c5m1m2m3m4p1p2p3p4p5
LSA space

17. 17
Shortcomings
 Social Network Analysis (SNA)
• Blindness to content
• Relationship discovery restricted to incidences captured
• Popular for analysis, visualization, simulation, intervention
(Sie et al., 2012)
 Latent Semantic Analysis (LSA)
• Blindness to purposes & structure (relations, groups, …)
• Lacking instruments for analysis
• No clear rule for number of factors to retain
• Popular for essay scoring, information retrieval, dialogue
tutoring, recommenders

18. 18
MEANINGFUL PURPOSIVE
INTERACTION ANALYSIS
Foundations in Matrix Algebra, Stretch Truncation

19. 19
Fundamental
matrix theorem
on orthogonality
Calculating the
Nullspace Ker A:
Ax = 0 Eq.1
(Wild, 2014, p.131; redrawn
from Strang, 1988, p.140)
(Wild, 2014, p.132)
“every matrix transforms
its row space to its
column space” (Strang,
1988, p.140)

20. 20
The Eigenvalue Problem &
Singular Value Decomposition
(Wild,2014,p.143)
For every symmetric, square matrix:
(Barth et al., 1998, p.90/E):
Bx = λx
n.b.: B = AAT or ATA
Any multiplication of the matrix B with an
Eigenvector x yields a constant multiple of the
Eigenvector, scaled by the Eigenvalue λ
A = UΣVT
U = Eigenvectors(ATA)
V = Eigenvectors(AAT)
Σ = UTAV

21. 21
Base transformation (from Term-Doc space
to orthogonal Eigenspace)
(Wild, 2014, p.144)
Same dims for both
Eigenvector types (row
and column), same
Eigenvalues!

22. 22
Stretch-Dependent
Truncation
0
100
200
300
400
0 20 40 60
index
eigenvalues
20%80%
0
100
200
300
400
0 20 40 60
index
eigenvalues

23. 23
Prediction of Threshold
Sum of Eigenvalues Σ2 = Sum of trace of matrix A
threshold = 0.8 * sum(A*A)
=> Calculate only the first k Eigendimensions, for which the
sum of Eigenvalues Σ2 does not yet pass the threshold

24. 24
Updating using ex post projection
v' = aT Uk Σ k
-1
a' = Uk Σk v' T
(Wild, 2014, p. 149f; see also Berry
et al., 1994, equation 7 and page 16

25. 25
Point, Centroid, Pathway
e1
e2
u2
u3
u1
p
1
1 3 2
Proximity, Identity
1
11
16
25
9
14
24
2
18
3
19
4
13
5
6
10
15
7
20
22
12
17
26
8
21
23
01234
Cluster dendrogram over all meaningvectors
Meaningvectors
Height
cutoff
Clustering

26. 26
Introducing Network Analysis Techniques
 Still: result is high-volume,
sometimes even big data
 Visualisation techniques
from (Social) Network Analysis
can help!

27. 27
In real examples:
too high-volume to
see structure

28. 28
Network
Visualisation
Proximity-driven
Link Erosion
(Wild, 2014,
p.162)
Layout with
spring-embedder
(Wild, 2014,
p.163)
Wireframe
Conversion (Wild,
2014, p.167)
Kernel Smoothing
(Wild, 2014,
p.169)
Hyposometric Tints
(Wild, 2014, p.171)

29. 29
Perspective plot
(Wild, 2014, p.172)

30. 30
Topographic map projection and overlays
(Wild, 2014, p.173ff)

31. 31
IMPLEMENTATION
Use Cases, Analysis Workflow, Classes, Demo
31

32. 32
Use Cases

33. 33
Class Diagram
(Wild, 2014, p.209)
> 10.000 lines of codeR package
MPIA
To be: Open Source
(GPL-3)
Test-driven
development

34. 34
APPLICATION EXAMPLES
Concept space, Quality requirements

35. 35
The SNA/LSA
example revisited
(Wild, 2014, p.231)
C = computer science
P = pedagogy
M = math + stats

36. 36
MPIA foundational example
(path of Peter)
interface
socialweb
access
review
system
timeusage
html
management
trees
clustering
intersection
agglomerative
knowledge
learning
organisational
system
html
usage
c3
learning
knowledge
p2
system
html
social
c4
clustering
trees
agglomerative
m2

37. 37
c3, c4
p2
m2
Competences extracted

38. 38
Example 2: Essay scoring

39. 39
Essays
Collection 1: Programming:
define ‘information hiding’

40. 40
EVALUATIONS
Concept space, Quality requirements

41. 41
Evaluations
The role of verification and validation
(Schlesinger, 1979, as cited in
Oberkampf & Roy, 2010, p.23)
(Wild, 2014, p.276)

42. 42
Verification Results
+ 22 examples in the documentation
(tested by the documentation checker)
[...]
* this is package ‘mpia’ version ‘0.60’
[...]
* checking examples ... OK
* checking for unstated dependencies in tests ... OK
* checking tests ...
Running ‘tests.R’
OK
[...]

43. 43
Validation Experiments
 No standardised test collections
for conceptual development
 Effectiveness:
• Accuracy in application (Essay Scoring)
• Convergent and divergent validity
• Annotation accuracy
• Degree of loss in the visualisation
 Efficiency:
• Performance gain

44. 44
Evaluation of
Scoring
Accuracy
Example of feedback
Using holistic scoring
(essay = avg. ~ of 3
model solutions)

45. 45
Performance Gains
Savings in calculation time through using
the threshold prediction method for SVD
calculation truncation (predicted from
original doc-term matrix)

46. 46
CONCLUSION
Revisiting Objectives, Summary, Outlook

47. 47
Innovation in TEL
Three Grand Challenges (Fischer et
al., 2014) addressed:
• “new forms of assessment for social
TEL environments” (Whitelock,
2014a)
• “assessment and automated
feedback” (Whitelock, 2014b)
• “making use and sense of data for
improving teaching and learning”
(Plesch et al., 2012)
47
learning
analytics
automated feedback using
interaction data to predict
performance.
#6
e-assessment
New forms of assessment of
learning in social TEL
environments
#5
dataTEL
Utilising real-time data to
improve teaching and
learning.
#8

48. 48
Summary

49. 49
The END