In this talk I will introduce the emerging technology of Open Social Student Modeling (OSSM) and review several projects performed in our research lab to investigate the potential of OSSM. OSSM is a recent extension of Open Student Modeling (OSM), a popular technology in the area of personalized learning systems. While in traditional personalized systems, student models were hidden “under the hood” and used to personalize the educational process; open student modeling introduced the ability to view and modify the state of students’ own knowledge to support reflection, selforganized learning, and system transparency. Open Social Student Modeling takes this idea one step further by allowing students to explore each other’s models or an aggregated model of the class. The idea to make OSM social was originally suggested and explored by Bull [1; 2]. Over the last few years, our team explored several approaches to present OSSM in a highly visual form and evaluated these approaches in a sequence of classroom and lab studies. I will present a summary of this work introducing such systems as QuizMap [3], Progressor [4], and Mastery Grids [5] and reviewing most interesting research evidence collected by the studies.

1. Intelligent Interfaces for
Open Social Student Modeling
Peter Brusilovsky
Sharon Hsiao,Tomek Loboda, Julio
Guerra, Jordan Barria-Pineda
PAWS Lab,
University of Pittsburgh

2. Overview
• Goals
– Why we are doing it?
• Open Student Models
– From ANS to OSM
• Open Social Student Models
– QuizMap, Progressor, Progressor+
• Mastery Grids
– Topic-level OSLM in Mastery Grids
– Concept-level OSLM in Mastery Grids

3. From Goals to Technologies
• Technologies
–Adaptive Navigation Support
–Open Student Models
–Open Social Student Modeling
• Why to use it
–Increase user performance
–Increase motivation and retention

4. Targets Engaged
• Adaptive Navigation Support
• Topic-based Adaptation
• Open Student Modeling
• Social Navigation and Comparison
• Open Social Student Modeling
• Social Educational Progress Visualization
• Multiple Content Types
• Open Source
• Concept-Based Adaptation

5. Adaptive Link Annotation: InterBook
1. Concept role
2. Current concept state
3. Current section state
4. Linked sections state
4
3
2
1
√

6. Questions of
the current
quiz, served
by QuizPACK
List of annotated
links to all quizzes
available for a
student in the
current course
Refresh
and help
icons
QuizGuide = Topic-Based ANS

7. Topic-Based Adaptation
Concept
A
Concept
B
Concept
C
 Each topic is associated with a number of
educational activities to learn about this topic
 Each activity classified under 1 topic

8. QuizGuide: Adaptive Annotations
• Target-arrow abstraction:
– Number of arrows – level of
knowledge for the specific
topic (from 0 to 3).
Individual, event-based
adaptation.
– Color Intensity – learning
goal (current, prerequisite
for current, not-relevant,
not-ready). Group, time-
based adaptation.
 Topic–quiz organization:

9. QuizGuide: Success Rate

10. QuizGuide: Motivation
Average activity
0
50
100
150
200
250
300
2002 2003 2004
Average num. of
sessions
0
5
10
15
20
2002 2003 2004
Average course
coverage
0%
10%
20%
30%
40%
50%
60%
2002 2003 2004
 Within the same class QuizGuide session were much
longer than QuizPACK sessions: 24 vs. 14 question
attempts at average.
 Average Knowledge Gain for the class rose from 5.1 to 6.5

11. • Topic-Based interface organization is
familiar, matches the course
organization, and provides a
compromise between too-much and
too-little
• Two-way adaptive navigation
support guides to the right topic
• Open student model provides clear
overview of the progress
Topic-Based ANS: Success Recipes

12. Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
• Social Navigation and Comparison
• Open Social Student Modeling
• Social Educational Progress Visualization
• Multiple Content Types
• Open Source
• Concept-Based Adaptation

13. Social Navigation
• Concept-based and topic-based navigation support
work well to increase success and motivation
• Knowledge-based approaches require some
knowledge engineering – concept/topic models,
prerequisites, time schedule
• In our past work we learned that social navigation –
“wisdom” extracted from the work of a community
of learners – might replace knowledge-based
guidance
• Social wisdom vs. knowledge engineering

14. Knowledge Sea – Social Navigation
Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling.
10th International User Modeling Conference Lecture Notes in Artificial Intelligence, vol. 3538. Berlin: Springer Ve

15. Open Social Student Modeling
• Motivation
– Combine benefits of Open Student Models with social navigation
and social comparisons
• Key steps
– Assume simple topic-based design
– Show topic- and content- level knowledge progress of a student
in contrast to the progress of the class
– The design should guide students to most appropriate topics and
content
• Main challenge
– How to design the interface to show student and class progress
over topics?
– We went through several attempts…

16. QuizMap
16

17. Parallel Introspective Views
17

18. Progressor
18

19. • Topic organization should follow the
natural progress or topics in the
course
• Clear comparison between “me” and
“group”
• Ability to compare with individual
peers, not only the group
• Privacy management
OSLM: Success Recipes

20. The Value of OSLM
205.73
113.05
80.81
125.5
0
50
100
150
200
250
Attempts
Progressor
QuizJET+IV
QuizJET+Portal
JavaGuide
68.39%
71.35%
42.63%
58.31%
0.00%
20.00%
40.00%
60.00%
80.00%
Success Rate
Progressor
QuizJET+IV
QuizJET+Portal
JavaGuide

21. The Mechanism of Social Guidance
stronger students left the traces for weaker ones to
follow
21Time
Topics





















22. The Secret

23. Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Comparison
Open Social Student Modeling
Social Educational Progress Visualization
• Multiple Content Types
• Open Source
• Concept-Based Adaptation

24. Progressor+ OSLM for two types of content
• macro- and micro- comparisons (group or peers)
24

25. Students Spent More Time in Progressor+
Quiz =: 5 hours
Example : 5 hours 20 mins25
60.04
150.19
224.7
296.9
69.52
121.23
110.66
321.1
0
50
100
150
200
250
300
350
400
QuizJET JavaGuide Progressor Progressor+
Total time spent (minutes)
Quiz
Example

26. Students Achieved Higher Success Rate
26
42.63%
58.31%
68.39%
71.20%
0.00%
20.00%
40.00%
60.00%
80.00%
QuizJET JavaGuide Progressor Progressor+
Success Rate
p<.01

27. Mastery Grids
27

28. Mastery Grids: Content Access
28

29. Mastery Grids: Group and Peer OSLM
29

30. MG flexibility
• Parameters to set the visualization:
– show hide toolbar or any of its elements
– set the (sub) groups: top N, other sub groups
– preset values (for example load individual view by
default)
– enable/disable recommendation
• Parameters can be specified by group or
by user

31. Mastery Grids Engage More
31
0
10
20
30
40
50
60
Problems Solved
0
5
10
15
20
25
30
35
40
45
50
Examples Viewed
And social comparison (OSSM) features
strengthen the effect

32. OSSM Engages Persistently
32
10
15
20
25
30
PART 1 PART 2
Activity by Session
OSM OSSM
Step-wise
regression:
being in the
OSSM group
means an
increase of
about 30
activities, as
compared to
being in the
OSM group.

33. OSSM Group Becomes More Effective
• Instructional Effectiveness (Paas & Van Merriënboer, 1993)
Relates performance in problems and time spent
33
-0.4
-0.2
0
0.2
PART 1 PART 2
Effectiveness Score
OSSM
OSM

34. Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Comparison
Open Social Student Modeling
Social Educational Progress Visualization
Multiple Content Types
Open Source
• Concept-Based Adaptation

35. Concept-Based Student Modeling
Example 2 Example M
Example 1
Problem 1
Problem 2 Problem K
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Concept N
Examples
Problems
Concepts

36. These cells (first row) shows your
progress in the topics of the course
This bar chart shows
your progress in the
concepts of the course
Each topic has several concepts
associated to it. Mouseover a topic
to highlight its concepts
This bar chart (upside-down)
shows the average progress of
the rest of the class on the
concepts
Middle row shows the difference
between your progress and the
progress of the group
Third row shows the progress of the
group in blue
Concept level OSLM

37. An overlayed pane opens
indicating which topic you
are inspecting (in this case
the topic "Comparisons")
The concepts within
the selected topic are
highlighted

38. Mousing over this
activity
Concepts in the selected
activity are highlighted
This gauge estimates the
how much you can learn
in the selected activity.
You will probably learn
more in activities that
have more new concepts
See more in IUI 2017 Demo!
"Concept-Level Knowledge
Visualization for Supporting Self-
Regulated Learning"

39. Targets Engaged
Adaptive Navigation Support
Topic-based Adaptation
Open Student Modeling
Social Navigation and Comparison
Open Social Student Modeling
Social Educational Progress Visualization
Multiple Content Types
Open Source
Concept-Based Adaptation

40. Acknowledgements
• Joint work with
– Sergey Sosnovsky
– Sharon Hsiao
– Julio Guerra
– Jordan Barria-Pineda
• NSF Grants
– EHR 0310576
– IIS 0426021
– CAREER 0447083
• ADL “PAL” grant to build Mastery Grids

41. Read About It!
• Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive
links: The motivational value of adaptive link annotation. New Review of
Hypermedia and Multimedia 15 (1), 97-118.
• Brusilovsky, P., Hsiao, I.-H., and Folajimi, Y. (2011) QuizMap: Open
Social Student Modeling and Adaptive Navigation Support with TreeMaps.
Proceedings of 6th European Conference on Technology Enhanced
Learning (ECTEL 2011), pp. 71-82
• Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B.
(2013) Progressor: social navigation support through open social student
modeling. New Review of Hypermedia and Multimedia
• Brusilovsky, P., Somyurek, S., Guerra, J., Hosseini, R.,
Zadorozhny, V., and Durlach, P. (2016) Open Social Student Modeling
for Personalized Learning. IEEE Transactions on Emerging Topics in
Computing 4 (3), 450-461.
• Jordan, B.-P., Guerra, J., Huang, Y., and Brusilovsky, P. (2017)
Concept-Level Knowledge Visualization for Supporting Self-Regulated
Learning. In: Proceedings of Companion of the 22nd International
Conference on Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM,
pp. 141-144 also available at https://doi.org/10.1145/3030024.3038262.