Increasing MOOC completion rates through social interactions Emoocs2016 Attrition and forum activity - labarthe bachelet bouchet

1. Increasing MOOC completion rates
through social interactions
A recommendation system
Hugues Labarthe, Rémi Bachelet, François Bouchet & Kalina Yacef
Download this slideshow : http://goo.gl/lkba7HFebruary 22nd, 2016, Graz, Austria

2. MOOC completion rates are notoriously low
• What we know:
– MOOC students interacting with each other have better odds to
get certified (Yang, Wen & Rose, 2014)
• Problem:
– How to improve user-centered experience and facilitate user’s
involvement with peers?
2

3. Questions & data
Research questions:
1. Impact of social interactions between students?
2. Best way to foster peer interaction?
3 data sets:
– Session 2 to 5: Impact of social interactions? Longitudinal Study
– Session5: Students groups. Experiment.
– Session6: Chat & contact recommendation module. Experiment
3

4. Context: MOOC GdP
4
GdP2 GdP3 GdP4 GdP5 GdP6
Date Fall 2013 Spring 2014 Fall 2014 Spring 2015 Fall 2015
Duration 5 weeks
+ 1 week for
final exam
5 weeks including
1 elective course
+ 1 week for final
exam
6 weeks including
2 elective course
+ 1 week of exams
6 weeks including
2 elective course
+ 1 week of exams
6 weeks including
2 elective course
+ 1 week of exams
Attendance:
- Enrolled
- > 1 quiz in basic
track
- > 1 deliverable in
adv. track
10848
5711
1011
11827
5899
705
19171
8120
1197
17579
4842
725
22300
7305
970
Track success:
- Basic
- Advanced
61%
78%
38%
67%
41%
73%
47%
69%
53%
71%
• French MOOC on
project management
• Platform: Canvas
Engineering: Unow
• 2 sessions/year:
enhanced content
• 3 tracks: basic,
advanced, team
• Barcamp
"richer learning
activities for MOOCs :
14+
ideas " EMOOCs2015

5. 5
“Funnel of participation” (Clow, 2013)
Attrition rate increasing, “September effect” (in-cursus students = less drop out in these sessions)
Study #1 (Longitudinal): attrition & interactions in 5 sessions (1/3)

6. 6
“Participation funnel” (Clow, 2013)
Attrition rate increasing, “September effect” (in-cursus students = less drop out in these sessions)
Study #1 (Longitudinal): attrition & interactions in 5 sessions (1/3)

7. 7
> 50,000 posts & PM - Social networks not included
+8k contributors (13.5%), only 3.5% used both posts & PM
3 functions: socializing, learning reinforcement, experience sharing
Longitudinal study : attrition & interactions (2/3)

8. Longitudinal study : attrition & interactions (3/3)
8

9. Longitudinal study : attrition & interactions (3/3)
9
The drop in exam pass rate for 5+post comes
mostly from the high forum activity of our
teaching assistant staff. In our newly published
papers, this bias is controlled, as well as
involvement in the technical help thread

10. Longitudinal study : Findings
Interacting is a strong predictor of success
+35% exam pass on average
10

11. Increase social interactions: Recommendations
– « Introduce yourself » thread
• Trigger contributions (“must read” in week zero)
• Boost visibility of this thread: pre-MOOC announcement
– Geolocalization
• « where are you ? » map
– Facebook, Google+, Twitter…
• Capitalize on existing social networks
• Foster them : news, answers…
– “Question of the week”
• starting point for the weekly live Q&A
– Strategies
• Reward active posters: Animator badge
• TA team : Do not answer questions too fast, leave this to the other students
11

12. Study #2: students group to increase interactions –
methodology
• Session 5 : Canvas groups features (private forums, wiki, list of group members
with link to profile)
• Experiment : 34 groups of variable size:
– 4 groups of 50 students
– 10 groups of 10 students
– 20 groups of 4 students
• 5 grouping by similarity criteria (data from initial research questionnaire)
– Geographical proximity (Country)
– Study level
– Age
– Family status
– Previous experience with MOOCs
12

13. Study #2: findings & recommendations
Not many interactions,
yet much to be
learned!
In groups of 4 & 10, no
more than 1 post/person =
no real interaction, not
many visits after first
attempt
13
Criteria Observation Recommendation
Teammates visibility Group page quickly
ignored
Make the list of other group members
more visible
Contact visibility No visit to check
replies to one’s post
Improve notification system
Sample size Many students do
not interact
Larger sample size next time
# of recommended
contacts
Low activity even in
large groups
More contact recommendations increase
the chances of finding other students
willing to interact
Individualized
interaction
Similar interface
prevented students
from seeing what
more it brought
Allow students to create small groups of
their own

14. Study #3: contact recommendation & chat modules
14
A contact recommendation widget in
every page of the platform below the
navigation column
Advanced ID card with:
- profile info,
- ways to contact (PM and instant message)
- adding to favorites/ignored
Available with a simple mouse-over
A permanent chat widget
Notification (+ sound) when a message arrives!
Contact manager, to initiate conversation
with favorite(s) contact(s)
Chat manager, to start or pick up a
conversation

15. GdP6: preliminary findings
15
Findings:
- Strong curiosity for students’ profiles
- Asynchronous chat interactions
- Impact on attrition and performance:
currently under review
GdP6:
- 22,300 enrolled
- 16,127 connected once
- 8,053 active
Experiment: 8,674 students
- 1792 in control group (no
recommendation, no chat)
- 2025 accepted the terms of use
- 271 interacted with others
through chat

16. Conclusion
• Interactions are correlated w/ success, and are key to
reducing attrition. Need to find:
• The right combination of factors to recommend students to their peers
• The right interface for them to interact with one another
Can only be achieved through experiments (work in progress)
• Perspectives:
– Technology  course –interaction design
– Identify students who want to interact = maximize chances of
response
16

17. Thanks for listening!
• Twitter : @R_Bachelet, Googleplus : +Rémi Bachelet
Read the paper online : https://hal.archives-
ouvertes.fr/view/index/docid/1277664

18. Does peer grading work? How to
implement and improve it?
Comparing instructor and peer assessment in
MOOC GdP
Rémi Bachelet, Drissa Zongo, Aline Bourelle
Download this slideshow : http://goo.gl/GiFvXb