Presentation delivered to the Nevada Conference on Digital Learning, April 12, 2014 in Las Vegas, Nevada.

1. Understanding Student Learning
Using Learning Management Systems
and Basic Analytics
Michael Wilder and Matt Bernacki

2. What is learning analytics?
“Learning analytics is the
measurement, collection, analysis, and reporting of
data about learners and their contexts, for the
purposes of understanding and optimizing learning
and the environments in which it occurs.”
(1st International Conference on Learning Analytics and Knowledge, 2011)

3. Characterized as “an engine with five stages”
• Capture
• Report
• Predict
• Act
• Refine
(Campbell, DeBlois, and Oblinger, 2007)

4. Application
• Predicting outcome achievement
• Course and program dashboards
• Curricular evaluation
• Prioritizing learning outcomes
• Set course and instructional policies
• Defining academic quality
(Bach, 2011)

5. Also good for
• Curricular improvement
• Intervention and support services
“Early warning system”
• Individualizing learning per student based on our
interpretation of their data
• Researching how students use the tools we build and
learn with them.
• Improving operational efficiency of the university
• Comparison of performance with peer universities

6. Major research groups
• Educational data mining
“How can we extract value from these big sets of learning-related data?”
• Learning analytics
“How can we optimize opportunities for online learning?”
• Academic analytics
“How can we substantially improve learning opportunities and
educational results at national or international levels?”
(Ferguson, 2012)

7. Predictors and indicators
• Activity indicators
(Number of logins, time spent in the learning environment)
• Performance indicators
(Grades, quiz scores)
• Dispositional indicators
(Age, GPA, prior learning experience, financial status)
• Student artifacts
(Essays, blog and discussion forum posts, media productions
(Brown, 2012)

8. A CASE STUDY:
BLACKBOARD LEARN

12. Basic Data
Analysis Tools
Where to find them

14. Course analytics
• All User Activity inside Content Areas
• Course Activity Overview
• Course Performance
• Overall Summary of User Activity
• Student Overview for Single Course
• User Activity in Forums
• User Activity in Groups
Mapping It Out, Part III:
Closing The Loop

19. • select cm.course_name, count(cc.pk1) as
LearningModules
from course_main cm, course_contents cc
where cm.pk1 = cc.crsmain_pk1
and cc.cnthndlr_handle = „resource/x-bb-lesson‟
and cm.pk1 in
[list of courses in given term]
group by cm.course_name
order by cm.course_name;

20. • select cm.course_name, count(*) replies
from forum_main fm
left join msg_main mm on fm.pk1 = mm.forummain_pk1
left join conference_main confmain on confmain.pk1 =
fm.confmain_pk1
left join conference_owner co on co.pk1 =
confmain.conference_owner_pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where mm.msgmain_pk1 is not null
and co.owner_table = 'COURSE_MAIN„
and co.owner_pk1 in
[list of courses in given term]
group by cm.course_name;

22. Analytics in education can be viewed as existing in various
levels, ranging from individual student, classroom,
department, university, region, state/province, national, and
international.
Buckingham Shum (2012) groups these organizational levels as
micro-, meso-, and macroanalytics layers.
What can analytics reveal about learning in general?

23. ANALYTICS FOR
LEARNING

24. Research
Question
How does motivation and learning behavior influence
reading comprehension?
Environment nStudy, a web browser with tools to support learning
Method 1) Survey on motivation
2) Pretest
3) Observe behavior during 20 minutes of reading
4) A comprehension test
5) ANALYTICS
Analytics Look at number of times a student used a tool and its
relationship to learning
Findings …

25. Mastery  SRL  Learning
Performance Avoidance goals are not ideal for learning
25
Highlighting
Note taking
Reviewing
annotation
s
Seeking more
information
Monitoring
progress
Mastery
Approach
Performance
Approach Textbase
Comprehension
Increase (%)
Situation Model
Comprehension
Increase (%)
Performance
Avoidance
ACHIEVEMENT
GOALS
SRL BEHAVIORS
LEARNING
OUTCOMES

26. Research
Question
Does the student’s reason for seeking help from software
help affect whether the help helps?
Environment Cognitive Tutor, an intelligent tutoring system for math
Method 1) Survey students‟ achievement goals
2) Observe their problem solving behaviors in CT
3) ANALYTICS
26

27. Analytics Findings
• Achievement goals
predicted students‟
tendency to learn from
hints
• Students with Mastery
approach goals were
more likely to learn
from hints
• Those with strong
performance approach
goals were less likely
to learn from hints
Student ID Outcome Duration
stu_324w8b BUG 69
stu_324w8b BUG 6
stu_324w8b BUG 5
stu_324w8b INITIAL_HINT 6
stu_324w8b HINT 2
stu_324w8b HINT 4
stu_324w8b OK 19
stu_324w8b OK 12
stu_324w8b ERROR 15
stu_324w8b OK 4

28. Research
Question
Can educating students about learning principles and
showing them how use them influence their behavior and
learning in an online course?
Environment Research Methods course in Blackboard Learn
Method 1) Train 30 students re: Learning Principles; don‟t train 30 others
2) Let them learn
3) ANALYTICS
28Analytics =