An overview of the Draft Issue Brief prepared by SRI International for the US Department of Education on Educational Data Mining and Learning Analytics

1. Enhancing Teaching and
Learning Through
Educational Data Mining
and Learning Analytics
An Issue Brief Prepared for the US Department of Education
Office of Educational Technology
April 10, 2012

2. Purpose
 Look at data analytics techniques in the context of a
new framework for educational evidence
 See how commercial techniques (state-of-the-practice)
might apply to education
 Define educational data mining and learning analytics:
What questions can they answer?
 Categorize applications and understand benefits, and
challenges

3. Issue Brief Questions
 What is educational data mining, and how is it applied? What kinds of
questions can it answer, and what kinds of data are needed to answer these
questions?
 How does learning analytics differ from data mining? Does it answer different
questions and use different data?
 What are the broad application areas for which educational data mining and
learning analytics are used?
 What are the benefits of educational data mining and learning analytics, and
what factors have enabled these new approaches to be adopted?
 What are the challenges and barriers to successful application of educational
data mining and learning analytics?
 What new practices have to be adopted in order to successfully employ
educational data mining and learning analytics for improving teaching and
learning?

4. Data Sources
 A review of selected publications and fugitive or gray
literature (Web pages and unpublished documents) on
educational data mining and learning analytics
 Interviews of 15 data mining/analytics experts from
learning software and learning management system
companies and from companies offering other kinds of
Web-based services
 Deliberations of a technical working group of eight
academic experts in data mining and learning analytics.

5. Inspiration for Issue Brief
National Educational Technology Plan
“When students are learning online, there are multiple opportunities to
exploit the power of technology for formative assessment. The same
technology that supports learning activities gathers data in the course
of learning that can be used for assessment…. An online system can
collect much more and much more detailed information about how
students are learning than manual methods. As students work, the
system can capture their inputs and collect evidence of their problem-
solving sequences, knowledge, and strategy use, as reflected by the
information each student selects or inputs, the number of attempts the
student makes, the number of hints and feedback given, and the time
allocation across parts of the problem.” (U.S. Department of
Education 2010, p. 30)

6. Interconnected Feedback
System for Education
National Educational Technology Plan
“The goal of creating an interconnected feedback system
would be to ensure that key decisions about learning are
informed by data and that data are aggregated and made
accessible at all levels of the education system for
continuous improvement.” (U.S. Department of Education
2010, p. 35)

7. Research Areas
 Educational Data Mining: develops new techniques, tests
learning theories and informs educational practice. Looks for
patterns in unstructured data. Generally automates
responses to learners.
 Learning analytics: Applies techniques and takes a
“system-level” view of teaching and learning, including at the
institutional level. Generally supports human decision
making vs. automating responses.
 Visual Data Analytics: taps the ability of humans to discern
patterns in visually represented complex datasets.

8. EDM/LA Enables Adaptive
Learning Systems

9. Application Areas
 User Modeling: Model a learner’s knowledge, behavior,
motivation, experience, and satisfaction.
 User Profiling: Cluster users into similar groups.
 Domain Modeling: Decompose content to be learned into
components and sequences.
 Effectiveness: Test learning principles, curricula, etc.
 Trend Analysis: Track changes over time.
 Recommendations and Improvements: Suggest
resources and actions to learners; adapt system to learners.

10. Challenges
 Technical: Handling big data; interoperability of data
systems; asking the right questions
 Institutional: Requires a culture of data-driven
decision making and transparency in models that
analyze data
 Privacy and Ethics: Maintain student and teacher
privacy while allowing data aggregation to drive
powerful models

11. Excerpt of Recommendations
 Educators: develop a culture of using data for making
instructional decisions. Understand and communicate
sources of data
 Researchers and Developers: Study usability and
impact of “dashboards.” Understand how models can
move to new contexts of use.
 Collaboration: Work across sectors to build capacity
and knowledge. Include learning system designers
(often commercial entities), learning scientists, IT
departments, administrators, and educators on teams.