The past five years have seen a dramatic growth in interest in the emerging field of Learning Analytics (LA), and particularly in the potential the field holds to address major challenges facing education. However, much of the work in the learning analytics landscape today is closed in nature, small in scale, tool- or software-centric, and relatively disconnected from other LA initiatives. This lack of collaboration, openness, and system integration often leads to fragmentation where learning data cannot be aggregated across different sources, institutions only have the option to implement "closed" systems, and cross disciplinary research opportunities are limited. Beyond the immediate concerns this fragmentation creates for educators and learners, a closed approach dramatically limits our ability to build upon successes, learn from failures and move beyond the "pockets of excellence (and failures)? approach that typifies much of the educational technology landscape. The potential benefits of openness as a core value within the learning analytics community are numerous. Learning initiatives could be informed by large scale research projects. Open-source software, such as dashboards and analytics engines, could be available free of licensing costs and easily enhanced by others, and OERs could become more personalized to match learners' needs. Open data sets and reproducible papers could rapidly spread understanding of analytical approaches, enabling secondary analysis and comparison across research projects. To realize this future, leaders within the learning analytics, open technologies (software, standards, etc.), open research (open data, open predictive models, etc.) and open learning (OER, MOOCs, etc.) fields have established a "network of practice" aimed at connecting subject matter experts, projects, organizations and companies working in these domains. As an initial organizing event, these leaders organized an Open Learning Analytics (OLA) Summit directly following the 2014 Learning Analytics and Knowledge (LAK) conference this past March as means to further the goal of establishing "openness' as a core value of the larger learning analytics movement. Additional details on the Summit and those involved can be found at: http://www.prweb.com/releases/2014/04/prweb11754343.htm. This panel session will bring together several thought leaders from the Open Learning Analytics community who participated in the Summit to facilitate an interactive dialog with attendees on the intersection of learning analytics and open learning, open technologies, open data, and open research. The presenters represent a broad range of experience with institutional analytics projects, an open source development consortium, the sharing of open learner data, and academic research on open learning environments.

1. Open Learning Analytics Panel
JOSH BARON
ST IAN HÅKLEV
NORMAN BIER
HASH TAG #OPENLA

2. Panel Session Overview
Session Goal: Stimulate discussion around the
importance of open learning analytics to the future of
the larger open education movement
Longer-Term Objective: Form connections between
the OpenEd and Open Learning Analytics networks
Session Format:
 Setting the Context – Open Learning Analytics
 Examples from the Real World
 Discussion/Q&A

3. What is Learning Analytics?
Academic Analytics Learning Analytics
A process for providing higher
education institutions with the data
necessary to support operational
and financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement
of specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the
primary audience
* - Source: Analytics in Higher Education: Establishing a Common Language

4. 2014 Open Learning Analytics Summit
Society for Learning Analytics (SoLAR) began
exploring openness in learning analytics in 2011
International OLA Summit held in March 2014
Participants identified OLA “knowledge domains”
as means to organize future work

5. OLA Knowledge Domains
Open Data and Models
 Releasing data sets and models under open licenses
Open Research
 Publishing research in open-access journals
Open-Source Software/Platforms
 Open software, standards and APIs
Open Strategy and Policy
 Open documents on strategy and policy
Open Learning Designs
 Combine OER & LA to create new models of learning

6. Openness = Science
NORMAN BIER
DIRECTOR, OPEN LEARNING INI T IAT IVE
CARNEGIE MEL LON UNIVERSI TY

7. The changing value of content
Changing focus in OER community
Commoditization of content (Wiley: ‘content is
infrastructure’)
Instrumenting content is difficult and expensive
Well-instrumented content and the tools to analyze
student interactions with that content will continue to
increase in importance

8. Problems with Black-Box Systems

9. Examples
Challenges Opportunities
CC-OLI Research
MOOC Research
UC Davis
Common measures of
outcomes and
achievement
Simon DataLab
Design Analytics
Stanford Outcomes
Analytics Service
Swappable Models
Learner Centered

10. MOOC RESEARCH AND
REPRODUCIBLE SCIENCE
ST IAN HÅKLEV
INST I TUT IONAL RESEARCHER, OPEN UTORONTO
UNIVERSI TY OF TORONTO

11. Supporting three MOOC research projects
(MRI)

12. Tools to collaborate and document

13. Connecting database with other data

14. Clicklog: big data, making it queryable,
increasing levels of abstraction

15. 20

24. Open Data Models & OS
Learning Analytics Platform
JOSH BARON
SENIOR ACADEMIC TECHNOLOGY OF F ICER
MARIST COL LEGE

25. OAAI: Overview and Impact
EDUCAUSE Next
Generation Learning
Challenges (NGLC)
Funded by Bill and
Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”

26. Student Aptitude Data
(SATs, current GPA, etc.)
Student Demographic
Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Step #1: Developed
model using historical
data
Predictive
Model
Scoring
Identifies
students
“at risk” to
not
complete
course
LMS Data SIS Data
OAAI Early Alert System Overview
Intervention Deployed
“Awareness” or Online
Academic Support
Environment (OASE)
“Creating an Open Academic
Early Alert System”
Model Developed
Using Historical Data
Academic Alert
Report (AAR)

27. Research Design
Deployed OAAI system to 2200 students across
four institutions
 Two Community Colleges
 Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections
 One section was control, other 2 were treatment groups
Each instructor received an AAR three times during
the semester:
 Intervals were 25%, 50% and 75% into the semester

28. Intervention Research Findings
Final Course Grades
Analysis showed a
statistically significant
positive impact on final
course grades
 No difference between
treatment groups
Saw larger impact in
spring then fall
Similar trend amount low
income students
Mean Final Grade for "at Risk" Students
100
90
80
70
60
50
Awareness OASE Control
Final Grade (%)

29. Intervention Research Findings
Content Mastery
Student in intervention
groups were statistically
more likely to “master the
content” then those in
controls.
 Content Mastery = Grade
of C or better
Similar for low income
students.
Content Mastery for "at Risk" Students
1000
800
600
400
200
0
Yes No Yes No
Control Intervention
Frequency

30. More Research Findings…
JAYAPRAKASH, S. M. , MOODY, E. W. , LAURÍA, E. J. ,
REGAN, J. R. , & BARON, J. D. (2014) . EARLY ALERT
OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN
SOURCE ANALYTICS INITIATIVE. JOURNAL OF
LEARNING ANALYTICS, 1(1) , 6 -47.

31. Strategic Vision: Open
Learning Analytics Platform
Collection – Standards-based
data capture from any
potential source using
Experience API and/or IMS
Caliper/Senor API
Storage – Single repository
for all learning-related data
using Learning Record Store
(LRS) standard.
Analysis – Flexible
Learning Analytics
Processor (LAP) that can
handle data mining, data
processing (ETL), predictive
model scoring and reporting.
Communication –
Dashboard technology for
displaying LAP output.
Action – LAP output can be
fed into other systems to
trigger alerts, etc.

32. Access to Predictive Model and
related OS Software…
OAAI PREDICT IVE MODEL DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/ 8AWCB
APEREO LEARNING ANALYT ICS PROCESSOR DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/KWCVBQ

34. Discussion and Q&A

35. Discussion Questions
Do you feel LA will be important to OER and Open
Education in the future? How important?
Where do you see connections between the OLA
network and Open Education?
How might we best facilitate making connections
across different “networks”?
[insert more questions]

36. Additional Resources
European OLA Summit – December 1st (LACE)
 http://www.laceproject.eu/
The Asilomar Convention for Learning Research in
Higher Education
 http://asilomar-highered.info
Apereo Learning Analytics Initiative
 https://confluence.sakaiproject.org/x/rIB_BQ
Society for Learning Analytics and Research
 http://solaresearch.org

37. What is Student Success?
X
Z
Engagement
Learning Y
Credit: mike.sharkey@phoenix.com