«Adaptation, Adoption and Learning Analytics Pilots in Latin America». Pedro J. Muñoz Merino

1. Pedro J. Muñoz-Merino
Universidad Carlos III de Madrid
Adaptation, Adoption and Learning
Analytics Pilots in Latin America
Co-funded by the
Erasmus+ Programme
of the European Union

2. •Initial date: October 2017
•Final date: April 2021
•To improve the quality, efficiency and relevance of
Higher Education in Latin America, developing local
capacity to create, adapt, implement and adopt
Learning analytics tools to improve the academic
decision making process
General objective of the LALA project

3. Countries

4. Partners
2021-01-18 4

5. Web Page
https://www.lalaproject.org

6. Motivation (I)
• Learning analytics is a very useful tool-> analysis of
educational data to improve the learning process
• There are many decisions to take: objectives, tools,
integrations, indicators, dashboards, etc.
• There are many factors that affect the decisions. The same
solution is not valid for all cases
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should
not promote one size fits all: The effects of instructional conditions in predicting
academic success. The Internet and Higher Education, 28, 68-84

7. Motivation (II)
• In the LALA project, we help to use learning analytics
solutions to 8 different Latin American higher education
institutions
• Different tools for each institution
• Same tool with different indicators, dashboards, etc.

8. Analysis in Europe:
Barriers
Tsai, Y. S., Rates, D., Moreno-
Marcos, P. M., Muñoz-
Merino, P. J., Jivet, I.,
Scheffel, M., Drachsler, H.,
Delgado Kloos, & Gašević, D.
(2020). Learning analytics in
European higher education–
trends and barriers.
Computers & Education,
103933.

9. LALA framework
Pérez-Sanagustín M, Hilliger I,
Maldonado-Mahauad J, Pérez-
Álvarez R, Natera Ramirez L,
Muñoz-Merino PJ, Tsai, Y., Ortiz-
Rojas, M., Broos, T., Zuñiga-Prieto,
M.A., Sheihing, E. Whitelock-
WainwrightBuilding, A. Capacity to
use learning analytics to improve
higher education in Latin America:
LALA Framework. Version 2.0 ed.
European Commission, 2019. 38 p.

10. Institutional
dimension
• LALA Canvas. Based on the ROMA
framework
• Adapted artifacts from the SHEILA
project (https://sheilaproject.eu/)
• Students survey and focus group
• Teachers survey and focus group
• Managers interview
Young, J., Shaxson, L., Jones, H.,
Hearn, S., Datta, A., & Cassidy, C.
(2014). A guide to police engagement
and influence. Overseas Development
Institute.

11. Technological
dimension
• Different aspects based on the ORLA framework
• Forms from managers, researchers, developers and academic staff
• Guide of technical considerations for the development and
implementation/adaptation of the tool
• Guide of considerations for the design of the procedure for
evaluation and testing of the tool
Prieto, L. P., Rodríguez-Triana, M. J., Martínez-Maldonado, R., Dimitriadis, Y.,
& Gašević, D. (2019). Orchestrating learning analytics (OrLA): Supporting
inter-stakeholder communication about adoption of learning analytics at the
classroom level. Australasian Journal of Educational Technology, 35(4).

12. Ethical dimension
• Adapted artifacts from the SHEILA project
(https://sheilaproject.eu/)
• Students survey and focus group
• Teachers survey and focus group
• Managers interview
• Selection of some relevant previous works
• Examples of different forms from different institutions

13. Communal
dimension
• LALA community
• LALA SOLAR SIG
• LALA conference
• LALA workshops
• LALA training events

14. Application of the
LALA framework

15. Analysis in Latin
American: Needs
Hilliger, I., Ortiz-Rojas, M.,
Pesántez-Cabrera, P., Scheihing,
E., Tsai, Y. S., Muñoz-Merino, P.
J., Broos, T., Whitelock-
Wainwright, A. & Pérez-
Sanagustín, M. (2020).
Identifying needs for learning
analytics adoption in Latin
American universities: A mixed-
methods approach. The
Internet and Higher Education,
45, 100726.

16. Institutional factors
(I)
• Objectives and expectations
• Culture
• Capacity to involve key stakeholders
• Type of stakeholders: professors, students, managers,
administrative staff
• Funding and initial resources

17. Institutional factors
(II)
• They have an effect on the types of learning analytics tools
and services
• Example in LALA
• Some institutions selected a counselling tool and an early dropout
prediction tool at the academic level.
• Some institutions selected a counselling and prediction tool for
specific courses in MOOCs

18. Selected tools for
each institution
University /Tool NMP: Counselling
for courses
Counselling for
degrees
Dropout for
courses
Dropout for
degrees
OnTask
PUC X X
ESPOL X X
UCuenca X X
UACH X X
UChile X X
UPS X X
UPernambuco X
I. de Zitacauro X

19. • Tools adapted, adopted or inspired from tools developed in
Europe,
• A couselling tool at Katholike Universite of Leuven,
• A dropout prediction tool at Universidad Carlos III de Madrid,
• OnTask tool in which University of Edinburgh has been involved
Adapted and
Adopted Tools (I)
Ortiz-Rojas, M., Jimenez, A., Maya, R., Muñoz-Merino, P. J., Moreno-
Marcos, P. M., Marín, J. I., Delgado Kloos, C.,Zuñiga Prieto, M.A., Ulloa,
M., Pérez, R., Pérez-Sanagustín, M., Henriquez, V., Guerra, J., Ferreira, R.,
Broos, T., & Millecamp, M., WPD3. O. 4 (2019), “Design for Learning
Analytics tools for LALA”

20. Adapted and
Adopted Tools (II)
• NMP (developed at PUC) Perez, R. A., Maldonado, J., Sharma, K., Sapunar, D., &
Perez-Sanagustin, M. (2020). Characterizing Learners'
Engagement in MOOCs: An Observational Case Study
Using the NoteMyProgress Tool for Supporting Self-
Regulation. IEEE Transactions on Learning Technologies.

21. Adapted and
Adopted Tools (III)
• Dropout for courses in MOOCs

22. Adapted and
Adopted Tools (IV)
• Counselling for degrees (developed at ESPOL, UCuenca,
UACH and UPS)

23. • Dropout for degrees: ESPOL
Adapted and
Adopted Tools (V)

24. • Dropout for degrees: UCuenca
Adapted and
Adopted Tools (VI)

25. • Dropout for degrees: Universidad Austral de Chile
Adapted and
Adopted Tools (VII)

26. • OnTask
Adapted and
Adopted Tools (VIII)
Pardo, A., Bartimote, K., Shum, S. B.,
Dawson, S., Gao, J., Gašević, D., ... &
Vigentini, L. (2018). OnTask: Delivering
data-informed, personalized learning
support actions. Journal of Learning
Analytics, 5(3), 235-249.

27. • Common architecture
Adapted and
Adopted Tools (IX)

28. LALA pilots (I):
Potential users
University /Number of stakeholders Number of
students
Number of teachers
/ counsellors
PUC 1.294 30
ESPOL 9.485 641
UCuenca 1.873 74
UACH 5.000 47
Universidad de Chile 1.252 4
UPS (Universidad Politécnica Salesiana) 4.652 119
Universidad de Pernambuco 112 3
Instituto de Zitacauro
Total numbers > 22.000 918

29. LALA pilots (II)
• The level of use of the tools was good after analysing the
logs
• The effectiveness of the tools was good. However, there is a
difficulty to measure increase in learning since this is difficult
to measure, and specially with a short period of years.
• The tools are perceived as useful.

30. LALA pilots (III)
• Some lessons learned
• Importance of involvement of key stakeholders and availability of
resources
• Identification of the more engaged stakeholders to extend the
process
• Importance of a multidisciplinary team
• Messages, recommendations, visualizations of the tools should be
checked before the interventions.
• Involvement of users in the design of the tools
• Training is necessary. Stakeholders can make correct
interpretations and take proper decisions

31. LALA pilots (IV)
• Some lessons learned
• Socialization of the pilots results with the authorities and users
• Capacity of adaptation depending on the context
• Pilots as a step to adoption

32. • Self questionarie about SRL does not achieve good predictive power.
• SRL patterns have a good predictive power by themselves
32
LALA pilots: Research
conclusions example
Moreno-Marcos, P. M., Muñoz-Merino, P. J., Maldonado-Mahauad, J., Pérez-
Sanagustín, M., Alario-Hoyos, C., & Delgado Kloos, C. (2020). Temporal
analysis for dropout prediction using self-regulated learning strategies in
self-paced MOOCs. Computers & Education, 145, 103728.

33. • Good predictive power since the beginning
33
LALA pilots: Research
conclusions example
Moreno-Marcos, P. M., Muñoz-Merino, P. J., Maldonado-Mahauad, J., Pérez-
Sanagustín, M., Alario-Hoyos, C., & Delgado Kloos, C. (2020). Temporal
analysis for dropout prediction using self-regulated learning strategies in
self-paced MOOCs. Computers & Education, 145, 103728.

34. Conclusions
• The LALA framework helps institutions for the adoption of
learning analytics: 4 dimensions: institutional, technical, ethical
and communal
• Challenging process to adopt LA. Many stakeholders involved
(need to encourage them), many different decisions and analysis.
Different velocities depending on the context of each institution
(6 months or more than 2 years)
• Some commonalities: selected tools, general architecture,
functionality, data
• Some differences: selected tools, adapted architecture, data
extensions, indicators, different look and feel and functionality in
dashboards, etc.

35. Conclusions
• Source code available
• LA tools evolve during the time
• Continuous monitoring with stakeholders. Phases of design
• Results from the pilots
• Pilots with potential users: > 22.000 students > 900 teachers
• Good impact, effectiveness and usefulness
• LALA handbook available: LALA framework, LALA adaptation of tool,
LALA pilots

36. LALA project
Work partially funded by the LALA project (grant no.
586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP). This project has
been funded with support from the European Commission.
This communication reflects the views only of the author,
and the Commission and the Agency cannot be held
responsible for any use which may be made of the
information contained therein

37. Pedro J. Muñoz-Merino
Universidad Carlos III de Madrid
Adaptation, Adoption and Learning
Analytics Pilots in Latin America
Co-funded by the
Erasmus+ Programme
of the European Union