EMPOWER: Artificial Intelligence in online education webinar week, day 3. Presentation by Prof. Olaf Zawacki-Richter

1. AI applications in higher education –
challenges and opportunities in ODE
Prof. Olaf Zawacki-Richter
University of Oldenburg, Germany
EMPOWER
Artificial Intelligence webinar week
EADTU
June 18, 2020

2. Folie 2
§ What is AI and AI in Education?
§ What are potential areas of AI application in higher education?
§ How has research on AI in higher education developed over time?
§ What is the current state of AIEd?
§ What are opportunities, challenges and risks in ODE?
Overview

3. Folie 3
https://en.wikipedia.org/wiki/Moore%27s_law (CC-BY-SA)
Moore's Law
BIG Data

4. Folie 4
§ EDUCAUSE Horizon Report 2019 Higher Education Edition:
Experts anticipate AI in education to grow by 48 anually until 2022
§ Contact North (2018): "there is little
doubt that the [AI] technology is inex-
orably linked to the future of higher
education" (p. 5)
§ Heavy investments: TU of Eindhoven
will launch an AI Systems Institute with
50 new professorships for education
and research in AI
Relevance of AI in Education (AIEd)

5. Folie 5

6. Folie 6
§ "the ability to reason, plan, solve problems, think abstractly,
comprehend complex ideas, learn quickly and learn from
experience" (Gottfredson, 1997, p. 13)
§ John McCarthy (1956, cited in Russel & Norvig, 2010, p. 17):
§ The study [of artificial intelligence] is to proceed on the
basis of the conjecture that every aspect of learning or any
other feature of intelligence can in principle be so precisely
described that a machine can be made to simulate it.
Intelligence is...
Gottfredson, L. S. (1997). Mainstream science
on intelligence. Intelligence, 24(1), 13–23.
Russell, S. J., Norvig, P., & Davis, E. (2010).
Artificial intelligence: A modern approach
(3rd ed). Upper Saddle River: Prentice Hall.

7. Folie 7
§ General Artificial Intelligence
o aka "strong AI"
o still only science fiction
§ Narrow Artificial Intelligence
o aka "weak AI"
o aka "good old-fashioned AI" (Haugeland, 1985)
o aka "machine learning"
o a one-trick horse
Haugeland, J. (1985). Artificial intelligence: The very idea. Cambridge, Mass.: MIT Press.

8. Slide 8
Artificial Intelligence – Machine Learning – Deep Learning
Copeland (2016): https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-
machine-learning-deep-learning-ai/

9. Folie 9
Hinojo-Lucena et al. (2019, S. 1):
§ "This technology [AI] is already being introduced in the field
of higher education, although many teachers are unaware
of its scope and, above all, of what it consists of."
Hinojo-Lucena, F.-J., Aznar-Díaz, I., Cáceres-Reche, M.-P., & Romero-Rodríguez,
J.-M. (2019). Artificial Intelligence in Higher Education: A Bibliometric Study on
its Impact in the Scientific Literature. Education Sciences, 9(1), 51.
https://doi.org/10.3390/educsci9010051
So… what is AI in Education (AIEd)?

11. Folie 11
§ learner-facing (e.g. adaptive LMS or ITS)
§ teacher-facing
(e.g. assessment and plagiarism detection tools)
§ system-facing AIEd
(e.g. monitoring tools on institutional level)
AI in Education (AIEd)
Baker, T., & Smith, L. (2019). Educ-AI-tion Rebooted? Exploring the future of
artificial intelligence in schools and colleges. Nesta Foundation.

12. Systematic Review on AIEd in HE

13. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial
intelligence applications in higher education – where are the educators? International Journal of Educational
Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Downloaded over 18k times
since October 2019

14. Zawacki-Richter, O., Kerres, M.,
Bedenlier, S., Bond, M., & Buntins,
K. (Eds.). (2019). Systematic
reviews in educational research:
Methodology, perspectives and
application. Heidelberg: Springer
Open, Verlag für Sozialwissen-
schaften.

15. Folie 15
Systematic Review Questions
§ Mapping:
How have publications on AI in higher education developed over
time, in which journals are they published, and where are they
coming from in terms of geographical distribution and the author's
disciplinary affiliations?
§ Concept and Ethics:
How is AI in education conceptualised and what kind of ethical
implications, challenges and risks are considered?
§ Applications:
What is the nature and scope of AI applications in the context of
higher education?

16. Folie 16
Context of AIEd Studies
§ Higher Education (n = 146)
§ K-12 (n = 6)
§ Cont. Ed. / Corporate Training (n = 1)
§ Vocational Training (n = 1)

17. Folie 17

18. Folie 18
Only 13 papers (8.9%) by first authors with an Education background.

19. Folie 19
Student life-cycle (Reid, 1995):
§ administrative services (e.g. admission, counselling, library
services) – 92 studies (63.0 %)
§ academic support services (e.g. assessment, feedback,
tutoring) – 48 studies (32.8 %)
§ Six studies (4.1 %) covered both levels
Reid, J. (1995). Managing learner support. In F. Lockwood (Ed.), Open
and distance learning today (pp. 265–275). London: Routledge.

21. Folie 21
Profiling and prediction: admissions
§ Chen and Do (2014): "the accurate prediction of students' academic
performance is of importance for making admission decisions as
well as providing better educational services" (p. 18).
§ Acikkar and Akay (2009) predict admission decisions based on a
physical ability test, scores in the National Selection and Placement
Examination, and GPA
§ Cukurova University, Adana,
Turkey
§ Accuracy: 97% in 2006 using
SVM

22. Folie 22
Intelligent Tutoring Systems (ITS)
§ Meta-analysis of 39 ITS studies (Steenbergen-Hu & Cooper, 2014):
o ITS had moderate positive effect on college students' learning
o ITS were less effective than human tutoring
o but ITS outperformed all other instruction methods (traditional
classroom instruction, reading printed or digital text, CAI,
laboratory or homework assignments, and no-treatment control)
§ Simulation of one-to-one tutoring – enormous potential for ODE
o make decisions about the learning path and content,
o provide cognitive scaffolding and
o help, to engage the student in dialogue.
Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring
systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331–347.

23. Folie 23
Duolingo chat bots for
language learning

24. Folie 24
Assessment and evaluation
§ Automated Essay Scoring (AES): Machine learning for assignment
classification, grading
§ Practical for large courses due to the need to calibrate the system
with pre-scored assignments (supervised machine learning)
§ Research focused on undergraduate courses (n = 10)

25. Folie 25
Automated Essay Scoring (Gierl et al., 2014)
§ AES for large scale assessment
§ Medical Council of Canada Qualifying Examination
§ Clinical Decision Making Constructed-Response Items (CDM-DCR)
§ 5.540 students took the test in 2013
§ 100 raters need 4 days (2.800 to 3.200 hours) to score the items
Gierl, M. J., Latifi, S., Lai, H., Boulais, A., & Champlain, A. (2014). Automated essay scoring and the
future of educational assessment in medical education. Medical Education, 48(10), 950–962.

26. Folie 26
Automated Essay Scoring (Gierl et al., 2014)
§ Structure of CDM short-answer write-in questions

27. Folie 27
Automated Essay Scoring (Gierl et al., 2014)
§ LightSIDE software (open source) for machine learning
classification
§ Agreement between
97.3 and 98.2 %
§ Kappa values: almost
perfect agreement
§ Required 3.5 hours of
one rater of coding to
prepare the data, and
10 s to score the 2013
examination

28. Folie 28
Automated Essay Scoring
§ Commercial software
§ IntelliMetric: http://www.intellimetric.com

29. Folie 29
Adaptive systems and personalisation
§ Kose & Arslan (2016) developed and evaluated an "Intelligent E-
Learning System" for Computer Programming courses at Usak
University in Turkey
§ System based on ANN that is able to evaluate students’ responses
on applications and predict their learning levels on different aspects
of computer programming.
§ Based on student's learning level values appropriate content is
provided to support student's learning
Kose, U., & Arslan, A. (2016). Intelligent E-Learning System for Improving Students’
Academic Achievements in Computer Programming Courses. International Journal of
Engineering Education, 32(1), 185–198.

30. Folie 30
CENTURY Intelligent Learning Platform
§ "CENTURY is the first teaching and learning platform to use AI.
Our technology, named CAI, provides students with a truly
personalised education and enables teachers to make evidence-
based interventions."
§ https://www.century.tech/the-platform/

31. Folie 31
Ethical issues and challenges
Only 2 out of 146 included studies discussed ethical
issues associated with AIEd applications!
"All AI researchers should be concerned with the ethical
implications of their work"
Russel & Norvig (2010, p. 1020)
Russell, S. J., & Norvig, P. (2010). Artificial intelligence: A modern
approach (3rd ed). Upper Saddle River: Prentice Hall.

32. Folie 32
Intelligent Classroom Behavior Management System
https://www.businessinsider.de/china-school-facial-recognition-technology-2018-5?r=US&IR=T

33. Folie 33
Conclusion and implications for ODE
§ Huge student numbers in ODE, learning and teaching facilitated by
digital media = BIG DATA
§ High potential of AIEd applications along the student life-cycle
o Admissions, administrative services
o Intelligent student support systems
o Assessment for large classes: Calibration of AES systems
§ Overcoming the dilemma of economies of scale and
personalized learning and teaching in ODE?

34. Folie 34
Conclusion and implications for ODE
§ Dramatic lack of concern of ethical issues
§ Driven by computer scientists (especially from China)
§ Misunderstandings about the nature of learning (behaviourist
approaches: present – test – feedback)
Lynch, J. (2017). How AI Will Destroy Education.
https://buzzrobot.com/how-ai-will-destroy-education-20053b7b88a6.

35. Folie 35
Conclusion and implications for ODE
§ Gap between expectations and reality
§ Current state of AI in higher education is disappointing, no
application in higher education on a broader scale
"…there is little evidence at the moment of a major breakthrough in the
application of ‘modern’ AI specifically to teaching and learning, in
higher education, with the exception of perhaps learning analytics."
(Bates et al., 2020, p. 4)
Bates, T., Cobo, C., Mariño, O., & Wheeler, S. (2020). Can artificial intelligence transform higher
education? International Journal of Educational Technology in Higher Education, 17(1),
https://doi.org/10.1186/s41239-020-00218-x

36. Prof. Dr. Olaf Zawacki-Richter
Carl von Ossietzky Universität Oldenburg
Center for Lifelong Learning (C3L)
Center for Open Education Research (COER)
olaf.zawacki.richter@uni.oldenburg.de
@Zawacki_Richter
http://www.uni-oldenbur.de/coer/
Thanks for your attention!