Development of adaptive microlearning apps for testing competence in multiplication tables, educational data mining edm, learning analytics

1. Martin Schön Martin Ebner
Life Long Learning Social Learning
Graz University of Technology Graz University of Technology
Mandellstraße 13, A-8010 Graz Münzgrabenstraße 35A/I, A-8010 Graz
+433168734931 +433168738540
martin.schoen@tugraz.at martin.ebner@tugraz.at
Why micro testing is a necessity for teaching
and learning.
Learning analytics as teacher’s assistant.
Microlearning 6.0 6th International Conference 2012, July 9th & 10th

2. Content
Or: this is a story,
b) how we began to think about an educational
problem (multiplication tables)
c) how we tried to solve it with a small
application
d) and how this leaded us to Learning
Analytics & Educational Data Mining

3. Idea
At the Graz University of Technology some students of Martin Ebner had
to develop applications during their studies of information technology
(Master of Science).
Task: Simulate a diagnostic situation: A student is introduced to a teacher.
He/she tries to gain an overview of the tasks which the students can do
reliable well. This should be done in an optimal way.

4. Task
• The application should test the competence in practicing
the multiplication tables
• The system should provide appropriate tasks according to
the competence grade of the learner.
• The system should ensure that already well-done exercises
are repeated and practiced. After succeeding a problem the
probability for a repeated display should decrease (similar
to a Leitner System )

5. Checklist cont. 2
• Nevertheless the tasks should tend to be challenging.
• In general the system should be motivating and show that
learning can be fun.
• The system should record and safe fine-grained data of all done
exercises, test results and the current competence grade of the
learner in order to prepare the next sessions in an adequate way.
• It should relieve the teachers from the unsolvable task of
remembering all data from every child – it should be user as an
information processing tool
• That means the teachers should get help and no additional time
consuming tasks

6. Checklist – cont. 3
• during this spring we defined another goal:
We wanted to give teachers the opportunity to set up and watch
a whole class - Today we are able to present a database for
organizing schools, classes and individuals – with an interface
for the conventional desktop browsers and also for Android or
iOS operating mobile devices.
• the web-application is written in php/mysql –
• the apps in objectiveC (iPhone/iPad) & JAVA (Android).

7. Measurement
• the goal is to generate a complete table to inform learners
as well as teachers in a deterministic way about the
competences in every single task, in every single
multiplication fact
This is opposite to our experience in interviews:
• Teachers talk about quantity:
“I have 4 pupils who …. And 6 who … ….

8. Algorithm
The presentation of the tasks is not only
randomly generated..
At the very first contact a moderate
problem is presented. According to the
results more or less difficult tasks are
following.
The competence level of the test subject
is afterwards estimated and after every
task recalculated. This determines the
difficulty of the next tasks.

9. Competence level
1. estimate the competence level:
2. After every new solved or unsolved problem we choose a
smooth way for adopting this first estimation of competence
to the experience during the sessions

10. Difficulty
• Without any empirical data we decided to use the ranking for
the tables as follows (You´ll find this hierarchy in most
didactical concepts):
easy …. 1, 2, 4, 3, 5, 8, 6, 7, 9 … difficult
• we transformed these ranking in difficulty levels between
1 and 0.
• We discussed to integrate a statistical founded ranking, but at
this point we fear, the teachers would be somewhere confused
about the results. I’ll speak about this problem later.

11. Stored Data
• The answers of the learners are marked with 0,1 or 2:
• 1 shows that the user knew the correct answer once
• 2 indicates that the student had two consecutive correct answers
(we say a question is “well known”)
• 0 indicates, the last answer was incorrect (or this item was
presented never before)
Additional stored: a time line with needed time, results, task no., ever
presented,

12. Selecting the next item
We use a random number between 0 and 1 to decide, which
category is activated to generate the next multiplication
problem: Therefore three cases are defined:
• Case 1: If the random number is 0<x<0.05 than a well-known
question (marked with 2) is chosen.
• Case 2: The random number is 0.05 > x >= 0.15 than a known
question marked with 1 is chosen.
• Case 3: The random number is x > 0.15 than an unknown
question out of the extended and actual learning area is chosen.
These parameters can be changed for the total system, not for
Indivuduals

13. Extended Area
• Extended area means the idea, that we
choose items not only in the learning area,
that means under the level of competence.
• To produce some dynamic and the chance
to get a higher level, we add now 0.15 to
the actual estimation of the degree of
competence (0 …1) of the student

14. Prototype

15. Study
• first research study was carried out at a primary school in Austria.
• Begin: summer semester 2011
• 43 pupils of the primary school Laubegg (age: 9-10).
• at least 4 weeks. Some of the learners ignored this time restriction and
played the game again and again over months.
• Learners learned on computers at the school as well as on their
personal computers at home.
• 12.926 answers where given which means that on average each learner
answered 308 questions- they did 3.4 times the whole multiplication
table.
• Bearing in mind that there was no real pressure from teacher’s side
using the program it is a considerable pleasant high number.
• Furthermore it can be stated that pupils seemed to enjoy using the
application or at least get not bored.

16. Highscores

17. Demotivated learner
One learner with a weak performance
attracted our attention because of a
very high number of trials (513). A
detailed inspection showed that he did
not work very intensively. In the first
two tasks he/she failed, then eleven
tasks were ok, his performance rose
abruptly. However, afterwards, he
continued approximately 400 times to
wait the whole answering time
without doing anything but asking for
a new assignment.

18. Motivated captain
The most diligent learner.
In the beginning, the
assignments were solved
correctly, then some mistakes
occurred, afterwards a
learning process can be
recognized and finally with
some occasional mistakes the
learner works on a high
performance level.
Obviously, the learner was
highly motivated to deal with
the assignments given by the
program.

19. Medium learner
In the beginning, the learner made
mistakes in every second
assignment (0.5), followed by 7
mistakes consecutively. This is the
reason for the big decrease (0.15).
Afterwards, the learner gave a
number of right answers and the
rate of correct answers increased
back to 0.5. In the following phase
an up and down can be seen till a
number of right consecutive
answers helps to reach a level of
0.7. But then the number of
mistakes rose again and the rate
went down to about 0.5.

20. Medium learner 2

21. 0
2
4
6
8
10
1
11
21
31
41
51
61
71
81
91
101
Weak learner
111
121
131
141
151
161
171
181
Classified as 2 id156
191
201
211

22. We don‘t know (now):
More than half of the learners did not reach the 100 percent level (= 90
items are well-known). Therefore we have to think about this group of
learners: Perhaps they
• didn´t get used to / have problems with the interface
• do not know the necessary operations; are not able to solve
the learning problem correctly
• misinterpret an assignment
•are distracted by the environment, are badly concentrated for several
reasons.
-> Could we gather more information with additional devices to know
more?

23. Educational Data Mining
• At this point, after all the preparations and programming, with
the first incoming data we began to realize that we had taken a
step into a whole new age.
• We had begun to program and improve a simple cybernetic
loop. At the beginning we saw just the necessity to store the
performance data. Now we find us in the situation, that we
want to collect “everything” – We want to store all tracks left
by a learner. And we want even to produce more data, even
in that we use additional instruments and sensors.
• We realized that this idea of collecting more and more data
implies completely new perspectives to perceive and to
reconstruct some learning processes (not any!)

24. Learning Analytics
Learning analytics is an educational approach
aimed to improve education and facilitate learning
by the systematic analysis and interpretation of digital tracks.
Through the use of more and more intelligent ubiquitous
information technology for communication and organization,
more and more events are created for leaving digital traces.
With the increasing data floods also the importance of this
paradigm increases.

25. Interim conclusions
• It is as easy as never before in history to collect data
• We perceive much more details about the learners – the learning
process – as never before!!
• Teachers get precise information
• This application was designed for diagnostic purposes, to get an
overview on problems with multiplication facts. Now we see, it
can be effectively used for training and facilitate learning

26. mathe.tugraz.at
Our latest design:

27. Informationprocessing
• The matrix at the left shows the history: The beige colour
indicates “well known” results, the brown is “one right
answer”. The rabbit comes from left to right to the carrot
and catches one - if the pupil produces consecutive right
answers.

28. Details
If you point over with the mouse on
the matrix, you see the
multiplication fact behind the
symbol.

29. Register (it`s simple!)
If you want to test this application you can use “no
school” for an individual registration.
If you want to administer a whole school and a
couple of class lists … contact
martin.ebner@tugraz.at to send you an
administration-account.

30. Analysis
I´ll show here some graphs of the collected statstical data.
We made the experience, that the teachers in our interviews
are interested in very short compact information, overviews.
Therefore the following graphs are only for scientifc purposes
We know, that if we would produce too much information many
teachers will ignore the entire system ….

31. New findings
This new applications are public available since March 2012.

32. Session time (s)

33. Well - known
Perhaps someone is interested to get more information on the
difference of signing the tasks with „known“ and well known:

34. Results for teachers
If the analysis shows more than 12 correct answers during the last 20
tasks the traffic light will show green, with less than 5 correct answers it
is indicated red, otherwise the student and of course also teachers will see
yellow. In retrospection to the current situation this would lead only to
5% red and yellow alerts.5% red and yellow alerts.

35. Future Work
• closer look at the learners: perhaps more
intelligent analyses of the data? – more data
– more concepts? Hints for Learners?
• Mathe-Multi-Trainer (multi digit
multiplication)
• Cooperative Learning
• Test and Training in Reading

36. Acknowledgements
We express our gratitude to the teachers of the
primary school in Laubegg (Styria, Austria) as well
as all participating school children.
We are equally indebted to our funding agency
“Internet Foundation Austria (IPA)” for supporting
our ideas and helping us with netidee to work on
the future of education.

37. Thank you!
Martin Schön
TU Graz – Austria
martin.schoen@tugraz.at