How higher education learning and teaching can learn from serious game developers. Keynote at the 5th annual SeGAH conference concurrent with WWW 2017 held in Perth, Western Australia
Analytic and strategic challenges of serious games
1. Curtin University is a trademark of Curtin University of Technology
CRICOS Provider Code 00301J
SeGAH 2017
Analytic and Strategic
Challenges of Serious Games
3. Learning Futures
Develops strategic innovations that advance the
mission of the university
Builds human and technological capacity
Leads and manages early stage innovation
projects :
formal and informal learning innovations,
pathways & partnerships, UniReady, learning analytics
Promotes faculty-based research and
continuous improvement using learning
analytics.
7. Aim of the Chair
To advance global knowledge, practice and policy
concerning the application of data science in the
transformation of higher education learning and teaching
toward improved personalization, access and effectiveness
of education for all.
8. Objectives of the UNESCO Chair
Data Science
Collaboration
Professional Learning
Expand Open Education
Ethical and Social
Impacts
Multicultural &
Interdisciplinary
Research
Open Assessment
Resources
9. Today’s Agenda
What do Serious Games teach?
Data challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
11. A Series of Interesting Decisions Wrapped
with Fun and Competition
Immersive affordances
Research-based learning progressions
Epistemic challenges and experiences
Unobtrusive assessment and immediate feedback
Which can teach-train-reinforce
Embodied Intelligence – Heuristics - Strategic thinking
Social Interaction Skills – Communication & Collaboration
12. Today’s Agenda
What do Serious Games teach?
Challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
14. Why New Psychometrics
There is a need for new frameworks,
concepts and methods for measuring
what someone knows and can do
based on game interactions and
artifacts created during serious play
Why? Ubiquitous, unobtrusive,
interactive big data created by people
working in digital media performance
spaces
15. New Psychometric Landscape
A “do over” for performance
assessment
New ways of performing & new
methods of data capture, analysis and
display
Complex tasks create evidence:
higher order thinking (e.g. decision sequences)
physical performances demonstrating skills
emotional responses
16. New Space for Performance
Unfold in time
Cover a multivariate space of possible actions
Assets contain both intangible (e.g. value, meaning,
sensory qualities, and emotions) and tangible components
(e.g. media, materials, time and space)
NOTE: Asset utilization during performance provides
evidence of what a user knows and can do
17. Performance Space Features
Unconstrained complex multidimensional stimuli and
responses
Dynamic adaptation of items to user, which entails
interactivity and dependency
Nonlinear behaviors with both temporal and spatial
components
NOTE: Higher order and creative thinking is supported in
such a space
18. New Analysis Perspectives
Hypothetico-deductive methods cannot induce or discover
a new hypothesis or rule to explain particular observations
Thus the need for data mining, network analysis, and
probability-based methods to augment IRT
Themes: Data-driven Science & Complexity
GOAL: To characterize and understand high-resolution
multidimensional time-based data
19. Types of Evidence
Intentional (e.g. constructed responses, whole documents,
short answers, writing, speaking, tool utilization, action
sequences, utilization of help or scaffolds)
Unintentional (e.g. gestures, utterances, eye movements,
affective states, time taken to respond)
20. Analysis Concepts
Segmentation of time into events, slices, episodes, activity
segments or n-grams, which are atomistic points for
aggregation
Segments must be recognizable in relationship to some
structure of meaning – attributes in some frame
Components of event structure are identified as situations
eliciting action-product learning trajectories made by users.
21. Adaptive Performance Assessments
Assessments can now be fully embedded and
synonymous with adaptive changes in the digital learning
environment
There is a thin line between formative and summative,
primarily differentiated by the purposes and audiences of
assessment
22. Analysis Challenges of a Multidimensional
Landscape
Time (e.g. historical preconditions, longitudinal
data, recurring patterns and autocorrelations)
Space (e.g. brain use patterns, neighbor
effects, socioeconomic topologies)
Scale (e.g. neurons to social communities)
Dynamics (e.g. unique behavioral profiles even
under highly similar conditions)
23. Today’s Agenda
What do Serious Games teach?
Data challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
24. Learning Theory Framework
for Software Agents
Content
that adapts
to group
and
individual
profiles
Agents for
selecting,
personalizin
g, organizing
& reusing
Agents for
translating,
reformatting,
time shifting,
monitoring,
summarizing
Agents for
critiquing,
“just in time”
feedback &
adaptive
testing
Community
AssessmentLearner
Knowledge
25. Learning analytics
Educational data mining (EDM) refers to the process of
extracting useful information out of a large collection of
complex educational datasets (Romero, Ventura, Pechenizkiy, &
Baker, 2011)
Academic analytics (AA) is the identification of meaningful
patterns in educational data in order to inform academic
issues (e.g., retention, success rates) and produce
actionable strategies (e.g., budgeting, human resources)
(Campbell, DeBlois, & Oblinger, 2010)
26. Learning analytics
Learning analytics use static and dynamic information
about learners and learning environments - assessing,
eliciting and analysing it - for real-time modelling,
prediction, and optimisation of learning processes and
learning environments (Ifenthaler, 2015)
27. Types of Analytics
The different types of analytics can be thought of as a continuum
with increasing value and difficulty. The type of analytics chosen
will be dependent on the business value of the problem.
Value
Difficulty
Descriptive
Analytics
Diagnostic
Analytics
Predictive
Analytics
Prescriptive
Analytics
What happened?
Why did it happen?
What will happen?
What should
I do?
28. Today’s Agenda
What do Serious Games teach?
Data challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
32. What is Complexity?
A characteristic of an agent or system
(complexity is not a thing, it exists via relationships)
Complicated, yes but more…
Surprising, yes but could be via simple rules…
Hovers (not sits) between chaos and order
33. Today’s Agenda
What do Serious Games teach?
Data challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
34. 34
Blackboard
StudentOne
Online Library
Cluster Method of Data Integration for Insight
Condense, classify and map
hypotheses
Surveys
• eValuate
• CASS
• CEQ
• School
Classifications
(constructed)
Conduct analysis &
interactively validate results
Model Construction
(Student Discovery Model)
Analytical Data Set
10 Sources
300GB Data
12 billion data
elements
51,181
Students
1273 Attributes
8 Clusters
Voice of Business, Voice of
Students and Voice of Data
External Data Sets
• Census
• SES indexes
• Geocoding
35. • A unit of one – the student
• Micro-segmentation is appropriate
• Techinique is Kohonen or Self Organising
Map (SOM) through the Viscovery tool
• SOM is a neural network
• 1273 attributes viewable on the map
• for each of the 51,181 Students in scope
• Map built from 274 simultaneously
considered attributes
Attrition StudentsUnderlying statisticsClusters Map International Students
Self-Organizing Map Model
eValuate Sentiment
+ 1270 more attributes
Blackboard Logins
39. Today’s Agenda
What do Serious Games teach?
Data challenges of new psychometrics
Learning analytics
Educational research with big data
Methods of educational data science
What higher education can learn
40. Learning for Tomorrow
Technology
Project SummariesCurtin University is a trademark of
Curtin University of Technology
CRICOS Provider Code 00301J
Page 9
The vision is for analytics to provide insight in domains that span the student experience:
Leveraging internal
knowledge and external
datasets together to provide
insight to economic,
population and industry
trends.
Generating an understanding
of students, their behaviours
and experiences to better
target, tailor and engage with
them.
Transformation of teaching
& learning content in
response to changes in
student behaviour, desires
and external factors.
Measurement and rapid
reaction to student
interactions, leading to a
dynamic adoption of best
practice teaching and
assessment techniques.
Leverage of rich university
datasets and analytical
skillsets to promote depth
and breadth of research,
innovation and knowledge
advancement.
Current LearnersCommunity of Future Students Community of Advocates
Market Analytics Curriculum Analytics Teaching Analytics Graduate AnalyticsLearner Analytics
Education Analytics Domains
learning experience; executed at a global scale.
Track and Assist the Learner’s Journey
The university needs to build staff leadership &
capacity in data-driven decision-making across
all of its delivery domains to promote game
based (challenge based) learning.