2022_03_22 EDUCON 2022 “Using e-Learning Standards to Improve Serious Game Deployment and Evaluation” - Baltasar Fernández Manjón

1. Using e-Learning Standards to Improve
Serious Game Deployment and Evaluation
Pérez-Colado, Ivan José; Pérez-Colado, Victor Manuel; Martinez-Ortiz, Ivan;
Freire, Manuel; Fernández-Manjón, Baltasar
balta@fdi.ucm.es
Dept. of Software Engineering and Artificial Intelligence
Complutense University of Madrid
EDUCON 2022 | 28-31 March 2022 | Tunis, Tunisia

2. Teachers experience
Deployment
Bad integration with real
systems leads to not being
usable in practice (results
gathering, fit in time, etc...)
Serious games use and deployment issues
Serious Games are effective
Teaching, creating awareness, changing behaviours, etc.
But there are several issues to address for generalization in schools
Effectiveness
Good as motivational
tools, but difficult to use
and to include in
curriculum
Monitoring
I would like to have more
control of what are my
students doing to assist
them properly

3. Game Learning Analytics (GLA) as a solution
Serious games were black-boxes but Game Learning analytics may help with it
Game Learning Analytics: analyzing in-game student interaction data
● Help understanding if students achieve certain educational goals
● Reveal learning process of students (in real time) as a white-box
● But GLA is still costly and requires more complex infrastructure
Deployment
Effectiveness Monitoring
Educational Standards
Common Technical Solution

4. Dev.
Using games in an LMS
Integrate games
in the classroom
Provide / deploy
games
Searching
Find / create
games
Engines /
Stores
Repositories
Authoring (tools)
Adapt Create
Technology
Evaluation
Web-Based Games Native Games
Game Learning Analytics
No results
Capture
scores
Capture
GLA
Analyze /
visualize
GLA
Tool
Assistance
Difficulty

5. The Learning Object standard evolution
Learning Objects evolved to an ecosystem of standards addressing those issues
➢ e-learning specifications can help serious games use and deployment too
Discovering Serious
Games
Finding
Accessing
Activities
Communication
Tagging Packaging Delivery
Report
Protocol
Results
Data
Model
Provide
Educational Standards
Integrate

6. Protocol
Data
Model
Report
Protocol
Results
Discovering Serious
Games
Accessing Activities
IMS LOM
- Adds Metadata
- Facilitates indexing
- Used in Learning
Objects
- Included in:
● IMS CP
● SCORM
IMS CP
- Stores content
- Structures
Learning Objects
- Simple to use
- Web-based
deploy
- Flash & Java
IMS CC
- More flexible
- Alternative to
SCORM
- External
content access
through LTI
IMS LTI
- Communication
for LMS and tools
- Forwards
students auth
- Can return info
- Simple link use
- No native
Tagging Packaging Delivery Results

7. Communication
Discovering
Serious
Games
Accessing
Activities
ADL SCORM
- All-in-one for LOs
- Includes LOM + CP
+ SS (sequencing)
- Only in Web
- Reports only
result-score pairs
ADL CMI5
- SCORM successor
addressing its limitations
- Web/native support
- Simplifies Learning
Analytics support
- Lack of adoption
ADL / IEEE xAPI
- Data model & API
- Flexible format for
interactions (actor,
verb, object & context)
- Domain profiles
=> xAPI for SGs
IMS
Caliper
- Share
similar
goals to
xAPI
- Less
adoption
Tagging Packaging Delivery
Data
Model
Report
Protocol
Results

8. Evolution of Learning Object standards ecosystem

9. Standard proposals for SG deployment
e-learning standards can ease game deployment with learning analytics
support for effectiveness and monitoring.
Considering the current adoption of standards to create realistic solutions.
We describe two approaches for two use cases:
● (Web) Hosting the serious game in the LMS
● Native serious games integrated in the LMS

10. Grading (black-box): SCORM / CC
Web based contents in WebGL (.zip)
● Engines:
IMS CP (recommended) / CC / SCORM
Learning Analytics with tracker and PINs
Hosting the serious game in the LMS: web games
Features:
● Simplicity and compatibility
● Simple grading and evaluation

11. Using and deploying native serious games from LMS
Features:
● All the previous + simple LA
● Solves CMI5 lack of support
Grades (LTI) & effectiveness (LA)
Native and web contents
● LTI connects to a mediator
● Mediator launches CMI-5 packages
Full Game Learning Analytics

12. Experiments: from authoring to deployment
uAdventure authoring tool
● Narrative and geolocated
gameplay styles
● Easy to use by non-experts
● GLA out-of-the-box
● Exports native and WebGL
Simva management tool
● Multi-purpose validation tool
● Manages experiments, data,
users, data-science
● GDPR with PIN system
● Integrated with uA (wizard)
We implemented both approaches as proof of concept with our open-code tools
● uAdventure helps with developing and provides packaging and xAPI
● SIMVA acts as a mediator connecting with LTI, storing traces (LRS) and doing a
CMI5 style launching

13. Second approach
Interoperable ecosystem for complete GLA lifecycle
Export IMS CP
WebGL content
Host in LMS Sends
traces to SIMVA
using PIN
Export IMS CP
WebGL or Native
CMI5 package
Hosted in SIMVA
Provides access
through LTI links
First approach
Clicking the link
from LMS opens
the game
configures and
sends analytics
Traces
Traces
Activity Monitoring
LRS

14. Discussion and conclusions
Game based learning is beneficial, but games must be easier to deploy in
the classroom and to integrate into the curriculum
e-learning Standards can help to systematize the process
Still Game Learning Analytics should be supported
To integrate learning analytics we can use different approaches
● Web(GL) content in IMS CP packages and individual PINs to identify traces
● Any content using CMI-5 but with mediator tools (+ SCORM / + LTI / + CC)
We have tested both approaches with uA and SIMVA

15. Thanks!
Contact: balta@fdi.ucm.es
This work has been partially funded by the Regional Government of Madrid (eMadrid S2018/TCS-4307, co-funded by the
European Structural Funds FSE and FEDER), by the Ministry of Education (PID2020-119620RB-I00) and by the
Telefónica-Complutense Chair on Digital Education and Serious Games.
https://github.com/e-ucm/uAdventure
https://github.com/e-ucm/simva
Our publications: https://www.e-ucm.es/publications/all