The document summarizes the key findings from analyzing text from NATO workshop reports on serious games and military training over time. The analysis used topic modeling, latent semantic analysis, and clustering to identify trends. Key findings include that scenario realism improved alongside discussions of social media and human factors. New requirements took about 1.5 years to emerge as solutions. User experience topics have persisted in discussions. Assessment of training outcomes has recently emerged as a focus. Qualitative trends across technology, industry, and government are also summarized.

1. Behind the
Page:
Text Analysis
Uncovers Trends
011
01000111101010
100100100100100100100
111010010100100100101
101000101010101001010
010100101001001001101
0010110
Jay Gendron, SimIS Inc.
www.linkedin.com/in/jaygendron
Hampton Roads, Virginia  April 15-17, 2014

2. Consider…

3. The Plan
• Understand the Problem (Use Case)
• Look at Past Predictions. Met?
• Analysis – speed boat, not SCUBA dive
• Results of Analysis
• Trends in Commercial Technologies and
Gamification

4. Problem (NATO Use Case)
Management Questions
• What are trends in industry and gamification?
• Have we met the predictions?
• How do we know this?
2003
Small Biz…
2008
Predictions
2013
Now What

5. Predictions
1. Highly realistic scenarios would become
affordable those outside military training
2. Training games would be become more
realistic than those for entertainment
3. New types of training scenarios would drive
new types of training technologies
4. Joint missions would drive interoperable
solutions
5. End-user experience would command greater
emphasis in development processes

6. Analysis Plan
Workshop Date and Location Page Count
MSG-074 May 18 – 20, 2009; Farnborough, UK 12
MSG-078 September 22-24, 2009; Suffolk, US 4
MSG-093 November 2-4, 2010; Rome, IT 12
MSG-108 October 25-28, 2011; Farnborough, UK 24
MSG-113 April 16-18, 2012; Genoa, IT 31
MSG-114 November 13-15, 2013; Kjeller, NO 35
MSG-130 October 9-11, 2013; La Spezia, IT 39
Topic
Model LSA/SVD
Cluster
Analysis
Topic
Definition
Topic
Traces

7. Topic Model
ID Importance Topic Keys
A 1.34305
mr nato game technologies learning provided development dr system
immersive buck virtual presentation education games examples commercial
technical world
2 0.04586
italy genoa immersive information university bruzzone fear diptem scorm
thurkettle lms april plans web united procedural miss busetta belief
3 0.09772
maritime navy things data people domain includes noted naval threats
series number effects platform cmre knowledge team large case

8. Latent Semantic Analysis (LSA)

9. Singular Value Decomposition

10. Clustering

11. Topic “Gut Check” from Clusters

12. Topic Definition – People Power
Topic ID: 1 Topic: Immersive Learning Environments for Education and Training
Topic Keys:
italy,genoa,immersive,information,university,bruzzone,fear,diptem,scorm,thurkettle,lms,april,plans,web,united,p
rocedural,miss,busetta,belief
Topic ID: 2 Topic: Assessing Pedagogical Effectiveness of Maritime Training
Topic Keys:
maritime,navy,things,data,people,domain,includes,noted,naval,threats,series,number,effects,platform,cmre,know
ledge,team,large,case
Topic ID: 5 Topic: Social Media Dimension of Gamification
Topic Keys:
gave,worlds,garcia,learners,aims,free,simulation,gamification,crowd,johnny,november,digital,cdr,social,traditio
nal,technology,supervision,unlimited,coffee
Topic ID: 6 Topic: Future of Commercially Available Games in Military
Applications
Topic Keys:
mosbe,bluefor,dive,civilian,lto,big,test,actual,fit,subject,simon,sensurprys,summation,versus,programme,strategi
es,back,belief,insurgency
Topic ID: 8 Topic: Human Dimension of Serious Games Implementation
Topic Keys:

13. Topics Overlaid on Cluster

14. Finding 1: Scenario realism continues to improve and
shows an association with the level of discussion of social
media and the human dimension during workshops
Recommendation: Future events should strive to balance
discussions on technological and human dimension topics by
considering different use cases at each event.

15. Finding 2: Discussions focused on solving new
requirements tend to lag the introduction of the topic by
three workshop events.
Recommendation: Workshop planners should use an outreach
plan to align topic introduction with desired solution timeframe
and assume an 18-month lead-time for a trend to emerge.

16. Finding 3: Discussions focused on user experience have
emerged within the workshop reports and persist in current
discourse
Recommendation: Process topics seem a likely evolution from
technology to people to process. Harness user experience
discourse to highlight process-oriented games and training
requirements.

17. Finding 4: Assessment of training effectiveness, return on
investment, and time efficiencies recently emerged as a
workshop topic and this will likely continue
Recommendation: Assessment includes topics ranging from
pedagogy to return on investment to time savings. Extend calls for
papers on these topics for future workshops.

18. Qualitative Trends
Technology Industry Government
 Massive Open
Online Course
(MOOC)
 Mobile Learning
Technologies
 Alternate Reality
Games (ARG)
 Networks and
Architectures
 Open Source or
Proprietary
 Understanding the
Needs of Government
eLearning
 MOOCs as a
Disrupting Technology
 Standards for Gaming
 Collaboration
 Architecture for
MOOCs
 Augmented Reality
and Virtual Worlds

19. Summary
• Management
questions on ROI
• Predictions stated
• Application of
humanities
approaches
• Supported ROI and
predictions

20. Consider…
• Text Analysis
combined with
clustering can see the
unseeable
• Models may also
support predictive
analysis
• All work done with
open-source tools