Some thoughts based on our practical experience, gained after years of R&D on the topic and hundreds of hours of training in our Cyber Range.

1. Dr. Jorge López Hernández-Ardieta
Head of Cybersecurity Solutions & Digital Specialist
Cyber Ranges: The (R)evolution
in Cybersecurity Training
Barcelona, 6 December 2016
Cybersecurity Unit

2. 2
Contents

3. 3
Contents

4. 4
Technology evolution
01. CURRENT SITUATION
Big Data/
Analytics
Smart X
BYOX/
Mobility
Unmanned
systems
Systems-of-
systems
Social
networksIoT/
Wearables
Blockchain
SDN/NFV
Cloud/
Virtualisation
(SaaS/PaaS/IaaS

5. 5
Technology evolution
01. CURRENT SITUATION
Big Data/
Analytics
Smart X
BYOX/
Mobility
Unmanned
systems
Systems-of-
systems
Social
networksIoT/
Wearables
Blockchain
SDN/NFV
Cloud/
Virtualisation
(SaaS/PaaS/IaaS
Interdependence & Interconnection

6. 6
Cyber threats evolution
01. CURRENT SITUATION
ATM/Bank attacks
First attacks
to phone
network Morris
worms
Massive
attacks to
EEUU phone
system
1900 1980 1990 20001970
Kevin
Mitnick
2010 20121930
Enigma
is hacked
Datastream
hacks
DoD, NASA,
USAF
Tenenbaum
Hacks
Pentagon
Anti-
sec
Conficker
Estonia
DDoS
Anonymous
Stuxnet
APT – Ghostnet, Night
Dragon, Titan Rain,
Shady Rat, Aurora
Worms CodeRed,
Nimda, Kornoukova,
Sadmind, slapper,
Iloveyou,
Mellissa,
Blaster, etc
2014
APT –
Careto
DragonFly
Ransomware
(mobile)
DDoS/IoT
2016

7. 7
The need for qualified professionals
01. CURRENT SITUATION
Constant evolution of technology and cyber threats require
constant efforts in professional education and training
Decision-makers should also be educated on risks and security
matters at strategic level
Qualified professionals are paramount for organisations to deploy
and implement effective cybersecurity practices
secure SW/systems engineers, network security engineers, incident
responders, malware & forensic analysts, security consultants, etc.

8. 8
 Current efforts and initiatives do not
suffice
 Knowledge entry barriers slow down
training process and increase costs
 Requires hands-on training: significant
trainer resources (high costs)
Our aim is to identify some
desirable properties that technology
should have in order to provide effective
massive-scale cybersecurity training,
detect which ones present
technical challenges, and suggest novel
approaches to achieve them
 Recent explosion in the demand (91%
increase in US 2010-20141)
 Expectations are ‘worse’: 6M until 20192
 Offer-demand imbalance: Lack of highly
skilled and trained cybersecurity
professionals
Problems
01. CURRENT SITUATION
2 Estimations by Symantec and CISCO reports (2014).
1 Job Market Intelligence: Cybersecurity Jobs, Burning Glass Technologies (2015)

9. 9
Contents

10. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 10
USABILITY
Easy access regardless when and where
(remotely) students access from.
Easy-to-use HMI and functionality.
ROLE ORIENTED
Adapt the training dynamics to the role
of the student (strategic, operational,
tactical).
REALISM
Information systems and communication
networks that reproduce real-world
scenarios with real-time feedback and
operation.
Hands-on approach.
GROWTH
Set up new exercises at a steady pace
(and cost-effective), according to the
evolution in technology and cyber
threats.
Desirable properties
02. CHALLENGES IN CYBERSECURITY TRAINING

11. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 11
CUSTOMIZABLE
Easily adapt and tailor the exercises to
the organisation’s needs, without the
need to stick to predefined scenarios and
exercises.
SECURITY
High security: isolation from production
environments, isolation between
exercises, access control, sound product
engineering, etc.
SCALABILITY
Support large networks with hundreds
and even thousands of assets.
Transparently accommodate new users up
to reasonable orders of magnitudes
(hundreds, thousands).
RICHNESS
Support a wide array of scenarios,
techniques, defensive and offensive
tools, attackers’ profiles, configurations
etc.
Desirable properties
02. CHALLENGES IN CYBERSECURITY TRAINING

12. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 12
SUPERVISION
Automatically monitor and assess the
student’s actions and performance.
GUIDANCE
Provide automatic guidance and hints
to the student to help him during the
training activity to enhance the
learning process.
REPRODUCIBILITY
Repeat, pause, resume and restore the
exercises at any time (student).
CONTROL
Automatically control the execution of
the exercise to know its progress as
well as state of the underlying network.
Desirable properties
02. CHALLENGES IN CYBERSECURITY TRAINING

13. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 13
ADAPTABILITY
Adapt the level of difficulty of the
training to the student’s skills and
performance, including dynamically.
Automatically and dynamically propose
new challenges to the student.
AUTOMATED ADVERSARY
Play automatically adversarial roles
(defender, attacker, ally).
PEDAGOGICAL
Embed a variety and effective learning
processes and pedagogical strategies,
such as:
 Observational learning (play
automated exercises).
 Trial and error approaches (active
attitude, capability to undo actions
and take different courses of action,
etc.).
 Quantitative scoring system and
gamification mechanisms to
encourage competitiveness and self-
improvement.
Desirable properties
02. CHALLENGES IN CYBERSECURITY TRAINING

14. 14
Contents

15. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 15
Cyber ranges
have become valuable tools for
civil and military organisations
Hands-on
training
01
Experimentation
and test of
technology and
cyberweapons
02
CDX Cyber
Defence
Exercises
03
Research and
validation of new
concepts and
technology
04
Cyber ranges
03. CYBER RANGES: A NOVEL APPROACH

16. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 16
A classical cyber range
03. CYBER RANGES: A NOVEL APPROACH
ESXi serversVirtual SMP VMFS
Storage
Network
infrastructure
Virtual machines
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
Physical layer
Virtual layer
Management
layer
vCenter – Management platform
Advanced functions
DRS HA vMotion
Servers

17. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 17
A classical cyber range
03. CYBER RANGES: A NOVEL APPROACH
...
OS
App
OS
App
OS
App
OS
App
OS
App
Redes
MZ
DMZ
Virtual Switch
(VLAN A)
OS
App
Virtual
Firewall
Virtual
IPS
OS
App
Target system Red Team
OS
App
OS
App
OS
App
Red
Ataque
Virtual Switch
Plataforma Ataques
(VLAN B)
OS
App
Firewall
Virtual
Exercise B
OS
App
OS
App
OS
App
OS
App
OS
App
Redes
MZ
DMZ
Virtual Switch
(VLAN A)
OS
App
Virtual
Firewall
Virtual
IPS
OS
App
Target system Red Team
OS
App
OS
App
OS
App
Red
Ataque
Virtual Switch
Plataforma Ataques
(VLAN B)
OS
App
Firewall
Virtual
Exercise A
OS
App
OS
App
OS
App
OS
App
OS
App
Redes
MZ
DMZ
Virtual Switch
(VLAN A)
OS
App
Virtual
Firewall
Virtual
IPS
OS
App
Target system Red Team
OS
App
OS
App
OS
App
Red
Ataque
Virtual Switch
Plataforma Ataques
(VLAN B)
OS
App
Firewall
Virtual
Storage & Backup
Appliance Backup
WBS
Dedicated
DataStore
NetworkAppliance®
NetApp FAS2040
(storage)
DataStores
VMware
Overland NEO-
2000
SAS
Virtual Switch
(VLAN D)
Vmware Virtual
Center
Management
computer
Management network (VLAN C)
HostESX-01 HostESX-02
Cluster (servers)
Physical
switches
External
access
Management

18. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 18
MATURE
GROWTH
SCALABILITY
SECURITY
REALISM
RICHNESS
USABILITY
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY
INCIPIENT
REPRODUCIBILITY
CUSTOMIZABLE
ROLE ORIENTED
Maturity level in state-of-the-art solutions
03. CYBER RANGES: A NOVEL APPROACH

19. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 19
A mere virtualisation
infrastructure
with some tailored
functionality does not
suffice
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH

20. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 20
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH
IDEAS
UI-level and low-level monitoring of
students’ and automated actions on
virtual infrastructure and application
artefacts, and their effects.
Match student behaviour against
optimal performance models.
Discover blocks/performance level
decrease, and act accordingly through
reconfiguration of objectives and
adversarial actions, and hints.
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY

21. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 21
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH
IDEAS
Bind objective achievements to
constraints (time, accuracy, others).
Logic to detect incompletion of
objectives and launch preconfigured
hints.
Possibly adapt score based on hints
consumption.
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY

22. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 22
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH
IDEAS
Metrics and measures to highlight
achievements and failures.
Link actions and events to educational
content.
Implement complementary approaches:
• Trial-and-error (checkpoints +
restoration).
• Observational learning.
• Scoring for competitiveness and self-
improvement.
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY

23. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 23
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH
IDEAS
Integrate expert systems capable of
taking on roles inside the exercises.
M&S for artificial users.
Reprogramme automated actions
based on student’s reactions.
CHALLENGE
CONTROL
ADAPTABILITY
GUIDANCE
PEDAGOGICAL
SUPERVISION
A-ADVERSARY

24. TRAINING IN CYBERSECURITY: CHALLENGES AND NOVEL APPROACHES | 24
Covering the challenges
03. CYBER RANGES: A NOVEL APPROACH
MATURE
GROWTH
RICHNESS
INCIPIENT
CUSTOMIZABLE
CHALLENGE
How to implement a cost-effective
and sustainable model that
ensures growth, richness and
customizable properties, while meeting
time-to-market demands?
i.e. objective = reasonable TCO
Sophisticated tools for
scenario generation based
around automation,
reutilisation and constantly
updated knowledge DB

25. 25
Contents

26. 26
We conclude…
Our experience…
04. OUR EXPERIENCE AND FUTURE WORK
5 years of R&D
Own product on the market: FEEP Cyber Range
+300 users in remote and on-site training sessions
+4,000 hours of hands-on training
Used in 2 large CTF events (CyberCamp 2015 and 2016)
Users appreciate fine-grained supervision and guidance
Tailored training is becoming a must
Automated (smart) adversary works well even for expert users
Metrics for user performance assessment are paramount

27. 27
Some real-time metrics
04. OUR EXPERIENCE AND FUTURE WORK

28. 28
Some real-time metrics
04. OUR EXPERIENCE AND FUTURE WORK

29. 29
Future work
04. OUR EXPERIENCE AND FUTURE WORK
Static intelligent attack scheduler as an exercise design tool
Dynamic intelligent attack scheduler to provider greater
intelligence for the automated adversary
SCADA/ICS exercises

30. 30
Dr. Jorge López Hernández-Ardieta
jlhardieta@minsait.com
THANK YOU!
QUESTIONS?