1. This whitepaper assesses how modern security assessment’s fail as a means to assess Information Technology
(IT)/Industrial Control Systems (ICS), how cyber ranges work, and how the future of ICS cybersecurity depends on
the use of Cyber Ranges as a means of assessment.
SECURING OUR FUTURE
WHY TODAY’S SOLUTIONS CANNOT SOLVE
TOMORROWS PROBLEMS

2. 1
SECURING OUR FUTURE
WHY TODAY’S SOLUTIONS
CANNOT SOLVE TOMORROWS
PROBLEMS
INTRODUCTION
INDUSTRIAL CONTROL SYSTEM (ICS)s are control systems including supervisory
control and data acquisition (SCADA) systems, distributed control systems (DCS), and
other configurations using programmable logic controllers (PLC) to provide a desired
function, often in unauthenticated network environments. ICSs are a critical component
to important national infrastructure, yet ICS are many times forgotten in an organization’s
security plan and one of the biggest cyber threats. ICS have a history of making the
news when compromised and have real world consequences when penetrated and
exploited by a perpetrator. Examples such as the Ukrainian power utility hack left
225,000 people without power in March of 2015, prove the threat ICS hacks pose[1]. At
the Risk Management Summit, Applied Control Systems surmised of the 750 ICS hacks
reported, the financial cost has been $30 billion dollars[2]. ICS-CERT responded to 295
ICS incidents across a wide variety of industries in 2015 as indicated in Figure 1. A
IT/ICS. A cyber range includes hardware and software simulating and emulating a
system for operation and security testing and training. An IT/ICS Cyber range for testing
is crucial to ensure information assurance, safety, and correct functionality. As the Centre
for the Protection Of National Infrastructure stated, “Another significant advantage of a
KEY
POINTS
Industrial Control Systems
(ICS) impact almost every
aspect of life in America and
it is one of the Department
of Homeland Security’s
leading initiatives. Every
effort must be taken to
ensure its security. Cyber
Ranges represent the next
step in securing our
nation’s critical
infrastructure for
tomorrow’s threats.
Communications
5%
Government
Facilities
6%
Unknown
10%
Critical
Infastructure
44%
Critical
Manufacturing
35%
Incidents Responded to by ICS-CERT in
2015
FIGURE 1 INCIDENTS RESPONDED TO BY ICS-CERT IN 2015

3. 2
laboratory assessment is the ICS will be separate from the
production version. This fact means the team will have a green
light to non-destructively test any and all parts of the ICS
without the possibility of causing a real-world impact.”[4] Cyber
ranges allow for a more thorough and accurate assessment of
ICS without the fear of compromising the ICS. An IT/ICS Cyber
Range will allow for cyber analyst to test devices beyond what
they were designed for and determine what functions they are
capable of performing. Cyber ranges allow for a more
thorough and accurate assessment of ICS without the fear of
compromising the ICS.
AUDITS ARE NOT ENOUGH
An IT/ICS cyber range should be part of information security
programs because audits are inadequate to ensure the
systems are secure. Audits indicate if a security mechanism
is in place and configured according to industry standards
without specifying if the mechanism is effective. A router
SECURITY TECHNICAL IMPLEMENTATION GUIDE (STIG)
may tell a tester if administrators locked a router according to
Defense Information Systems Agency (DISA) standards. The
STIG neglects to tell the tester if the lock improves the security
posture of the system, resulting in systems locked to an arbitrary standard but no more secure as a result. STIGs and audits
such as DoD Information Assurance Certification and Accreditation Process (DIACAP) and Risk Management Framework
(RMF), fail to account for traffic and integration. CPNI states, “A secure ICS does not exist, which means that hidden
vulnerabilities are still possible in an ICS, even after a clean report from a cybersecurity assessment”[4]. For example, a
penetration test exploits the vulnerabilities in a system, as it exists compared to assuming an all-purpose approach such as
DIACAP and RMF. Pairing a penetration test on your ICS equipment in a cyber range gives you a more accurate assessment
of how secure your system is than an audit, which seeks to determine the status of security settings. Using a cyber range for
security assessments addresses these issues and allows additional testing including penetration tests without fear of
compromising the system and save millions by preventing significant loss of data or personally identifiable information (PII).
Auditors use tools; testing components of a system in isolation, but the tools fail to test how the system works when the system
is running. Auditors conducting an assessment use tools such as ASSURED COMPLIANCE ASSESSMENT SOLUTION
(ACAS) testing security settings, but disregarding the effectiveness of security settings. This is a flawed but accepted security
testing approach, because it lacks empirical evidence to support the assertion the system is secure. Imagine if you went to a
mechanic with an error code appearing on the console in your car and for some reason the car is smoking when you accelerate.
You tell the mechanic your issue and show him the error code. He tells you to come back in 2 days. When you return he tells
you he has resolved the error code, you pay your bill and leave. You get 2 miles away to find the error code returns and the car
continues to smoke. When you return to the mechanic, he tells you he resolved the error code but failed to crank the car to see
if the actual issue was resolved. Doesn’t make sense, does it? Only by testing the performance of the car can you have an
accurate idea of the performance. Modern tools are limited in their assessment of security because they fail to account for the
system operating. By using a cyber range and simulating the traffic a system will experience, your testing tools will give you a
more accurate assessment of the system. Testing the system using a IT/ICS cyber range allows integrated assessment,
Security Goals of Traditional IT versus ICS
IncreasingImportance
Confidentiality
Integrity
Availability
Availability
Integrity
Confidentiality
IncreasingImportance
Confidentiality
Integrity
Availability
Availability
Integrity
Confidentiality
FIGURE 2 SECURITY GOALS OF TRADITIONAL IT VS ICS

4. 3
reflecting how the system will function when operational where other methods ensure implemented settings without addressing
how the system works when it is operating.
Audits fail to address the effect traffic will have on the system, such as bottlenecks or capacity issues. Winters states, “A good
example of such tests is investigating shifts in traffic workload patterns. Adding new components such as workstations in an
office or new sensors or reporting thresholds in an industrial control system can cause unexpected critical traffic flow changes
in parts of the system quite remote from the location where the new components were added. This, in turn, can make a single
router in yet another part of the system a critical component.”[6] While security has been the focus of cyber ranges, ranges also
have tremendous implications for engineers in observing traffic in an operational system. While engineers can plan and design
for capacity, unless a system is operational the engineer cannot be sure the design has addressed the actual system bandwidth
requirement. Cyber ranges would allow engineers to model and test systems using realistic traffic while identifying where
potential issues may occur once a system is running.
ICS TOO CRITICAL TO SECURE
As ICS’ adopt Internet Protocol (IP) or similar protocols, the risk of hacking increases. Dell reported, “Dell SonicWALL saw
global SCADA attacks increase against its customer base from 91,676 in January 2012 to 163,228 in January 2013, and 675,186
in January 2014.”[7] This warrants security reviews, which may also pose a risk to the system due to the risk of an auditor
breaking the system during testing. As NIST reported, “The nature of ICS means that when an organization does a risk
assessment, there may be additional considerations that do not exist when doing a risk assessment of a traditional IT system.
Because the impact of a cyber incident in an ICS may include both physical and digital effects, risk assessments need to
incorporate those potential effects.”[8] Through various ways, a security auditor may break a system by accident. Tools have
design flaws with unintended operational consequences. One, now defunct, security testing application had a “Mitigate All’
button, which fixed identified security issues. The unintended consequence of locking these features resulted in a computer,
ICS attacks have seen a sharp increase
over the last few years due to the ease
of conducting ICS cyber attacks and
the growing number of attackers with
access to the tools and internet access
to conduct the attacks.
0
100000
200000
300000
400000
500000
600000
700000
800000
2012 2013 2014
Global SCADA Attacks
FIGURE 3 GLOBAL SCADA ATTACKS

5. 4
which would no longer function. Certain ICS’ being broken for a security test has the potential to cause serious injury. Mimicking
an ICS using a cyber range and testing it mitigates the risk of perilous harm.
Testers cannot test many ICS implementations because ICS functions are essential and testing risks shutting down the system.
As the Centre for the Protection of National Infrastructure states, “For example, several tools employed in such a test could have
a serious impact on the ICS itself. Various ICS’ will malfunction or halt completely when security tools, such as scanners, are
run on the network. Therefore, the asset owner and assessment team must understand the potential implications of testing on
a production system. Whenever possible, cyber security tests should be performed on a backup or offline ICS.”[4] At a scientific
research station in the Artic, testing would be life threatening if it broke the HVAC. However, this poses a security risk like the
2013 breach of Target’s payment system, initiated with the HVAC system, which seen in the diagram below. Auditor inability to
test ICS with traditional tools leaves few options to ensure maintenance of security. Creating a virtualized version and testing
security issues through a cyber range delivers higher assurance of system resilience to hacking.
When loss of human life is a non-determinant, like the Artic example above, for testing an ICS, a company’s bottom line may be
the issue preventing accurate security assessment. As Ashford reports “This means almost 100% availability is required, which
in turn means it is difficult and expensive to interrupt these systems for things like security updates.”[9] Ashford maintains when
detecting malware there is little that can be done because of the fear of breaking a system. “It is not uncommon for organisations
responsible for critical infrastructure to continue running control systems even though a malware infection has been detected.”[9]
In one example an engineer shutdown a bottling plant’s systems because they changed a timer for a maintenance controller;
resulting in a $100,000 loss for the company. Using a cyber range to test ICS’ security system strength avoids the cost of a
shutdown associated with a hack and reduces the risk of a shutdown because of testing.
BUT DOES IT WORK?
Cyber ranges are for security purposes but have tremendous potential for other forms of testing. OS updates, configuration
changes and patching in ICS remain an issue because of the difficulty determining how patches or upgrades may affect a
particular ICS. The Department of Homeland Security states, “As mentioned earlier, patch testing is of special importance in
control systems because of the requirement for very high uptime. The following recommendations should be included in patch
FIGURE 4 CAPTION TO BE ADDED
FIGURE 5 2013 TARGET POS BREACH

6. 5
testing: Test bed/simulation hardware should be dedicated for
testing purposes”[10]. Again if we use a little common sense to
the idea of testing we have all been the recipient of a patch or
upgrade to our operating system on our computer which had
unforeseen consequences. Cyber ranges address this issue
by allowing developers to determine how changes affect
functionality without applying the patch to production systems.
Developers could also test new security devices, determining if
they will damage system functionality without risking system
shutdown. This would include new hardware and software
tests against a system assessing how it would function against
a specific ICS environment. One important illustration
regarding a cyber range’s use to test new security hardware
can be found in Winter’s work (Figure 5) where he notes “A
recent example of such testing in the FCR found an intrusion
prevention device deployed in a system model that could be
made to fail open when subjected to the right kind of
overloading. It would simply give up and pass all traffic through,
good or bad. This is not something you would want to find out
in a real system under attack.”[5] This illustrates the
requirement to perform additional levels of scrutiny on products
before adding them to an ICS. In this case, a simple flood
attack resulted in a complete breach of the ICS although the
ant-intrusion device was intended to prevent intrusion into the
network. Security testing could also include unintended use of
ICS devices. Consider the Nest Thermostat, which has an Application program interface (API) allowing third party developers
to create new applications with the thermostat. What would happen if malicious manipulation occurred through the API?
Without simulating traffic, functional testing offers a limited
assessment of a system, lacking identification of unknown
traffic created issues. Cyber ranges allow new hardware test
and integration for specific environments and identify unknown
hardware issues pre-installation. Functional Testing follows a
script as indicated in the diagram to the right. These scripts
list the steps and procedures to verify the system functions as
expected, and can be automated or manual. The problem with
testing this way is it fails to account for traffic and therefore
how the system will function when operational. For example,
what if 100 users on a network attempt to access the same
resource at the same time. But the system is integrated in a
way allowing 1 user access at any given moment. Here we
can see where a cyber range is the only way to discover this type of bottleneck.
FIGURE 6 FAILED IPS DEVICE
FIGURE 7 RANGE BASED FUNCTIONAL TESTING

7. 6
MAKING IT ALL WORK
Cyber Ranges go through five phases of development. In the first phase, the system goes thorough documentation for the
Cyber Range to approximate the actual network. Details included during this phase would resemble the documentation included
in system accreditation packages such as the number of laptops, printers, versions of software and hardware. One important
aspect of the system captured during the enumeration phase is traffic analysis. This is accomplished with a mix of passive and
active network traffic analysis tools; striking a balance between collecting detailed traffic patterns and operational network
performance. Network SME’s make assumptions during the enumeration phase if the system is in development and traffic
patterns are unavailable. In the next phase of cyber range development, we reconstruct the system in a virtual environment.
Next, we replicate details gathered during the enumeration phase through virtual machines (VMs) of the target system. If
replicating a non-operational system, VMs can be produced which mirror the clients you intend to integrate. Once created,
settings are verified through a functional test in the 3rd phase, Testing. The testing phase resembles a functional test and
ensures the virtualized system functions. In the 4th phase, we model the target system by adding traffic to the virtualized network.
We do this by configuring traffic emulators with the data gathered during the enumeration phase. The fifth and final stage is
where the real value of the cyber range comes in. Sample uses of a cyber range include:
Red/Blue Exercises
Testing Hardware
Testing Software
Modeling and Simulation
Independent Validation and Verification
Research and Development
Tabletop Exercises
Comparative Solution Analysis
Integration Environment
Patch Testing
Load Testing
Configuration Testing
Functional Testing
Penetration Testing
Certification
Training
Hypotheses Testing
Team Assessment
FIGURE 8 CYBER RANGE DEVELOPMENT PHASES

8. 7
At Honeywell’s Cyber Solutions Lab, we have developed a cyber range for traditional systems and ICS solutions. With
Honeywell’s background in various markets and deep engineering experience, we are able to provide a cyber range meeting
the challenges of today’s cyber threats, and the threats of tomorrow. With minimal time from enumeration to emulation,
Honeywell’s cyber range allows system owners to stop making assumptions regarding security and secure them with a higher
level of confidence than ever before.
CONCLUSION
ICS are important to the nation’s infrastructure and yet ICS are some of the most neglected systems due to availability
requirements. These systems have become essential to our way of life and the risk of examining the system to ensure its
stability and security present too much risk to the system. IT/ICS Cyber ranges offer an affordable risk-based approach to
securing IT/ICS; enhancing overall security posture of the system in a way that is impossible with other testing methodologies.
Honeywell’s Cyber Range meets the demands of today’s customers and addresses tomorrow’s challenges.
Courtney “Brock” Rabon is Honeywell Technology Solutions Inc. (HTSI)’s Cyber Evangelist and has 11
years of experience helping Commercial and Federal clients meet their cyber security goals. He manages
their Cyber Security Technologies Lab in Charleston, SC and can be reached at
courtney.rabon@honeywell.com.
HTSI is a diverse professional and technical services leader offering world-class managed solutions to federal, commercial and
international clients. HTSI’s core capabilities include engineering and space operations, physical and cyber security, engineering
and development services, logistics, facility and equipment planning, and testing and calibration.

9. 8
BIBLIOGRAPHY
[1] “iTWire - Darkness in the Ukraine – hackers turn the lights off.” [Online]. Available: http://www.itwire.com/business-it-
news/security/72709-darkness-in-the-ukraine-%E2%80%93-hackers-turn-the-lights-off.html. [Accessed: 10-Jun-2016].
[2] “Industrial_Control_Systems_at_Risk1.pdf.” .
[3] “Year_in_Review_FY2015_Final_S508C.pdf.” .
[4] “2011020-cyber_security_assessments_of_ics_gpg.pdf.” .
[5] “IEEE Xplore Full Text PDF.” .
[6] “Attacks Against SCADA Systems Doubled in 2014: Dell | SecurityWeek.Com.” [Online]. Available:
http://www.securityweek.com/attacks-against-scada-systems-doubled-2014-dell. [Accessed: 10-Jun-2016].
[7] “NIST.SP.800-82r2.pdf.” .
[8] “Industrial control systems: What are the security challenges?,” ComputerWeekly. [Online]. Available:
http://www.computerweekly.com/news/2240232680/Industrial-control-systems-What-are-the-security-challenges.
[Accessed: 13-Jun-2016].
[9] “RP_Patch_Management_S508C.pdf.” .
[10] H. Winter, “System security assessment using a cyber range,” in 7th IET International Conference on System Safety,
incorporating the Cyber Security Conference 2012, 2012, pp. 1–5.