1. S4 ICS Security Conference 2015
Accelerating OT Cyber Security - Case Study
Craig Heilmann, CISSP, CRISC
Global Lead, Critical Infrastructure Security Services
IBM Security Services
January 2015
2. S4 ICS Security Conference 2015
Sticky Bombs
Takeaway
2 IBM Security
Note to S4 slide reviewers: The reference is an attention-getter. Saving Private Ryan sticky bombs. This will carve a
takeaway into memory. “If you remember only one thing from this session, remember sticky bombs.” Explosives +
socks, coated with grease. A blunt response, but when used at the right time was effective against a high tech,
sophisticated attack. This is the theme really of the entire session … using our IT and OT capabilities we have today,
low tech and high tech, in rapid and effective ways to counter the high volume of persistent and sophisticated attacks
… and a case study to show how it is done.
3. S4 ICS Security Conference 2015
Regardless of industry, the necessary shift in security paradigm
needed to “fight the fight” today boils down to fundamental themes:
Security Requirements
3 IBM Security
Capability
• All about visibility and control
• More about process than technology
• Objective to disrupt the attack chain (not to be 100% breach-free)
Capacity
• More leverage for skilled resources
• Greater reach and scalability
• Working smarter not harder
Acceleration
• Reducing the time to detect
• Reducing the time to respond, contain and recover
• Reducing the time, effort or cost to transform
4. S4 ICS Security Conference 2015
Through this lens, let’s look at a recent and typical case study:
Case Study: Introduction
4 IBM Security
Client
• Multi-billion dollar manufacturer with global operations
• Long history of acquisitions leading to fairly autonomous business units
• Highly automated via extensive industrial control systems on the plant floor
• Considered critical infrastructure due to strategic nature of products and processes
Capability
• No SOC, heavily reliant on static perimeter defenses (firewall, IDS, …)
• Just beginning to deploy IT security and event monitoring (SIEM)
• Disconnected from OT (as well as telecom and physical)
• Ad hoc incident response and no IR Plan (heroic efforts of a few)
Capacity
• Few security resources; sharp troops but bogged down in daily manual tasks
• Limited security budget (historically 1~2% IT spend)
• No strategic partners (various local small players depending on geography)
Acceleration
• Desire to mature and transform but not clear where to begin
• Pressure from Board to show “results” quickly
5. S4 ICS Security Conference 2015
The client in this case study created a vision behind a 5 year plan
that would transform and modernize their security organization.
Case Study: Future State
5 IBM Security
Old
Paradigm
New
Paradigm
Security
Model
based
on
Defense
in
Depth
Security
Model
based
on
Rapid
Detec7on
+
Rapid
Response
Security
Opera4ons
Steady
State
and
Reac7ve
Security
Opera4ons
Elas7c
and
Agile
Governance,
Risk
&
Compliance
IT
and
Compliance
Focused
Governance,
Risk
&
Compliance
Enterprise
Risk
Management
Func4onal
Domains
IT,
OT,
Telecom,
Physical
Silos
Func4onal
Domains
Converged
Security
Analysis
Manual
and
Fragmented
Security
Analysis
Analy7cs
and
Intelligence
6. S4 ICS Security Conference 2015
Great vision, but the constraints seemed likely to stall out the plan
before it even got started.
Case Study: Constraints
6 IBM Security
§ Very limited budget
§ Culture resistant to security controls
§ Must show impact and results quickly
§ Only a small increase in headcount approved
§ Fighting tight market for security skills (unable to fill open reqs)
§ Directive to accelerate improvements in OT security
§ Pressure to pull forward much of the 5 year plan into a 3 year plan
7. S4 ICS Security Conference 2015
The solution was to develop an incremental plan, beginning with a
focus on operations where the most impact could be achieved with
the least amount of upfront spend:
Case Study: Solution Step One
7 IBM Security
Capability
• Inventory existing technologies and processes and optimize against attack chain
• Deploy one new technology (password vaulting) to enable rapid password changes
• Leverage NOC in short term with plan to outsource SOC long-term
• More SIEM logging and extend into OT environments (and protocols)
• Select global strategic partner for IR; co-develop IR plan
Capacity
• Dedicate strongest security resources to strategy, policy and oversight
• Retool and cross-train where possible; staff aug and outsource others
• Invest in external security intelligence and early warning providers
• Managed device administration with long-term transition to MSS
Acceleration
• Culture change management via governance restructuring, training and communication program
• Optimize technology and processes to detect faster and respond faster (and more effective)
• Analytics and automation in the area of SIEM (correlation and behavioral analysis)
8. S4 ICS Security Conference 2015
This new “Elastic and Agile” operating model looks like a stair
stepped response plan, throwing “big levers” that involve processes,
operations and technology.
Case Study: New Security Operating Model
8 IBM Security
9. S4 ICS Security Conference 2015
More than incident response and threat management, this approach
moves much bigger security levers designed to more substantially
disrupt, frustrate or stop modern attacks.
Case Study: New Security Operating Model
9 IBM Security
WHY – because most attacks need credentials
§ Identity and valid user credentials are crucial to most attacks.
§ Changing passwords is one of the top three remediation activities during and
after a breach, and often a wise precautionary activity to preclude an attack.
WHAT – all passwords for all accounts, everything
§ All passwords; users, administrators and service accounts in IT and OT
§ For many organizations this can be 100,000+ accounts.
§ Service accounts because attackers love them; ideally several of them that
have domain privileges and are hard-coded into custom critical business
applications.
HOW – in one 36 hour event
§ Must be done in one swift blow, typically over a weekend within a 36 hour
period
§ It takes most medium to large organizations 3 to 4 months to prepare for, plan
and finally execute this task.
§ A lot of house cleaning in Active Directory must occur. A lot of custom code
and even some vendor proprietary code must change to remove hard-coded
service account names and passwords.
§ Users must be notified. Business application owners and partners and
vendors are impacted.
§ And then the actual event, scheduling downtime and bringing down the entire
environment, changing passwords, and bringing it all back up – similar to a
DR exercise.
New Approach – turn a weakness into strength
§ Don’t wait for a breach that causes you to coexist with an attacker for 3-4
months.
§ Do the house cleaning today.
§ Work with the business to cleanup the application portfolio today.
§ Develop a procedure for an enterprise-wide password change.
§ Understand what criteria might trigger this response.
§ Train the business and train the users.
BENEFIT – disrupt and stop attacks in their tracks
§ Attackers are counting on your inability to respond in this fashion.
§ Creating levels of lockdown that package this capability with others like more
restrictive physical security access control, throttling the number of SOC
analysts’ “eyes-on-glass”, throttling the sensitivity of what constitutes
“suspicious” activity and so on disrupts and stops attacks.
§ By “operationalizing” these kinds of capabilities, you are involving the
business from the beginning; working out issues with validated systems, legal,
compliance, change control and a myriad of other related issues and
concerns well ahead of a crisis.
§ Everyone understands their part, understands the impact to them, and
understands the criteria that dictate the response.
§ Security becomes the responsibility of everyone, not just the security
organization.
Example: Consider an enterprise-wide password change …
10. S4 ICS Security Conference 2015
As designed, the new operating model is more of a program with a
framework and lifecycle, enabling continuous adaptability and
maturation.
Case Study: New Security Operating Model
10 IBM Security
Initial
Program
Setup
Security
Model
Gap
Record
Test
Results
Program
Refresh
Security
Model
Gap
Record
Test
Results
. . .
Levels 0-2 Levels 0-3
• Treat as POC
• Use existing inventory
• No net-new deployments
• Focus on optimization
• Focus on change and education
• Deploy some new tech
• Fill high priority gaps
• Fix high-priority test findings
• Implement budgeted and planned changes
• Adapt model with new attack scenarios
Might only have two alert
levels at first – that’s okay …
… and MANY
gaps identified,
programmed for
future mitigation
More maturity, capability and
flexibility may warrant more alert
levels over time …
… but gaps should reduce,
ideally to zero backlog
Timeline
11. S4 ICS Security Conference 2015
A collateral benefit of the approach enabled a quantifiable and more
predictable method for cost modeling and budget allocations,
rationalizing spend and pulling investments forward.
Case Study: Cost Modeling
11 IBM Security
Steady State / Level-Zero Cost
Level-
Dependent
Variable
Cost
Operating Budget = Level Zero “annual cost of business as usual”
+ (# of Level 1 events) x (Level 1 run rate) x (average duration)
+ (# of Level 2 events) x (Level 2 run rate) x (average duration)
+ (# of Level 3 events) x (Level 3 run rate) x (average duration)
+ (# of Level 4 events) x (Level 4 run rate) x (average duration)
12. S4 ICS Security Conference 2015
A post-deployment analysis identified several additional benefits of
the approach:
Case Study: Additional Benefits
12 IBM Security
§ More confidence at executive levels in ability to defend against attacks
§ Highly visible to the Board, the business and users
§ Security training more relevant and taken more seriously
§ Tighter integration between IR, DR, Safety, and other response plans
§ Clarification of security governance and responsibilities
13. S4 ICS Security Conference 2015
Question and Answer
13 IBM Security
Q&A
Capability
Capacity
Acceleration