Increasing cyber threats and changing NERC/CIP standards have caused Entergy to design and implement a new system for substation remote access. This system provides the access that engineers and technicians need, utilizes security best practices, leverages existing equipment, and is poised for future expansion and technologies.
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Substation Remote Access - Entergy Style
1. Substation Remote Access
Entergy Style
Chris Sistrunk, PE – RTU/SCADA SME
Sr. Engineer – T&D Technical Services
Entergy – Jackson, MS
9/26/2012
8th Security Summit
Portland, Oregon
2. Entergy SCADA
• Entergy has about 1600 substation RTUs
• 1500+ are “smart” microprocessor based
• Approximately 60 are “dumb” card file RTUs
• Approximately 500 Relay Communication
Processors connected to the “smart” RTUs
• Many IED types with several protocols
• About 98% of substations are serial only
8th Security Summit
Portland, Oregon
3. 1200 Baud to SCADAnet
• Most of Entergy’s RTU circuits are good ole’
Analog Leased Lines running at 1200 Baud
• ‘Ma-Bell’ won’t support forever
• OPGW, Digital µWave, Wireless, Leased T1
• Can support 4-wire to SCADAnet with same
telecom equipment
• SCADAnet uses hardened routers & switches
8th Security Summit
Portland, Oregon
4. Engineering Truth
“Engineering isn't about perfect
solutions; it's about doing the
best you can with limited
resources.”
-Randy Pausch, The Last Lecture
8th Security Summit
Portland, Oregon
5. via Dezeen
8th Security Summit
Portland, Oregon
6. A New RTU Standard
• Comparison of the major Comm
Processors/RTU/Gateways in 2008
• Management Directive: 1 BOX!!!
• Must be able to work with existing and future
substation designs
• I led Entergy-wide team that selected new
RTU standard in 2010
• KEY piece to moving toward IP connectivity
8th Security Summit
Portland, Oregon
8. A Hybrid Approach to SA
• New RTU is a flexible and upgradeable
solution that best met all of our requirements
• Migration path for existing RTU fleet
• HYBRID – more MPG for the Substation
– Old Stuff: 80% legacy relays, copper protocol
– New Stuff: SEL, IEDs, DNP, less copper
– New RTU can work with both
– Major building block for utilizing IP networks
8th Security Summit
Portland, Oregon
9. A Hybrid Approach to SA
SCADAnet
DA
Serial to Router
SCADA
Switch
RTU RTU Terminal Server
New RTU
New RTU
DNP
SEL 351 SEL 351
SEL 3 1
5
PMU
100% Serial BKR/XFMR
Monitor
8th Security Summit
Portland, Oregon
10. Challenges of a SCADA Engineer
8th Security Summit
Portland, Oregon
11. SUBCIP Project
• Started in fall of 2011
• Secure remote access to IEDs in the substation
• Old solution didn’t work – forced to roll trucks
• Must meet NERC/CIP standards
• Remember >>> Compliance != security
• Use new RTU with enterprise IED access
solution in a new remote access solution
8th Security Summit
Portland, Oregon
12. SUBCIP Project
• Implement NERC/CIP v3 at new sites by June
30, 2012 for Phase 1 & Phase 2 by June 2013
• We know SCADAnet is the future, but routable
protocols means locking cabinets or the entire
control house, which is a challenge
• Using only serial communications for SCADA,
engineering access, and file transfer will
eliminate CIP002-R3 CCAs
8th Security Summit
Portland, Oregon
14. SUBCIP Project: REAAP
• REAAP – Resilient External Access &
Authentication Project
• Provides a solution to address the need to
provide additional security controls for
external and remote access to Entergy’s
Energy Delivery process control environment
(e.g., EMS/SCADA) using additional security
controls for authorized employees and
contractors.
8th Security Summit
Portland, Oregon
15. SUBCIP Project: REAAP
• REAAP uses Two-Factor Authentication
– Hardened passwords
– Smart cards
• In addition to TFA, remote access is via a
virtual desktop environment
– Must use VPN if not on Corp network
– Virtual machines have security & virus scanning
– Short-term file storage for file transfers
8th Security Summit
Portland, Oregon
19. SUBCIP Project: Substation (No CCAs)
• Remote serial connection from REAAP Enterprise
system to RTU via channel banks
• 9600 Baud SCADA – 8X the bandwidth!
• Hardened Switch for SUB LAN & Future
• New RTU replaces old RTU and comm processors
• Relay techs only use serial in the Substation
– Zmodem (old school!) for file xfers to RTU
• Open USB & Eth ports are physically locked
8th Security Summit
Portland, Oregon
21. SUBCIP Project: Phase 3
• CIP v5 is on the horizon
• Some serial IEDs won’t be exempt anymore
from becoming CCA/BES Cyber Assets
• Roll out SCADAnet to IEDs where serial isn’t
sufficient or other requirements where IP is
more beneficial
• Implement automatic IED password
management & fault collection
8th Security Summit
Portland, Oregon
22. Final Thoughts
• SCADA Security isn’t easy
– Doing the best we can with what we have
• SCADA, Relay, & Security Labs
– Having a lab is so valuable for testing,
troubleshooting, breaking & fixing stuff
– Yes I have a fuzzer and I’m not afraid to use it
• DNP3/IP Secure Authentication v5
– Please tell your vendors you want it
8th Security Summit
Portland, Oregon