Learn how U.S. federal agencies have been working with AWS to expand their Continuous Diagnostics and Mitigation (CDM) programs to include AWS telemetry data. The Big Data Platform (BDP) is an accredited GOTS (government-off-the-shelf) cyber data platform to gather enterprise sensor data to provide full visibility and compliance across all federal agency network assets. While working through the AWS migration journey, AWS Professional Services worked with Enlighten IT Consulting to implement AWS native capabilities for BDP.

1. P U B L I C S E C T O R
S U M M I T
Washington, DC

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Continuous Diagnostics and
Mitigation (CDM) at Cloud
Scale
Mark Burr
Senior Consultant
AWS
3 0 2 9 0 2
Steve Goodman
Senior Director, Advanced Mission Support
Enlighten IT Consulting

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Agenda
The CDM mission
Sensors and data sources
AWS native telemetry
Big Data Platform
Scaling up and providing value

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Related breakouts
Security & Identity: the Continuous Mitigation &
Diagnostic Journey on AWS
Darren House and Evan Uhl
The AWS Playbook for Cloud Readiness & Large
Scale Migrations
Rodney Grilli and Christine Screnci

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
What is CDM?
The Continuous Diagnostics and
Mitigation (CDM) Program is a
dynamic approach to fortifying the
cybersecurity of government
networks and systems.
Provides Federal Agencies with
capabilities and tools to identify and
prioritize cybersecurity risks on an
ongoing basis, improve mitigation.

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
CDM capabilities
• Manage assets
• Manage accounts for people
and services
• Manage events
• Manage security lifecycle

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
CDM capabilities in detail
What is on the network?
• Hardware asset management
• Software asset management
• Configuration settings management
• Vulnerability management
• Boundary protection
What is happening on the network?
• Prepare for incidents and contingencies
• Detect suspicious events and patterns
• Respond to incidents and contingencies
Who is on the network?
• TRUST: manage trust in people granted
access
• BEHAVE: manage security related
behavior
• CRED: manage credentials and
authentication
• PRIV: manage privileges

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Multi-account approach
Developer
Sandbox
Dev Test
Team/Group Accounts
Security
Core Accounts
AWS Organizations
Shared
Services
Network
Log Archive Prod
Team Shared
Services
Optional Network Path
Network Path Log Flow
Data CenterDeveloper Accounts
Orgs: Account Management
Log Archive: Security logs
Security: Security tools, AWS Config rules
Shared services: Directory, limit monitoring
Network: Direct Connect
Dev Sandbox: Experiments, Learning
Dev: Development
Test: Pre-prod / Staging
Prod: Production
Team SS: Team Shared Services, AD, etc.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
AWS native logging sources
Service Type
AWS CloudTrail API calls
VPC FlowLogs Netflow
Amazon S3 Access Logs Bucket access
AWS Config Infrastructure change logs
AWS CLI Correlation for other log sources

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Additional AWS native logging sources
• Amazon GuardDuty findings
• Amazon Inspector findings
• AWS Systems Manager findings
• …

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Why the Big Data Platform?
• The cloud is the future, but no USG agency is 100% migrated yet. Hybrid
environments are a fact.
• Huge variety of network and host sensors, often with unique data
formats.

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Why BDP?
• Complex, heterogeneous, geographically-distributed networks are
difficult to monitor piece-meal

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Why BDP?
• Government needs collaborative incident response and investigation
while maximizing analyst resources

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Why BDP?
• A SIEM is the solution to these challenges, but commercial SIEM
software is expensive at petabyte scale

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Why BDP?
• DoD decided to build their own to control the spec and the costs. It’s
called the Big Data Platform (BDP).

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
What is the Big Data Platform?
The Big Data Platform is a US
Government owned, open-source,
RMF-accredited solution for large
scale data storage and analysis
COLLABORAT
E
DEVELOP REUSE

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Mission partners contribute capability to the BDP
• Joint funding between
agencies
• Community benefits from
lessons learned in common
scenarios
• Open source approach breaks
down information silos

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP Architecture
• Scalable architecture for ingesting and normalizing large datasets,
building analytics, and visualization

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP advantages
• Supports multi-tenant data
access
• RMF accredited for rapid
deployment
• Integrates easily with AWS-
native services like S3, EMR,
Lambda
• Large library of cyber analytics
• Open-source APIs for custom
integrations

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP CONOPS at a glance

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP Ingest of on-premises and AWS Telemetry

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP Ingest Parser Module
aws-flow-log:
...
parser:
description: ”Parse AWS VPC Flow logs."
class: bdp.ingest.parsers.csv.IndexableCSVParser
vis: "U&FOUO"
config:
delimiter-char: " " # optional
quote-char: """ # optional
skip-headers: false
mappings:
0: AWS_FLOWLOG_VERSION
1: AWS_ACCOUNT_ID
2: INTERFACEID
3: IP_SRC
4: IP_DST
…
ingest-info:
file-name: "parsers.file.name"
file-size: "parsers.file.size"
record-number: "parsers.record.number"
The feed name
Only the parser section
is shown
A description of the parser
The fully-qualified class name of
the parser
The default visibility for
produced records
The parser
configuration

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Scaling up and providing value
Collecting tens of billions of events per day from hundreds of thousands of
assets across hundreds of data sources

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Risk Management Scoring
• Evidence of Vulnerabilities + Evidence of Threat = Risk

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Supported CDM Data Sources
CDM Category Sensor Type
What is on the network?
AWS Resource Telemetry, Tenable Nessus,
McAfee EPO
Who is on the network?
Active Directory, IAM, Windows Event Logs,
Web Logs, SAML
What is happening on the network?
HTTP Proxy, Netflow, VPC Flow Logs, Bro,
DNS, Firewall, IDS

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
CDM + BDP
Unified Cyber risk management decisions can now be made using:
• Evidence of vulnerabilities provided by CDM sensors,
• Knowledge of threat from commercial and government feeds
• Evidence of threat provided by BDP analytics.

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
BDP + AWS
• AWS infrastructure is easy to observe
• BDP combines AWS observability with On-premise observability
• Answer “What is happening on the network?”
• Support continued migration to the cloud

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
Mark Burr
burrmark@amazon.com
Steve Goodman
sgoodman@eitccorp.com

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R
S U M M I T