Cloud Native platforms such as Kubernetes and Cloud Foundry help developers to easily get started deploying and running their applications at scale. But as this access to compute starts to become ubiquitous, how you secure and maintain compliance standards in these environments becomes extremely important. In this talk we'll cover the basics of securing Cloud Native platforms such as Kubernetes. We will also cover open source tools - such as Clair, Anchore, and Sysdig Falco - that can be used to maintain secure computing environment. Attendees will walk away with a good understanding of the challenges of securing a Cloud Native platform and practical advice on using open source tools as part of their security strategy.

1. @mfdii
Michael Ducy, Sysdig, @mfdii
Securing your
Container Environment
with Open Source

2. @mfdii
Layers Container Security
Infra, Build, Runtime
Container Security Challenges
Open Source Tools For:
- Infra
- Build
- Runtime
Container Security Architecture
Agenda

3. @mfdii
Layers of Container Security
Runtime
Build
Infrastructure

4. @mfdii
Infrastructure
Host Security
Networking
Cluster Security
Container Runtime

5. @mfdii
Build
Image/Software Provenance
- Signed Images/Layers
- Artifact Signing
Vulnerability Management
- Upstream OS
- Application Vulnerabilities

6. @mfdii
Runtime
Service/Container Admittance
Secure Secrets
Anomaly Detection
Forensics

7. @mfdii
Decisions Pushed to Edge
Ephemeral Nature of Containers
Attack Surface
Resource Isolation
Challenges of Container Security

8. @mfdii
Infrastructure Security
Network Storage
Host
Cluster
Container Runtime

9. @mfdii
Infrastructure Security
Cluster:
- RBAC, Security Policies, Affinity
Host/Container Runtime:
- Seccomp, SELinux, AppArmor, Resource Constraints
Network:
- Service Mesh, Network Policy, Network Filtering

10. @mfdii
Security Policies
Security Policies define:
- Access to host resources:
- Filesystem, Host Network, Namespaces
- User/Group of Container
- Read Only Filesystem
- Linux capabilities available:
- http://man7.org/linux/man-pages/man7/capabilities.7.html
- Seccomp, AppArmor, or SELinux profiles

11. @mfdii
Linux Security Modules
SELinux
System wide execution policy
Apparmor
System wide execution policy, focused on processes
Seccomp
Per process system call isolation

12. @mfdii
LSMs
$ docker run --security-opt "apparmor=<profile>"
$ docker run --security-opt
seccomp=/path/to/seccomp/profile.json

13. @mfdii
Security Policies
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default'
apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'
spec:
privileged: false
# Required to prevent escalations to root.
allowPrivilegeEscalation: false
# This is redundant with non-root + disallow privilege escalation,
# but we can provide it for defense in depth.
requiredDropCapabilities:
- ALL
# Allow core volume types.
volumes:
- 'configMap'
- 'emptyDir'
- 'projected'
- 'secret'
- 'downwardAPI'
# Assume that persistentVolumes set up by the cluster admin are safe to use.
- 'persistentVolumeClaim'
hostNetwork: false

14. @mfdii
Container Affinity/Constraints
Affinity ensures:
- Containers with sensitive data or data processing routines are
next scheduled next to other containers
Strong labeling schema encouraged/required.

15. @mfdii
Kubernetes Pod Affinity
apiVersion: v1
kind: Pod
metadata:
name: with-node-affinity
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/e2e-az-name
operator: In
values:
- e2e-az1
- e2e-az2
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: another-node-label-key
operator: In
values:
- another-node-label-value

16. @mfdii
Host/Container Runtime Security
CIS Benchmarks for Docker Hosts
- https://www.cisecurity.org/benchmark/docker/
- Chef’s Inspec to scan for policy violations
- https://inspec.io
- https://github.com/dev-sec/cis-docker-benchmark
- Docker Bench for Security
- https://github.com/docker/docker-bench-security

17. @mfdii
Networking
Standard Firewall Rules/Security Groups
- Common exploit point are dashboards or API ports open.
Kubernetes:
- Network Policies, Container Networking Interface
Network Filtering
- Kernel level L3/L4/L7
- Cilium - https://cilium.io/

18. @mfdii
Cilium
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
description: "L3-L4 policy to restrict deathstar access to empire ships only"
metadata:
name: "rule1"
spec:
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingress:
- fromEndpoints:
- matchLabels:
org: empire
toPorts:
- ports:
- port: "80"
protocol: TCP

19. @mfdii
Cilium

20. @mfdii
Cilium
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
description: "L7 policy to restrict access to specific HTTP call"
metadata:
name: "rule1"
spec:
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingress:
- fromEndpoints:
- matchLabels:
org: empire
toPorts:
- ports:
- port: "80"
protocol: TCP
rules:
http:
- method: "POST"
path: "/v1/request-landing"

21. @mfdii
Cilium

22. @mfdii
Build Security
Network Storage
Host
Cluster
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
Container Runtime

23. @mfdii
Build Security
Image/Artifact Provenance:
- Artifact signing, trusted registries, admittance control
Vulnerability Management:
- Image scanning, OS libraries, application libraries

24. @mfdii
Image Scanning
Clair
- Static analysis of vulnerabilities in application containers.
- Focuses on Operating System packages and libraries
- https://github.com/coreos/clair
Anchore
- Analyzes container images against user defined policies.
- https://github.com/anchore

25. @mfdii
CoPilot & Openhub
CoPilot
- Open Source Application Dependency Vulnerability Management
- https://copilot.blackducksoftware.com/
- Incorporate into your build process
OpenHub
- Compare open source project usage, and project health
- https://www.openhub.net/

26. @mfdii
Notary & Portieris
Notary
- Signs collections of digital content (Artifacts)
- Project from Docker - Docker Content Trust
- Implementation of The Update Framework
- https://github.com/theupdateframework/
Portieries
- Kubernetes Admission controller for enforcing Content Trust
- https://github.com/IBM/portieris
- https://schd.ws/hosted_files/kccnceu18/41/kubernetes-notary-tuf.pdf
- https://www.youtube.com/watch?v=JK70k_B87mw

27. @mfdii
Notary & Portieris

28. @mfdii
Runtime Security
Network Storage
Host
Cluster
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
App Code
App Runtime
Libraries
OS
Container Runtime

29. @mfdii
Runtime Security
Service/Container Admittance
- What’s Allowed to Run/Join a Service
Secure Secrets
- How do applications authenticate
Anomaly Detection
- Is my runtime environment being tampered with?
Forensics
- What happened if something was compromised?

30. @mfdii
Service Identity
How can you verify a service is who it says it is?
SPIFFE
- Secure Production Identity Framework For Everyone
- Cryptographically verifiable Service IDs
- https://github.com/spiffe/spiffe

31. @mfdii
Service Identity

32. @mfdii
Anomaly Detection
- Containers are isolated processes.
- Processes are “scoped” as to what’s expected.
- Container images are immutable, runtime environments
often aren’t.
- How do you detect “abnormal” behavior.

33. @mfdii
Sysdig Falco
A behavioral activity monitor
•Detects suspicious activity defined by a set of rules
•Uses Sysdig’s flexible and powerful filtering expressions
With full support for containers/orchestration
•Utilizes sysdig’s container & orchestrator support
And flexible notification methods
•Alert to files, standard output, syslog, programs
Open Source
•Anyone can contribute rules or improvements

34. @mfdii
Quick examples
A shell is run in a container container.id != host and proc.name = bash
Overwrite system binaries
fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
and write
Container namespace change
evt.type = setns and not proc.name in
(docker, sysdig)
Non-device files written in /dev
(evt.type = create or evt.arg.flags contains O_CREAT)
and proc.name != blkid and fd.directory = /dev and
fd.name != /dev/null
Process tries to access camera
evt.type = open and fd.name = /dev/video0
and not proc.name in (skype, webex)

35. @mfdii
Falco architecture
falco_probe
Kernel
Module
Kernel
User
Syscalls
Sysdig Libraries
`
Events
Alerting
Falco Rules
Suspicious
Events File
Syslog
Stdout
Filter Expression
Shell

36. @mfdii
Falco Rules
25 common rules available OOTB
Focused on common container best practices:
■ Writing files in bin or etc directories
■ Reading sensitive files
■ Binaries being executed other than CMD/ENTRYPOINT

37. @mfdii
Falco rules
.yaml file containing Macros, Lists, and Rules
- macro: bin_dir
condition: fd.directory in (/bin, /sbin, /usr/bin, /usr/sbin)
- list: shell_binaries
items: [bash, csh, ksh, sh, tcsh, zsh, dash]
- rule: write_binary_dir
desc: an attempt to write to any file below a set of binary directories
condition: bin_dir and evt.dir = < and open_write and not package_mgmt_procs
output: "File below a known binary directory opened for writing
(user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING

38. @mfdii
Active Security with Falco, NATS,
and Kubeless
Falco NATS Kubeless
Detects abnormal event,
Publishes alert to NATS
Subscribers receive
Falco Alert through
NATS Server
Kubeless receives
Falco Alert, firing a
function to delete the
offending Kubernetes
Pod

39. @mfdii
Join the community
• Website
•http://www.sysdig.org/falco
• Public Slack
•http://slack.sysdig.com/
•https://sysdig.slack.com/messages/falco
• Blog
•https://sysdig.com/blog/tag/falco/
• Sysdig Secure
•http://sysdig.com/product/secure

40. @mfdii
Learn more
Github
• https://github.com/draios/falco
• Pull Requests welcome!
Wiki
• https://github.com/draios/falco/wiki
Docker Hub
• https://hub.docker.com/r/sysdig/falco/

41. @mfdii
Thank You.
Questions?
michael@sysdig.com, @mfdii