Security Rationale For Istio
An introduction to Istio security, looking at how Istio helps to keeps your security team happy by satisfying Kubernetes security requirements for multi-tenancy, and your developers happy by reducing implementation effort. Istio is still an evolving technology, and outstanding issues and impending improvements will be discussed.
2. Talk Structure
● Secure Kubernetes multi-tenancy and Istio
● Istio security
○ Secure installation and configuration
○ Threats and issues
3. Securing Multi-tenanted Kubernetes Clusters
● Secure administration
○ Cloud IaaS & PaaS services
○ Kubernetes Platform
● CI/CD pipeline security
● Runtime security
○ This is where Istio fits in
4. A slice of the Kubernetes runtime threat model
What can go wrong?
Compromised microservice
attempts to:
● Eavesdrop
● Impersonate
● Escalate privilege
● Pivot & access other
services
● Initiate an outbound
connection
API
db
webadmin
API
db
webadmin
Compromised
Service
5. A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
API
db
webadmin
API
db
webadmin
Mitigated threat
6. A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
API
db
webadmin
API
db
webadmin
7. A slice of the Kubernetes runtime threat model
API
db
webadmin
API
db
webadmin
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
8. A slice of the Kubernetes runtime threat model
What are we going to do
about that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
Kubernetes v1.7+
Network Policy
API
db
webadmin
API
db
webadmin
9. Satisfying requirements at scale
Security requirements for every application team:
● Authentication
● Authorisation
● Encryption
Against a backdrop of:
● Different developer teams working with
different languages
● Security vs Speed dichotomy
● Migration of legacy applications.
10. Satisfying requirements at scale
● Option 1: In-house or 3rd party
libraries
○ In all the languages your company
uses
○ And maintain them for all your
services and teams
● Option 2: Sidecars
○ Repeatable
○ Abstracted from Devs
○ Could end up maintaining a large
number of sidecars over time
12. Security controls provided by Istio
● Mutual TLS - Encryption &
Authentication
● L7 Authorisation
● Rate limiting
● Whitelisting/Denials
● Egress control
Compromised services
have a blast radius
defined by Istio policy
API
db
webadmin
API
db
webadmin
13. Threats not mitigated by Istio
Amongst others:
● Injection Attacks
● Container breakout
API
db
webadmin
API
db
webadmin
Worker Node
15. Threat: Insecure Control Plane Configuration
● Secure the control plane:
○ Enable control plane mutual TLS
○ Protect Citadel
○ Don’t write authorization policies for Istio control plane
components
17. Threat: User Misconfiguration
● Avoid manual configuration
● Regularly apply config defined in git
○ Regular CI server job
○ GitOps
18. Threat: Compromised workload attacks Istio sidecar
API
db
admin
API
db
web service
K8s
Network
Policy
K8s
Network
Policy
19. Threat: Compromised workload attacks Istio sidecar
● Access to in-mesh
services subject to Istio
RBAC & Auth
● Attack other services
● Circumvent Istio egress
control
● Can mitigate with a
Kubernetes Network
Policy
O
API
db
admin
API
db
web service
K8s
Network
Policy
K8s
Network
Policy
20. Threat: Compromised workload attacks Istio sidecar
● Defence in depth: use dedicated Egress Gateway, K8S
Network Policy & IaaS FW rules
API
db
web service
K8s Network
Policy + IaaS FW
Worker Node (Egress)
Egress Gateway
Worker Node
21. Threat: Init Containers Run Off-mesh
● Init container for application runs before the Istio init container
○ Unconstrained by istio security
○ Use K8s network policy
App Init
Istio init
App Init
(completed)
Pod Initialising Pod Ready
Application
22. Issue: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a
restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities
Kubernetes API
Authentication &
Authorisation
Admission Controllers
Mutating Validating
apiVersion: apps/v1
kind: Deployment
...
securityContext:
privileged: false
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL Pod Security Policy
...
kind: PodSecurityPolicy
...
spec:
privileged: false
allowPrivilegeEscalation: false
requiredDropCapabilities:
- ALL
24. Issue: PodSecurityPolicy blocks Istio init & sidecar
● Must relax Pod Security Policy to run Istio
● Might be fixed by:
○ CNI Plugin
○ Sub-pod isolation proposals
25. Threat: Misconfigured app container could run as
privileged or use NET_ADMIN to exit the mesh
● Due to relaxed PodSecurityPolicy
● Workarounds (weak!)
○ Templating YAML
○ Reviews & process (e.g Kubesec.io review)
○ IDS
● Istio CNI Plugin intended to mitigate this long term
27. Pending Improvements
● TLS health-checking: coming in v1.1
● CNI Plugin: optional in v1.1
● Hardening, robustification, v2 workload attestation, plugable CA
adapters: all on the way
28. In Conclusion
● Istio is exciting
● Security functionality at scale for microservices
○ Authentication & Authorisation
○ Encryption in transit
● Still maturing
○ Some improvements required
○ Some features still in alpha
32. Security controls provided by Istio
● Compromised services only
have the blast radius of their
Istio policy
○ RBAC L7 policy is
granular: HTTP verb
and path
○ Tight configuration
increases cluster
compromise
resilience
API
db
webadmin
API
db
webadmin
33. Security controls provided by Istio
● Mutual TLS - Encryption &
Authentication
● L7 Authorisation
● Rate limiting
● Whitelisting/Blacklisting
● Egress control
+
● Kubernetes Network
Policy
API
db
webadmin
API
db
webadmin
35. A slice of the Kubernetes runtime threat model
What are we going to do about
that?
● Authn & Authz
● Encryption in transit
● L3/4 Firewalling
Kubernetes v1.7>
Network Policy
API
db
webadmin
API
db
webadmin
36. Threat Modelling Runtime Security
Threat Modelling
1. What are we building?
2. What can go wrong?
3. What are we going to do about
that?
4. Did we do a good enough job?
API
db
webadmin
API
db
webadmin
37. Threat: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities
● Istio init & sidecar runs as root and requires NET_ADMIN
● Threat: Misconfigured app container could run as root or use
NET_ADMIN to exit the mesh
● Workaround - by templating YAML, using Kubesec or IDS
● Istio CNI Plugin intended to mitigate this long term
38. Threat: PodSecurityPolicy blocks Istio init & sidecar
● I want my Pods to comply to a restrictive Pod Security Policy
○ Non-privileged
○ Drop linux capabilities
39. Threat: Compromised workload attacks Istio sidecar
● Compromised services could attack Istio sidecar proxy to exit
the mesh
○ access to in-mesh services subject to Istio RBAC & Auth
○ circumvent egress control
○ Mitigate using a combination of egress gateway, K8s
network policy and infrastructure firewall rules
40. Limiting Compromised Workloads
● Compromised services only have the blast radius of their Istio
RBAC policy
○ RBAC L7 policy is granular: HTTP verb and path
○ Tight configuration increases cluster compromise resilience
42. Risk: Accidental deployments outside of the mesh
● Improper namespace labelling
○ failure to trigger automatic sidecar injection
43. A slice of the Kubernetes runtime threat model
Worker node 1 Worker node 2
App1
Service A
App1
Service B
App2
Service A
App2
Service B
App2
Service B
App2
Service A
App1
Service B
App1
Service A
Worker node 1 Worker node 2
App1
Service A
App1
Service B
App2
Service A
App2
Service B
App2
Service B
App2
Service A
+
App1
Service B
App1
Service A
L3/L4 Network Policy - Kubernetes v1.7
