2. About me
¨ Distinguished Engineer
at Citrix
¨ Apache CloudStack
PMC
¨ Work on Citrix
Netscaler and
containers
3. Load balancers are important (again)
¨ Containers and Microservices
¨ Teaching old load balancers new tricks
¨ Emerging patterns for Load Balancing
¨ Future directions
4. A brief history of Load balancing
Internet
W W
High Availability
Internet
W W WW
HA + Load Balancing
Internet
Webserver
5. A brief history of Load balancing
Internet
W WW
App 1 App 2
W WW
foo.com/app1 foo.com/app2
L7 routing
Internet
W WW
SSL Traffic
Unencrypted Traffic
Internet
W WW
App Firewall
12. Load Balancing: Traditional vs. Cloud Native
Static Applications, well
defined topology
Dynamic Microservices,
changing topology
W W W
A A A A A
W
M
M
TrafficismostlyN-S
M
M
M
M
M
M
M
M
M
Traffic is mix
of N-S and E-W
30. Client-side LB
¤ Embedded into calling
application
n e.g., Netflix Ribbon
(with Eureka)
¤ Run as side-cars
(alongside each
application or one-
per-host) Eureka
M M
M
M
M
M
31. Client-side LB
¤ Run as side-cars, or
one-per-host
Service
Discovery
M
M
M
M
M
M
Clients
Service
LB
LB
32. Client-side LB Examples
¨ Netflix Prana - Hystrix as side car (deprecated)
¨ Linkerd from Buoyant.io (based on Twitter Finagle)
¨ AirBnB Smartstack/Synapse (uses HAProxy)
¨ Uber Hyperbahn
¤ Like Finagle, switched from HAProxy
¨ Kube Proxy
¤ Initially user-space, now based on IPTables
¨ Docker Swarm LB
¤ Based on IPVS
¨ Mesos LB
¤ Based on IPTables
36. A general-purpose L7 proxy?
M M
Client
Service A
L7 ProxyL7 Proxy
M Service B
L7 Proxy
Service
Discovery
CD
Pipeline
Policy
Engine
37. Zero-Downtime Deploys
¨ Canary deploys
¨ Traffic can be selected
based on source
(country), headers
M M M M
Version N
M M M M M
Version N+1
10%90%
CD
Pipeline
39. Advanced Resilience – Circuit Breaker
¨ Circuit Breaker is a
pattern that prevents
cascading failures due
to unresponsive
downstream services
¨ A load balancer can be
configured to ignore a
slow backend
Credit: http://martinfowler.com/bliki/CircuitBreaker.html
43. Multi-cloud
¨ Choose the best
location for the
microservice
¨ Automatic discovery
and resilience
¨ See Kubernetes
Federation for
example
M M M M
Cloud 1
M M M M M
Cloud 2
Further reading: http://kubernetes.io/docs/user-guide/federation/federated-services/
46. Encryption
M M
Client A
Service A
L7 ProxyL7 Proxy
M Service B
L7 Proxy
Client Certificate
Further reading: https://blog.cloudflare.com/how-to-build-your-own-public-key-infrastructure/
48. Authorization
M M
Client A
Service A
L7 ProxyL7 Proxy
M Service B
L7 Proxy
Authorization: Bearer: xxxxx.yyyyy.zzzzz
Authorization: Bearer: aaaaa.bbbb.cccccc
Token Insertion
Token Validation
Further reading: https://github.com/coreos/jwtproxy
49. Protocol Insights
¨ Microservice communication patterns are
changing
¤ New protocols/formats: gRPC, Thrift, TChannel
¤ Asynchronous / Event-driven
¤ Queues and real-time streams
50. Wrap-up
¨ Load balancing is
different for containers /
microservices
¨ Integration with container
cluster managers is
needed
¨ Consider using the same
LB technology for ingress
and intra-cluster
¨ Emerging patterns
solidify the importance
of the load balancer
¨ LB in the wire brings
¤ Simplicity
¤ Resilience
¤ Future proofing