Kubernetes users need to allow traffic to flow into and within the cluster. Treating the application traffic separately from the business logic allows presents new possibilities in how service to service traffic is served, controlled and observed — and provides a transition to intra cluster networking like Service Mesh. With microservices, there is a concept of both North / South traffic (incoming requests from end users to the cluster) and East / West (intra cluster) communication between the services. In this talk we will explain how Envoy Proxy works in Kubernetes as a proxy for both of these traffic directions and how it can be leveraged to do things like traffic shaping, security, and integrate the north/south to east/west behavior. Christian Posta (@christianposta) is Global Field CTO at Solo.io, former Chief Architect at Red Hat, and well known in the community for being an author (Istio in Action, Manning, Istio Service Mesh, O'Reilly 2018, Microservices for Java Developers, O’Reilly 2016), frequent blogger, speaker, open-source enthusiast and committer on various open-source projects including Istio, Kubernetes, and many others. Christian has spent time at both enterprises as well as web-scale companies and now helps companies create and deploy large-scale, cloud-native resilient, distributed architectures. He enjoys mentoring, training and leading teams to be successful with distributed systems concepts, microservices, devops, and cloud-native application design.

1. Kubernetes: Ingress to service
mesh (and beyond!)
Christian Posta
Field CTO – Solo.io

2. 2 | Copyright © 2020 @christianposta
CHRISTIAN POSTA
• Field CTO @ Solo.io
• Author of a few books
• Contributor to many open-source projects
• Architect, blogger, speaker, mentor, leader
@christianposta
christian@solo.io
https://blog.christianposta.com
https://slideshare.net/ceposta

3. 3 | Copyright © 2020 @christianposta
MY BOOKS
https://bit.ly/istio-in-action
http://bit.ly/AtlantaK8-Jan2020
Raffle (3 copies)

4. 4 | Copyright © 2020
01
02
03
04
05
06
Level set
Common challenges for
application networking
Envoy Proxy
API Gateway and
Service Mesh
Bringing it all together
Things to watch for in 2020
Approximate flow of talk

5. 5 | Copyright © 2020

6. 6 | Copyright © 2020
Microservices and Kubernetes

7. 7 | Copyright © 2020
Move fast, safely
https://puppet.com/resources/whitepaper/state-of-devops-report

8. 8 | Copyright © 2020
As we move to services architectures,
on cloud-native deployment platforms,
we increase the complexity between
our services.

9. 9 | Copyright © 2020
Cloud application networking challenges
• Service discovery
• Retries
• Timeouts
• Load balancing
• Rate limiting
• Thread bulk heading
• Circuit breaking

10. 10 | Copyright © 2020
Cloud application networking challenges
• Edge/DMZ routing
• Surgical / fine / per-request routing
• A/B rollout
• Traffic shaping
• Request racing
• Internal releases / dark launches
• Request shadowing
• Fault injection

11. 11 | Copyright © 2020
Cloud application networking challenges
• Adaptive, zone-aware routing
• Deadlines
• Health checking
• Stats, metric, collection
• Logging
• Distributed tracing
• Security

12. 12 | Copyright © 2020
How we typically like to solve this problem:

13. 13 | Copyright © 2020
Q: “What's the biggest technology mistake you ever made?”
A: “When I was at IBM, I started a product called Websphere
… I thought it needs to be scalable, reliable, have a single point of
control ... I tried to build something like a mainframe, a system that
was capable of doing anything… I call it the endgame fallacy. It
was too complex for people to master. I overdesigned it.”
- Donald Ferguson

14. 14 | Copyright © 2020
Cloud application networking challenges
• Netflix Hystrix (circuit breaking / bulk heading)
• Netflix Zuul (edge router)
• Netflix Ribbon (client-side service discovery / load balance)
• Netflix Eureka (service discovery registry)
• Brave / Zipkin (tracing)
• Netflix spectator / atlas (metrics)

15. 15 | Copyright © 2020
Cloud application networking challenges
• Require specific language to bring in new services
• A single language doesn’t fit for all use cases
• How do you patch/upgrade/manage lifecycle?
• Need strict control over application library choices
• Inconsistent implementations
• Incorrect implementations

16. 16 | Copyright © 202016 | Copyright © 2020
Common application-networking needs
Connecting your services in a cloud-native environment

17. 17 | Copyright © 2020
Getting traffic into your microservices

18. 18 | Copyright © 2020
Unifying microservices with existing investment

19. 19 | Copyright © 2020
Defining application boundaries for sets of services

20. 20 | Copyright © 2020
Connecting service to service

21. 21 | Copyright © 2020
Connecting across deployment footprints

22. 22 | Copyright © 202022 | Copyright © 2020
The Kubernetes “equivalent” for
application networking
Foundational technology to help solve these challenges in a
cloud-native application architecture

23. 23 | Copyright © 2020
Envoy is to Application Networking
what Kubernetes is to Container Deployment
http://envoyproxy.io

24. 24 | Copyright © 2020
Envoy implements:
• zone aware, least request load balancing
• circuit breaking
• outlier detection
• retries, retry policies
• timeout (including budgets)
• traffic shadowing
• request racing
• rate limiting
• access logging, statistics collection
• Many other features!

25. 25 | Copyright © 2020
Envoy to do application networking heavy lifting

26. 26 | Copyright © 2020
Why Envoy?
• C++
• Built ground-up for services environment
• Large, diverse, vibrant community
• Dynamic configuration model
• Highly extensible (in C++  we’ll come back to this)
• Many out of the box L7 filters (HTTP, HTTP2, grpc, redis, mysql,
DynamoDB, thrift, zookeeper, kafka, et. al.)
• Incredible trove of telemetry out of the box
• Very versatile deployment options (as we’ll see)

27. 27 | Copyright © 202027 | Copyright © 2020
Quick Envoy demo

28. 28 | Copyright © 2020
Versatility of Envoy: Edge proxy

29. 29 | Copyright © 2020
Versatility of Envoy: Middle proxy

30. 30 | Copyright © 2020
Versatility of Envoy: Service proxy

31. 31 | Copyright © 2020
Envoy needs a control plane.
Specify the API there.

32. 32 | Copyright © 2020
The importance of a decentralized, declarative API
• Works well with automation
• Document-oriented, easy to store in SCM
• Decoupled from underlying system
• Less complex than general programming languages
• Represents ”intent” or “desired state”
• Use reconciling controllers to implement desired state
• Controllers observe drift and bring into balance

33. 33 | Copyright © 202033 | Copyright © 2020
API Gateway and Service Mesh
Workflow-specific APIs for Envoy Proxy

34. 34 | Copyright © 2020
API Gateway pattern
“A service that’s the entry point into the microservices-
based application … is responsible for request routing,
API composition, protocol translation …
and edge functions like authentication,
authorization, rate limiting, caching, et. al.”
https://www.manning.com/books/microservices-patterns
See also:
https://microservices.io/patterns/apigateway.html

35. 35 | Copyright © 2020
API Gateway pattern
• Abstracts backend services, protocols, message shapes, etc
• Powerful matching/routing, discovery, transformation,
aggregation, protocol translation
• Backends for front-ends
• A developer-first construct
• Intended to be decentralized and change at the pace of your
applications
• Edge functions including rate-limiting,
authentication/authorization, caching, metrics collection

36. 36 | Copyright © 2020
API Gateway built on Envoy
https://github.com/solo-io/gloo

37. 37 | Copyright © 2020
Gloo API Gateway
• Unify backend APIs running in Kubernetes, VMs, Physical, FaaS, etc
• Decentralized configuration: allow service teams to move fast
• Declarative configuration
• Provides a control plane for Envoy
• Security (Oauth/ODIC, API Key, TLS, SNI, OPA, HMAC, custom)
• Kubernetes native / run outside Kube as well
• Highly pluggable/extensible
• “If you know Kubernetes, you know Gloo”  user quote

38. 38 | Copyright © 2020
API Gateway built on Envoy
ENVIRONMENT
SECRET
CONFIGURATION
Data Plane
Upstream
gRPC-JSON transcoder
Rate limiting
External AUTH
…
Control Plane
Configure and manage
envoy’s plugins
Router

39. 39 | Copyright © 2020
Gloo has a decentralized configuration model

40. 40 | Copyright © 202040 | Copyright © 2020
Gloo demo
Workflow-specific APIs for Envoy Proxy

41. 41 | Copyright © 202041 | Copyright © 2020
Service Mesh
Workflow-specific APIs for Envoy Proxy

42. 42 | Copyright © 2020
Control plane for managing fleet of service proxies

43. 43 | Copyright © 2020
Service proxy lives with application instance

44. 44 | Copyright © 2020
Service proxy lives with application instance

45. 45 | Copyright © 2020
Service mesh technologies provide the following:
• Service discovery / Load balancing
• Secure service-to-service communication
• Traffic control / shaping / shifting
• Policy / Intention based access control
• Traffic metric collection
• Service resilience
• API / programmable interface

46. 46 | Copyright © 2020
Service proxy lives with application instance

47. 47 | Copyright © 2020
Open source service mesh:
http://istio.io

48. 48 | Copyright © 202048 | Copyright © 2020
Istio demo
Workflow-specific APIs for Envoy Proxy

49. 49 | Copyright © 202049 | Copyright © 2020
Bringing it all together
Workflow-specific APIs for Envoy Proxy

50. 50 | Copyright © 2020
Service mesh and API Gateway can use same proxy
but for different purposes

51. 51 | Copyright © 2020
Service mesh and API Gateway can use same proxy
but for different purposes

52. 52 | Copyright © 2020
Service mesh and API Gateway can use same proxy
but for different purposes

53. 53 | Copyright © 2020
Role of API Gateway: abstracts details/implementation
https://blog.christianposta.com/microservices/do-i-need-an-api-gateway-if-i-have-a-service-mesh/

54. 54 | Copyright © 2020
Role of service proxy: expose details/implementation

55. 55 | Copyright © 2020
Complementary roles

56. 56 | Copyright © 202056 | Copyright © 2020
Envoy as the backbone of multi-cluster
service mesh
Even when a service mesh doesn’t use Envoy!

57. 57 | Copyright © 2020
Multiple clusters (potentially multiple meshes)

58. 58 | Copyright © 2020
Multiple clusters (potentially multiple meshes)
https://servicemeshhub.io

59. 59 | Copyright © 202059 | Copyright © 2020
What to watch for
Upcoming improvements for which to keep an eye out

60. 60 | Copyright © 2020
Web Assembly shaking up the data plane

61. 61 | Copyright © 2020
Web Assembly shaking up the data plane
https://github.com/envoyproxy/envoy-wasm

62. 62 | Copyright © 2020
Web Assembly shaking up the data plane
https://webassemblyhub.io

63. 63 | Copyright © 2020 @christianposta
THANK YOU FOR COMING OUT!
@christianposta
christian@solo.io
https://blog.christianposta.com
https://slideshare.net/ceposta

64. 64 | Copyright © 2020
• https://solo.io
• https://slack.solo.io
• https://gloo.solo.io
• https://envoyproxy.io
• https://istio.io
• https://webassemblyhub.io
• https://servicemeshhub.io
• https://blog.christianposta.com