Presented on March 16, 2017 through O'Reilly - http://www.oreilly.com/pub/e/3864 Modern day applications bring modern day infrastructure requirements. Whether you bring your own or you use your cloud provider's managed load-balancing services, even moderately sophisticated applications are likely to find their needs underserved.

1. Load-Balancing in
the Cloud using
Lee Calcote
http://calcotestudios.com/talks
Nginx Kubernetes&

2. Lee Calcote
linkedin.com/in/leecalcote
@lcalcote
blog.gingergeek.com
lee@calcotestudios.com
clouds, containers, infrastructure,
applications and their management

3. Private
Public

4. Application Delivery
Controllers
Load-Balancers
have evolved.
@lcalcote

5. Our Bloat-a-lith
A 1GB jar!

6. Our Case Study
is an IDE for game developers
created by game development studio
based in Austin
creates a real-time, collaborative
game development engine
“Google Docs for game development”
@lcalcote

7. Vert.x
Our Bloat-a-lith
EventBus Handler
Persistence Verticle
SockJS
Analytics
Analytics UI
Hazelcast
GridConnection
Telemetry
Telemetry Analysis
Projects, Streams,
Users, Sessions,
Tenants
Kafka Producer
Platform Client
Producer
Spark, R
GCE Network
Load Balancer
Kafka
Rendering Engines
Platform Publishing
Tool
Game Analysis
Single instance
@lcalcote
beloved

8. Shaping Up

9. Faster delivery, rolling updates
Horizontal scale out on-demand;
on an individual service basis
Modular architecture
Easy integration and deployments
Service isolation, resilience and fail-safe recovery
Benefits of Microservices
Democratization of language and technology choice
@lcalcote
a quick review
it's an excellent time to be a developer
The promise of...

10. Characteristics of Microservices
how small is small?
Who has a system that is
too big and that you’d
like to break down?
Can a small team manage it?
More moving parts
increases complexity.
Can you make a change to a service
and deploy it by itself without
changing anything else?
@lcalcote
small, autonomous services that work together.
independent, autonomous service
self-contained functional unit

11. App is Reactive
Leverages sockets
Limited resources
Culture (DevOps / Cloud maturity)
Support containers AND VMs
Propagation of huge data sets
The Challenge
On-the-fly intelligent rendering
Distribution of content
Extremely low latency for Reactive services
Blocking vs non-blocking
On-premises telemetry collection and
analytics
@lcalcote

12. Our Microbloat v2
Analytics UI
Authentication
Telemetry
Telemetry Analysis
Spark, R
Kafka
Rendering Engines
Platform Publishing
Tool
Game Analysis
Master
etcd
Locking
Projects
Authorization
kube-proxy kube-proxy
SockJS
Authorization
kube-proxy
DaemonSet
Node
Node
kube-api, etc.
@lcalcote
Node
Pod
Pod
Deployment
Authorization
DaemonSet
Service

13. Comparing
Services(not shoes)

14. SSL Termination
@lcalcote
Kubernetes 1.5 No
Swarm 1.13 No
Mesos+Marathon Yes
ELB Classic Yes
ELB L7 Yes
Beanstalk Yes (EC2)
IOT Yes
ECS Yes (EC2)
Load-Balancer No
App Gateway Yes
Container Service ?
Cloud LB (HTTP) Yes
Cloud LB (Network) Yes
GKE No
Container
Orchestrators
Clouds
AWS
Azure
GCP
Link
Link
Link
Link
Link
Link
Link
Link
Link
SSL Proxy
SSL Proxy

15. Websocket Support
Kubernetes 1.5 No
Swarm 1.13 No
Mesos+Marathon Yes
Container
Orchestrators
Clouds
AWS
Azure
GCP
Link
Link
Link
Link
Link
Link
Link
Link
Link
@lcalcote
ELB Classic Yes
ELB L7 Yes
Beanstalk Yes
IOT Yes
ECS Yes
Load-Balancer No
App Gateway Yes
Container Service ?
Cloud LB (HTTP) No
Cloud LB (Network) Yes
GKE No

16. Kubernetes & Nginx
to the Rescue
@lcalcote
There are soooo many ways to skin this cat.

17. Microbloat v3
Going deeper with Nginx & Kubernetes
As an ingress controller in Kubernetes
SSL termination
Path-based rules
Web socket support
@lcalcote
Service Discovery with Nginx Plus
Need for locating service instances
instantly without reconfiguring
On-the-fly Reconfiguration API
Work with etcd

18. Kubernetes

19. - group of co-scheduled containers and volumes
Replication Controller - reconciliation loop to keep current
state congruent with desired state
- a set of pods that comprise a common function
- manages updates for Pods and Replica Sets
- store and retrieve sensitive data
ConfigMap
Pod
Service
Deployment
Secrets
ˈnō-mən-ˌklā-chər
a brief Kubernetes construct review
@lcalcote

20. Exposing Kubernetes Services
- service to be reachable only from inside of
the cluster.
- It serves as a way to return an alias to
an external service residing outside the cluster.
- exposes service on a port on each node of
the cluster.
- cluster-internal IP and exposing service
on a NodePort, also ask the cloud provider for a load
balancer which forwards requests to the Service
exposed as a <NodeIP>:NodePort for each Node.
ClusterIP
ExternalName
NodePort
LoadBalancer
@lcalcote

21. runs on each node in the cluster
a network proxy that represents Services on each node
integral to how services are exposed in the cluster
limited to layer 4 (tcp/udp) load-balancing
kube-proxy
kube-proxy
no·men·cla·ture
@lcalcote
kube-proxy

22. iptables
Container AA
Container A
kube-proxy kube-proxy
Node A Node B
Client
Pod A
Service A
iptables
Inbound
Outbound
NodePort
@lcalcote
Traffic flow with
NodePort/LoadBalancer
Container BB
Container B
Pod B
Service B

23. An is a collection of rules
that allow inbound connections to
reach the cluster services. - how
you expose and route to the
.
Ingress
service
Ingress
no·men·cla·ture
@lcalcote
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: projects
spec:
tls:
- hosts:
- api.maxplay.io
secretName: api-secret
rules:
- host: api.maxplay.io
http:
paths:
- path: /projects
backend:
serviceName: tenant-svc
servicePort: 80
- path: /tenants
backend:
serviceName: user-svc
servicePort: 80
- path: /users
backend:
serviceName: user-svc
servicePort: 80

24. Secrets
$ kubectl create secret generic api-secret --from-file nginx.conf
@lcalcote

25. An is a
control loop that manages rules
enabling inbound traffic to
applications.
Ingress Controller
Ingress Controller
specification
@lcalcote
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-dp
labels:
app: nginx-ingress
spec:
replicas: 1
selector:
app: nginx-ingress
template:
metadata:
labels:
app: nginx-ingress
spec:
containers:
- image: maxplay/nginx-ingress:latest
imagePullPolicy: Always
name: nginx-ingress
ports:
- containerPort: 80
hostPort: 80
- containerPort: 443
hostPort: 443
args:
- /nginx-ingress-controller
- --default-backend-service=
$(POD_NAMESPACE)/nginx-default-backend
Make sure you review controller specific docs so you understand the caveats of each
one.

26. iptables
Container AA
Container A
Ingress
Controller
kube-proxy kube-proxy
Node A Node B
Client
Pod A Ingress B
Service A
iptables
Inbound
Outbound
@lcalcote
Traffic flow with
Ingress Controller

27. Annotation in the Ingress
resource definition specifies
which services are web socket
services -
"socks-svc"
Web Socket Support
Ingress Resource Annotation
@lcalcote
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: api-ingress
annotations:
nginx.org/websocket-services: "sockjs-svc"
spec:
tls:
- hosts:
- api.maxplay.io
secretName: api-secret
rules:
- host: api.maxplay.io
http:
paths:
- path: /sockjs
backend:
serviceName: sockjs-svc
servicePort: 8181

28. NGINX

29. Why Nginx
1. Hybrid needs
On-premises and Google Cloud Platform
2. Consistent administration and capabilities
central load balancing and proxy platform
3. Support for VM and container-based technologies
with minimal configuration change
4. Deeper feature set available as services/team matures
use as an Application Delivery Controller
Platform independence
@lcalcote

30. Common Administration w/Nginx Plus
App
App
App
@lcalcote

31. A/B Testing Using Nginx
Ingress Controller and Deployments
Requests
App v0.1
App v0.2
90% of requests go to v0.1
10% of requests go to v0.2
@lcalcote

32. GSLB & Content Caching
w/Nginx Plus
App
App
App
GeoDNS
US
Regions
Session Persistence and Sticky Routing help in performance of request
routing and localized content
Content Caching provides faster retrieval of data
Performance, reliability and availability
Global
Regions
@lcalcote

33. Lee Calcote
linkedin.com/in/leecalcote
@lcalcote
blog.gingergeek.com
lee@calcotestudios.com
Thank you.
Questions?
clouds, containers, infrastructure,
applications and their management
http://calcotestudios.com/ talks