Sharing the experience about how to host the Kubernetes Container Ecosystem in AWS

1. Build, Ship and Run
Your App, Production !!

2. HELLO!
I am smalltown (Tom Tsai)
I am here because I want to share
something with you
You can find me at
smalltown@awsug.tw

3. How Far...From Dev To Prod?
App Can Run As
Container
App Can Run As
Container In Prod
What The FXXK Gap?

4. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

5. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

6. What Happened?

7. Background
Exist Service Wants to Migrate
From AT&T Data Center to AWS

8. Technical Debt
- Service Feature Out of Date
- Trap in The Infrastructure
- Without Scalability

9. Not Just Migration
- Application Refactory
- Micro Service Adoption
- Container Introducing

10. Just Only Migration
V.S.
Leverage AWS Resource

11. Comparison
AWS RESOURCE OTHERS
Database RDS Self Host
Share Storage EFS Self Host
Docker Registry ECR Self Host
... ... ...
Container Orchestration ECS Host K8S

12. Container Ecosystem

13. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

14. Container Die Anytime !
- Is The Application Suit For
Living as Container?

15. Examine The Application
Stateful
Stateful Stateless
Session Centralization
Stateful
Adopt Shared Storage
Stateless
Keep Origin
App Cluster Mode + StatefulSets
Work

16. Container Liveness and
Readiness
- Liveness: When The App
Transition to Broken States
- Readiness: When The App
Ready to Receive Traffic

17. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

18. Why Kubernetes?
◉ Both ECS & K8S Can Solve All My Problems
◉ Interoperability V.S. Vendor Lock-in
◉ Familiarity & Tight Schedule
◉ Popularity & Community

19. Kubernetes Infra.

20. Kubernetes Module
Kubernetes Master Kubernetes Worker
Etcd Cluster
Flannel
Docker
Kubelet
K8S Proxy
K8S API Server
K8S Control
Manager
k8S Scheduler
Etcd Proxy
Flannel
Docker
Kubelet
K8S Proxy
CoreOS: See You Later !!

21. Kubernetes Scalability
Workers Pods
Horizontal # of Workers
(https://goo.gl/qrJuHE)
# of Pods
(https://goo.gl/mdoZFY)
Vertical Resources for a Workers
(Different EC2 Type?)
Resources for a Pod
(Under Development)

22. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

23. CoreOS Host Kubernetes

24. How K8S Run in CoreOS
K8S Master K8S Worker
Docker Container
(Hyperkube)
- K8S Proxy
- K8S API Server
- K8S Control Manager
- k8S Scheduler
Docker Container
(Hyperkube)
- K8S Proxy
RKT Container
- Etcd (etcd-wrapper)
- Flannel (flannel-wrapper)
- Kubelet (kubelet-wrapper)

25. How To Update Core OS?!
- Update Linux OS is Hard, But
Not CoreOS

26. CoreOS Release Channel

27. CoreOS Update Strategies
Strategy Description
reboot
(Dev)
Reboot Immediately After an Update is Applied
etcd-lock
(Stage or Prod)
Reboot After First Taking a Distributed Lock in Etcd
off
(Prod)
Do Not Reboot After Updates are Applied

28. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

29. Avoid Add AWS Resource Manually
- Using Hashicorp Terraform to Add
Public Cloud Resource
- Infrastructure As Code

30. What is Terraform?
- Building, Changing, and
Versioning Infrastructure Safely
And Efficiently
- Manage Existing and Popular
Service Providers

31. Not This Session
- It’s a Long Story to Say...
- Refer to Here For Slide
- Refer to Here For Sample

32. My Some Practices
- Load Balancer Define in
Terraform
- EBS Persistent Volume Define in
Terraform

33. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

34. Centralised V.S. Decentralised
- Official Suggestion
- Ship Log By Each Node
- Ship Log By Each Pod

35. How to Collect App Log
Collector Problem
Docker Log Driver Not Support in Kubernetes
Logstash Resource Monster
Beats Lack of Output
Fluentd Similar Logstash
Fluent Bit Lack of Output

36. Log Management

37. Docker
Monitor
Log
AWS
CoreOS
Kubernetes
Terraform

38. Monitor K8S Cluster
- Container View & Cluster View
- Yes, Prometheus (Not Again)
- Another Way For Integrating
With AWS ASG or SNS ?

39. Do You Remember EC2 era?
- Memory and Disk Metrics for
AWS EC2 Linux Instances.
(Ref)
- Turn it Into Container. (Ref)

40. Easy Monitor Cluster Status

41. Any questions ?
Thanks!