Why AIOps Matters For Kubernetes (Talk in Taiwan Kubernetes Day 2017)

1. Timothy Chen
Co-founder / CTO
Why AIOps Matters for
Kubernetes

2. About me
2
Apache PMC Mesos, Drill
Container engineer lead at Mesosphere
Help start community effort for Spark on Kubernetes (Google, Redhat,
Palantir, Salesforce, etc).
@tnachen

3. Agenda
3
Operating Kubernetes in production at scale
What is AIOps?
Operating Kubernetes in production at scale w/ AIOps!

4. Adoption

5. Infrastructure Maturity

6. Microservices

7. Microservices at scale

8. Open source software

10. Container Production Stack
Orchestration
Security Network Storage
Runtime
OS
Image

11. Infrastructure Monitoring
Orchestration
Network
Storage
Runtime
OS
Image
Users:
SRE
System Engineers
Ops
DevOps

12. Application Monitoring
Kafka
Microservice #1
Spark
Microservice #2
Microservice #3
Users:
SRE
DevOps
Developers

13. Metrics Explosion

14. Reactive monitoring new challenges
Containers != VM
Cannot focus on host metrics anymore with dynamic container
changes
Which metric(s) to track?
10000s of metrics, massive amounts of alert
How to identify or isolate the problem (correlate metrics)?
HW/Container/Host/Neighbor/Cluster/Dependency….
How to resolve these performance problems? (RCA)
Unmanaged resources -> Manual tuning

15. Proactive challenges
Goal: Maximize performance with minimum cost!
Capacity Planning / What-If Analysis
Cluster scheduling (interference, heterogenous, etc)
Optimal configuration
-
Orchestration
Network Storage
Runtime
OS
Application
VM

17. https://netman.cs.tsinghua.edu.cn/contacts/projects/

18. AIOps is not a new concept
Decades of academic research around managing
performance in computing….
- Cluster Scheduling
- SLA-driven interference, storage, network, caching...

19. Now is the time
- Workloads are more complex than before
Infrastructure interfaces and data are becoming more
standardized (Kubernetes, Prometheus, CTE, etc)
All metrics centrally collected
-

22. https://www.youtube.com/watch?v=dJxGtfTPVCg

27. The Difficulty of using the Public Cloud
Many instance types
Reserved, on-demand, spot instances
Many instant sizes (10s)
Application churn
Variability in load
Can we guarantee SLOs & minimize cost automatically?

28. Choosing the right VM/HW

29. Performance tuning

31. Live traffic load testing
https://engineering.linkedin.com/blog/2017/02/redliner--how-linkedin-determines-the-capacity-limits-of-its-ser

32. Live traffic bottleneck
https://research.fb.com/publications/kraken-leveraging-live-traf%EF%AC%81c-tests-to-id
entify-and-resolve-resource-utilization-bottlenecks-in-large-scale-web-services/

33. Chaos Engineering

35. Batch scheduling
Improve deadline by 5x - 13x, increase utilization 14 - 28%

36. Placement

37. Variability in user load
Bloated reservations due performance variability
TwitterGoogle

38. Oversubscription
Google search + Google brain (deep neural nets)
>90% hardware utilization
No latency violations for search
[Lo et al’15]
>90% HW utilization No tail latency
problems

40. Hyperpilot
40
Help enterprises
operate container in
production.
Using AI to learn how
applications are behaving with your
infrastructure and actively manage these
resources.
Processors, Memory, Flash, Disk, NIC
Operating system
Cluster resource manager
Distributed
analytics
Hadoop,
Spark, …
IaaS
Online,
data-intensive
(OLDI)
Search, social nets,
SaaS

41. Oversubscription in Action

42. Hiring Engineers in Silicon Valley / Taiwan!
tim@hyperpilot.io
Christos Kozyrakis (Stanford)
Michael Huang (ex-CoreOS)
Timothy Chen (ex-Mesosphere)