2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017
This talk covers the past, present, and future of Microservices at Squarespace. We begin with our journey to microservices, and describe the platform that made this possible. We introduce our idea of the “Pillars of Microservices”, everything a developer needs to have a successful production service. For each pillar we describe why we think it is important and discuss the implementation and how we utilize it in our environment. Next, we look to the future evolution of our microservices environment including how we are using containerization and Kubernetes to overcome some of the problems we’ve faced with more static infrastructure.
Cost Control and Rapid Innovation in Kubernetes with OpenRewrite
Similar to 2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017
Similar to 2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017 (20)
CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️
2017 Microservices Practitioner Virtual Summit: Microservices at Squarespace - Doug Jones and Kevin Lynch, Microservices Practitioner Virtual Summit 2017
1.
2. Part 1: Building the Pillars of
Microservices
Part 2: Containerization and
Orchestration (Kubernetes)
AGENDA
3. Part 1: Building the Pillars
01
The Journey to Microservices
02
Building the Pillars of Microservices
4.
5. Microservices Journey: A Story of Growth
2013: small (< 50 engineers)
build product & grow customer base
whatever works
2014: medium (< 100 engineers)
we have a lot of customers now!
whatever works doesn't work anymore
2016: large (100+ engineers)
architect for scalability and reliability
organizational structures
?: XL (200+ engineers)
6. Challenges with a Monolith
● Reliability
● Performance
● Engineering agility/speed, cross-team coupling
● Engineering time spent fire fighting rather than building new
functionality
What were the increasingly difficult challenges with a
monolith?
8. Challenges with a Monolith
● Monitoring typically starts at the edges
○ Think requests in, DB queries out, etc
● What about the guts of the app? How much visibility do you have
there?
● How long does it take you to recover from an issue? Find the cause
and fix the issue?
Challenges with Monitoring/Finding Faults
9. The Journey to Microservices
● Define Pillars: ideas we consider necessary for successful production
microservices
● Implement these pillars as part of our platform
● Reduce boilerplate and reinventing the wheel syndrome
● Service authors get these for free and can focus on their application
domain
Design a Platform for Production, Remove Challenges
13. Building the Pillars of Microservices
● HTTP + JSON
○ Industry standard. Tons of tools.
● Solid open source Java API server platforms
○ Started with Dropwizard
○ now on Spring Boot (configured to use Jetty and Jersey 2)
Pillar: HTTP APIs
14. Building the Pillars of Microservices
● Swagger (OpenAPI Specification)
● Code generation
○ Swagger spec → models, server API, client
Even Easier APIs
15. Swagger Path Example
paths:
/currency-info:
put:
tags:
- CurrencyInfo
description: "Creates a new {@link CurrencyInfo} resource."
summary: Create a new currency info
operationId: save
parameters:
- name: info
in: body
schema:
$ref: '#/definitions/CurrencyInfo'
responses:
200:
description: ok
schema:
$ref: '#/definitions/CurrencyInfo'
17. Building the Pillars of Microservices
● Services announce themselves, publishing their name and host/port
information
● Started with a simple announcement payload and found that was
enough
● Healthchecks to mark services down
Pillar: Service Discovery
18. Building the Pillars of Microservices
● First: Zookeeper
○ Complicated clients (no HTTP API)
○ Must build discovery on Zookeeper primitives
○ Strong consistency is unnecessary
○ Client heartbeats can’t be expanded upon
○ No great way to support multiple data centers
Service Discovery Systems
19. ● Now: Consul
○ First class discovery support
○ Built in multi-data center support
○ Simple HTTP API
○ Configurable healthchecks
○ key/value store
■ We use for dynamic config and leader election
Building the Pillars of Microservices
Service Discovery Systems
20. DC2DC1
Multi DC with Consul
ConsulConsulConsul ConsulConsulConsul
Service
Announce
Service
Announce
Primary DB Replica DB
Replicate
WAN Gossip
Consistent Set
21. DC2DC1
Multi DC with Consul
ConsulConsulConsul ConsulConsulConsul
Service Service
Primary DB Replica DB
Replicate
Service
Query
?dc=”DC2”
Remote DC forwarding
22. Building the Pillars of Microservices
● Avoid middleware/extra configuration
● Customizable logic
● Connection pooling
● System awareness to increase fault tolerance
● Builds on Netflix Ribbon OSS
Pillar: Software Load Balancers
23. Building the Pillars of Microservices
● Metrics
● Dashboards
● Distributed Tracing
● Structured Logging
● Healthchecks
● Alerts
Pillar: Observability
28. Building the Pillars of Microservices
● Addresses the Fanout problem, improved latency
● Reactive: RxJava with RxNetty
● Allows greater composition and reuse. Avoid “callback hell”
Pillar: Async Client
31. Async Execution
Client
Service A
Service Z
Application Container
Service B
Service C
Service D
1
2
2
2
1
Total Latency = max(A, Z)
A = max(B, C, D) + A’s latency
32. Building the Pillars of Microservices
● Circuit breakers
● Retry logic
○ Much easier to implement w/ RxJava
● Timeouts
● Fallbacks (cached or static values)
● Netflix Hystrix
Pillar: Fault Tolerance
34. Fault Tolerance
Service B
Service A
Service C
Service A Client
10 Threads
Service B Client
5 Threads
Service C Client
5 Threads
User
Request
Fail fast, fail silent, or fallback
Application Container
36. Building the Pillars of Microservices
● Entirely Async Systems
○ Async servers, Streaming, gRPC, Netty
● Distributed task management
○ Serverless computing
● Easier/better alert definition and management
● Better tooling to create and deploy services
Future Work
37. Part 2: Containerization & Kubernetes Orchestration
01
The problem with static infrastructure
02
Kubernetes in a datacenter?
03
Challenges
38. Containerization & Kubernetes Orchestration
● Engineering org grows...
● More services…
● More infrastructure to spin up…
● Ops becomes a blocker...
Stuck in a loop
39. Containerization & Kubernetes Orchestration
● Difficult to find resources
● Slow to provision and scale
● Already have discovery!
● Metrics system must support short lived metrics
● Alerts are usually per instance
Static infrastructure and microservices do not mix!
40. Traditional Provisioning Process
● Pick ESX with available resources
● Pick IP
● Register host to Cobbler
● Register DNS entry
● Create new VM on ESX
● PXE boot VM and install OS and base configuration
● Install system dependencies (LDAP, NTP, CollectD, Sensu…)
● Install app dependencies (Java, FluentD/Filebeat, Consul,
Mongo-S…)
● Install the app
● App registers with discovery system and begins receiving traffic
42. Containerization & Kubernetes Orchestration
● Provisioning/Scaling: Kubernetes
● Monitoring: Prometheus
● Alerting: AlertManager
● Discovery: Consul + Kubernetes
● Decentralization
So how do we make this magic work?
45. Spine and Leaf Layer 3 Clos Topology
● Each leaf switch represents a Top-of-Rack switch (ToR)
● All work is performed at the leaf switch
● Each leaf switch is separate Layer 3 domain
● Each leaf is a separate BGP domain (ASN)
● No Spanning Tree Protocol issues seen in L2 networks (convergence
time, loops)
Leaf Leaf Leaf Leaf
Spine Spine
46. Spine and Leaf Layer 3 Clos Topology
● Simple to understand
● Easy to scale
● Predictable and consistent latency (hops = 2)
● Allows for Anycast IPs
Leaf Leaf Leaf Leaf
Spine Spine
47. Calico Networking
● No network overlay required
● Communicates directly with existing L3 mesh network
● BGP Peering with Top of Rack switch
● Calico supports Kubernetes NetworkPolicy firewall rules
49. ● Graphite does not scale well with ephemeral instances
● Easy to have combinatoric explosion of metrics
Traditional Monitoring & Alerting
● Application and system alerts are tightly coupled
● Difficult to create alerts on SLAs
● Difficult to route alerts
55. Microservice Pod Definition
resources:
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
● Kubernetes assumes no other processes are
consuming significant resources
● Completely Fair Scheduler (CFS)
○ Schedules a task based on CPU Shares
○ Throttles a task once it hits CPU Quota
56. Microservice Pod Definition
resources:
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
● Shares = CPU Request * 1024
● Total Kubernetes Shares = # Cores * 1024
● Quota = CPU Limit * 100ms
● Period = 100ms
57. Java in a Container
● JVM is able to detect # of cores via sysconf(_SC_NPROCESSORS_ONLN)
● Scales tasks relative to this
58. Java in a Container
● Provide a base container that calculates the container’s resources!
● Detect # of “cores” assigned
○ /sys/fs/cgroup/cpu/cpu.cfs_quota_us divided by
/sys/fs/cgroup/cpu/cpu.cfs_period_us
● Automatically tune the JVM:
○ -XX:ParallelGCThreads=${core_limit}
○ -XX:ConcGCThreads=${core_limit}
○ -Djava.util.concurrent.ForkJoinPool.common.parallelism=${core_limit
}
59. Java in a Container
● Many libraries rely on Runtime.getRuntime.availableProcessors()
○ Jetty
○ ForkJoinPool
○ GC Threads
○ That mystery dependency...
60. Java in a Container
● Use Linux preloading to override availableProcessors()
#include <stdlib.h>
#include <unistd.h>
int JVM_ActiveProcessorCount(void) {
char* val = getenv("CONTAINER_CORE_LIMIT");
return val != NULL ? atoi(val) : sysconf(_SC_NPROCESSORS_ONLN);
}
https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling
61. Communication With External Services
● Environment specific services should not be encoded in application
● Single deployment for all environments and datacenters
● Federation API expects same deployment
● Not all applications are using consul