Distributed applications like microservices shift some of their complexities into the interaction of services. Such a service mesh, which can have hundreds of runtime instances, is very difficult to manage. You will be concerned with some of the following questions: Which service will be requested by which other services in which version and how often depending on the request content? How can you test the interaction and how can you replace single services with new ones? These and other questions will be discussed in this session. Tools to make your live easier with a service mesh will also be introduced.

1. Service Mesh
kilometer 30
in a
microservice marathon
Michael Hofmann
Hofmann IT-Consulting
info@hofmann-itconsulting.de

2. marathon at km 30 ...
●
fat burning, extreme fatigue
●
decision about finishing the marathon
●
proper training
●
spaghetti party, eat and drink during the run
●
mental preparation
how to prevent?
„the man with the hammer“

3. km 30 at microservices?
●
microservice projects start small (greenfield, splitting a monolith)
●
at the beginning without version problems
●
multiple versions parallel in production
●
number of services increases
●
establishing a service chain
●
how to test the interaction over different versions?
●
creeping loss of survey: who with whom in which version?

4. already there?
source: https://twitter.com/Werner/status/741673514567143424
(Werner Vogels, CTO Amazon)
source: Adrian Cockcroft (Netflix) / Martin Fowler

5. what else at km 30?
●
complexity lies between services
●
fallacies of distributed systems
●
who cares about?
– container system?
– resilience frameworks?
should be transparent to the application!
The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn't change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

6. where to place resilience?
●
frameworks (Hystrix, Failsafe, MicroProfile Fault Tolerance)
●
problems:
– suitable pattern recognized in production (another deployment!)
– framework dependant on progamming language
– mix of framework versions scattered over all services
– dependencies to other frameworks (service call → load balancing → service registry)
– service registry usable for other services? multiple service registries? (tight coupling
of service and infrastructure components!)
– Learning curve for different frameworks
●
alternative: Service Mesh tool

7. Service Mesh
„The term service mesh is used to describe the network of
microservices that make up such application and the interactions
between them.“ (istio.io)
you cannot manage a Service Mesh (big ball of mud) without
tooling!
requirements:
●
manage calls on layer 7 (application layer)
●
resilience, routing, security and telemetry
●
decentralized & transparent for services (implementation independent)

8. Service Mesh functions
●
service discovery
●
load balancing
●
resilience
●
dynamic routing (blue/green deployments, canary releasing, traffic
shadowing)
●
observability (metrics, tracing)
●
end-to-end authentication, access control
●
rate limiting

9. Service Mesh tool when?
●
high number of different services
●
invocation depth of services > 1
●
wish of uniform implementation of resilience
●
contemporary testing strategies: test in production (necessary?)

10. Service Mesh tools
comparison of Istio & Linkerd: https://abhishek-tiwari.com/a-sidecar-for-your-service-mesh/

11. Istio
IBM (Amalgam8)
content-based routing
(extended)
service discovery
resilience
load balancing
Google (Istio)
content-based routing
rate limiting
ACLs
telemetry
Kubernetes integration
Lyft (Envoy)
proxy (sidecar)

12. Istio architecture
source: istio.io
data plane
control plane

13. Envoy Proxy
design goal: „The network should be transparent to applications. When
network and application problems do occur it should be easy to
determine the source of the problem.“
●
container deployed together with service
●
„Man-in-the-Middle“
●
as a sidecar transparent for service
●
service discovery, load balancing, resilience, health checks, metrics,
fault injection
●
communication with Mixer (telemetry) und Pilot (policies)

14. Mixer
source: istio.io

15. Pilot
source: istio.io

16. Citadel
source: istio.io

17. automatic sidecar injection
heart of Istio:
automatic provisioning of sidecar (Envoy Proxy)
●
modification of ip-tables in pod
●
health checks through sidecar to service
●
restart pod if any of the two containers crashed
kubectl label
namespace my-namespace
istio-injection=enabled
helm install
install/kubernetes/helm/istio
--name istio --namespace
istio-system

18. traffic management
source: istio.io

19. traffic management
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
weight: 95
- destination:
host: reviews
subset: v2
weight: 5
●
95%: reviews v1
●
5%: reviews v2
kubectl apply
-f reviews-v1-95-v2-5.yaml

20. traffic management
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- match:
- headers:
x-canary:
exact: „v2“
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
HTTP header:
●
x-canary=v2:
– reviews v2
●
without header:
– v1
●
necessary sequence:
-match vor -route

21. apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
subsets:
- name: v1
labels:
version: v1
trafficPolicy:
connectionPool:
http:
http1MaxPendingRequests: 2
tcp:
maxConnections: 5
outlierDetection:
http:
baseEjectionTime: 15m
consecutiveErrors: 3
interval: 1m
circuit breaker
●
max. 5 connections to
reviews v1 & max. 2 pending
requests
●
everything else: HTTP status
code 503
●
at 3 succcessive errors (5xx)
within 1 min.: upstream-host
will be ejected from pool for
15 min.

22. retry - timeout
●
max. 3 attempts with 2s
timeout for each try
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
retries:
attempts: 3
perTryTimeout: 2s

23. fault injection apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- fault:
delay:
fixedDelay: 7s
percent: 90
abort:
httpStatus: 500
percent: 10
route:
- destination:
host: reviews
subset: v1
- route:
- destination:
host: reviews
subset: v1
●
90%: 7s delay
●
10%: HTTP status code 500

24. traffic shadowing
●
purpose: parallel testing of
a new version v2
●
100% of requests to
reviews v1 get mirrored on
reviews v2
●
response from reviews v2
will be discarded by Istio
(but executed in service!)
...
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
...
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
weight: 100
mirror:
host: reviews
subset: v2

25. monitoring
source: istio.io

26. tracing
source: istio.io

27. service graph
source: istio.io
little „Big Ball of Mud“

28. recommendations
●
training for marathon: start early in project to use tool for
Service Mesh
●
integrate into CI/CD (zero downtime deployments!)
●
place resilience preferred into sidecar
●
integrate tracing/monitoring into existing infrastructure
●
establish new test strategy