Microservice Architecture Patterns, by Richard Langlois P. Eng.

Microservice Architecture Patterns
Richard Langlois P. Eng. , Solutions Architect, Alithya,
April 2018

Agenda
1. What is Microservice Architecture
2. What is a pattern
3. Core patterns
4. Decomposition
5. Deployment patterns
6. Cross cutting concerns
7. Communication style
8. External API
9. Service discovery
10. Reliability
11. Data management
12. Security
13. Testing
14. Observability
15. UI patterns
2

Design Patterns (1994)
Patterns at the classes level
Microservice Patterns (2018)
Patterns at the services level
Design Patterns Books
3

Microservice Architecture
Is an architectural style that structures an application as a set of loosely coupled services.
Services implements business capabilities.
4

Microservice Architecture Pattern Language
A pattern is a "reusable solution to a problem that occurs in particular context".
The Pattern language is a collection of patterns that help you architect and design an application using the microservice
architecture.
A commonly used pattern structure includes three valuable sections: forces, resulting context, and related patterns:
The forces section describes the issues that you must address when solving a problem in a given context.
The resulting context section describes the consequences of applying the pattern.
The related patterns section describes the relationship between this pattern and other patterns.
5

1. Core Patterns
Context:
• Needs to develop a server-side enterprise application. Must support a variety of different clients. application handles requests
(HTTP requests and messages) by executing business logic; accessing a database; exchanging messages with other systems;
and returning a HTML/JSON/XML response.
Problem:
• What’s the application’s deployment architecture?
Forces:
• There is a team of developers working on the application
• New team members must quickly become productive
• The application must be easy to understand and modify
• You want to practice continuous deployment of the application
• You must run multiple copies of the application on multiple machines in order to satisfy scalability and availability requirements
• You want to take advantage of emerging technologies (frameworks, programming languages, etc)
7

1. Core Patterns
Context:
• Monolithic Architecture.
• Microservice Architecture.
8

1. Core Patterns
1.1 Monolithic Architecture
Solutions:
• Build an application with a monolithic architecture. For example, a single Java WAR file.
Benefits:
• Simple to develop - IDEs support well the development of monolithic applications
• Simple to deploy - deploy the WAR file
• Simple to scale - by running multiple copies of the application behind a load balancer
Drawbacks:
• Large monolithic code base intimidates developers.
• Overloaded IDE
• Overloaded web container
• Continuous deployment is difficult - redeploy the entire application.
• Scaling the application can be difficult - only scale in one dimension - different application components have different resource
requirements - one might be CPU intensive while another might memory intensive. cannot scale each component independently
• prevents the teams from working independently.
• Requires a long-term commitment to a technology stack
9

1. Core Patterns
1.2 Microservice Architecture
Solutions:
• Define an architecture that structures the application as a set of loosely coupled, collaborating services.
• Each service implements a set of narrowly, related functions (e.g. Order management service, Customer management
service).
• Services communicate using either synchronous protocols such as HTTP/REST or asynchronous protocols such as AMQP.
• Services can be developed and deployed independently of one another.
10

1. Core Patterns
1.2 Microservice Architecture
Benefits:
• Each microservice is relatively small.
• Easier to scale development - Each (two pizza) team is responsible for one or more single service.
• Improved fault isolation.
• Each service can be developed and deployed independently.
• Eliminates any long-term commitment to a technology stack.
Drawbacks:
• Developers must deal with the additional complexity of creating a distributed system.
• Deployment complexity - operational complexity of deploying and managing a system comprised of many different service
types.
• Increased memory consumption.
11

2. Decomposition Patterns
• Decompose by business Capability
• Decompose by Subdomain
12

Context:
• Microservice architecture structures an application as a set of loosely coupled services.
• The goal of the microservice architecture is to accelerate software development by enabling continuous delivery/deployment.
• A service must be small enough to be developed by a small team and to be easily tested.
• Apply the SRP to service design as well and design services that are cohesive and implement a small set of strongly related functions.
• Decomposed in a way so that most new and changed requirements only affect a single service.
Problem:
• How to decompose an application into services?
Forces:
• Services must be cohesive.
• Services must conform to the Common Closure Principle (CCP), i.e. things that change together should be packaged together
• Services must be loosely coupled - each service as an API that encapsulates its implementation.
• Service should be testable
• Each service be small enough to be developed by a “two pizza” team, i.e. a team of 6-10 people
• Each team that owns one or more services must be autonomous.
13

2.1 Decompose by Business Capability
Solutions:
• Define services corresponding to business capabilities - something that a business does in order to generate value.
Example:
• Business capabilities of an online store include:
• Product Catalog management
• Inventory management
• Order management
• Delivery management
• Services corresponding to each of these capabilities
14

2.1 Decompose by Business Capability
Benefits:
• Stable architecture since the business capabilities are relatively stable
• Development teams are cross-functional, autonomous, and organized around delivering business value rather than technical
features
• Services are cohesive and loosely coupled
15

2.2 Decompose by Subdomain
Solutions:
• Define services corresponding to Domain-Driven Design (DDD) subdomains.
• DDD refers to the application’s problem space - the business - as the domain.
• Each subdomain corresponds to a different part of the business.
Example:
• subdomains of an online store include:
• Product catalog
• Inventory management
• Order management
• Delivery management
• Services corresponding to each of these subdomains.
16

3. Deployment Patterns
Context:
• Your system architected as a set of services.
• Each service is deployed as a set of service instances for throughput and availability.
Problem:
• How are services packaged and deployed?
Forces:
• Services are written using a variety of languages, frameworks, and framework versions
• Each service consists of multiple service instances for throughput and availability
• Service must be independently deployable and scalable
• Service instances need to be isolated from one another
• You need to be able to quickly build and deploy a service
• You need to be able to constrain the resources (CPU and memory) consumed by a service
• You need to monitor the behavior of each service instance
• You want deployment to reliable
• You must deploy the application as cost-effectively as possible
17

• Multiple service instances per host.
• Service instance per host.
• Service instance per VM.
• Service instance per Container.
• Serverless deployment.
• Service deployment platform.
18

3.1 Multiple service instances per host
Solutions:
• Run multiple instances of different services on a host (Physical or Virtual machine).
• Deploy each service instance as a JVM process (e.g. a Tomcat instance per service instance).
• Deploy multiple service instances in the same JVM (e.g. as web applications).
Benefits:
• More efficient resource utilization
Drawbacks:
• Risk of conflicting resource requirements
• Risk of conflicting dependency versions
• Difficult to limit the resources consumed by a service instance
• Difficult to monitor the resource consumption of each service instance.
• Impossible to isolate each instance
19

3.2 Single Service instance per host
Solutions:
• Deploy each single service instance on its own host.
Benefits:
• Services instances are isolated from one another
• No possibility of conflicting resource requirements or dependency versions
• A service instance can only consume at most the resources of a single host
• Straightforward to monitor, manage, and redeploy each service instance
Drawbacks:
• Potentially less efficient resource utilization compared to Multiple Services per Host because there are more hosts
20

3.3 Service instance per VM
Solutions:
• Package the service as a virtual machine image and deploy each service instance as a separate VM.
Example:
• Netflix packages each service as an EC2 AMI and deploys each service instance as a EC2 instance.
Benefits:
• Straightforward to scale the service by increasing the number of instances.
• VM encapsulates the details of the technology used to build the service.
• Each service instance is isolated.
• VM imposes limits on the CPU and memory consumed by a service instance.
• IaaS solutions such as AWS provide a mature and feature rich infrastructure for deploying and managing virtual machines. For
example,
• Elastic Load Balancer (ELB)
• Autoscaling groups
Drawbacks:
• Building a VM image is slow and time consuming
21

3.4 Service instance per Container
Solutions:
• Package the service as a (Docker) container image and deploy each service instance as a container
Example:
• Each service is packaged as a Docker image and each service instance is a Docker container.
Benefits:
• Straightforward to scale up and down a service by changing the number of container instances.
• Container encapsulates the details of the technology used to build the service.
• Each service instance is isolated.
• Container imposes limits on the CPU and memory consumed by a service instance.
• Containers are extremely fast to build and start. Docker containers also start much faster than a VM.
Drawbacks:
• Infrastructure for deploying containers is not as rich as the infrastructure for deploying virtual machines.
22

3.5 Serverless deployment
Solutions:
• Use a deployment infrastructure that hides any concept of servers.
• The code is packaged (e.g. as a ZIP file), uploaded to the deployment infrastructure.
Example:
• AWS Lambda, runs the containers on EC2 instances.
• Google Cloud Functions
• Azure Functions
Benefits:
• Eliminates the need to spend time on the undifferentiated heavy lifting of managing low-level infrastructure.
• Extremely elastic.
• Pay for each request rather than provisioning what might be under-utilized virtual machines or containers.
Drawbacks:
• Significant limitation and constraints, e.g. supports a few languages, suitable for deploying stateless applications
• Limited “input sources”.
• Applications must start quickly.
• Risk of high latency.
23

3.6 Service deployment platform
Solutions:
• Use a deployment platform, which is automated infrastructure for application deployment. It provides a service abstraction,
which is a named, set of highly available (e.g. load balanced) service instances.
Example:
• Docker orchestration frameworks including Docker swarm mode and Kubernetes.
• Serverless platforms such as AWS Lambda.
• PaaS including Cloud Foundry and AWS Elastic Beanstalk.
24

4. Cross-Cutting Concerns Patterns
• Microservice Chassis.
• Externalized Configuration.
25

4. Cross Cutting Concerns
4.1 Microservice chassis
Context:
• Spend a significant amount of time putting in place the mechanisms to handle cross-cutting concerns:
• Logging,
• Health checks,
• Metrics,
• Distributed tracing
Forces:
• Creating a new microservice should be fast and easy.
• When creating a microservice you must handle cross-cutting concerns.
Solutions:
• Build your microservices using a microservice chassis framework, which handles cross-cutting concerns.
26

4.1 Microservice chassis
Example:
• Spring Boot
• Spring Cloud
• Dropwizard’s Metrics
Benefits:
• Can quickly and easy get started with developing a microservice.
Drawbacks:
• Need a microservice chassis for each programming language/framework that you want to use.
27

4.2 Externalized configuration
Context:
• An application typically uses one or more infrastructure and 3rd party services. E.g.:
• Service registry
• Message broker.
• Database server.
Problem:
• How to enable a service to run in multiple environments (e.g. DEV, QA, UAT) without modification?
Solutions:
• Externalize all application configuration including the database credentials and network location.
• On startup, a service reads the configuration from an external source.
28

Example:
• Spring Cloud Config
Benefits:
• application runs in multiple environments without modification and/or recompilation.
Drawbacks:
• How to ensure that when an application is deployed the supplied configuration matches what is expected?
29

Configuration Management:
30

5. Communication Style Patterns
• Remote Procedure Invocation (RPI)
• Messaging
• Domain-specific protocol
31

5. Communication Style
Context:
• Services must handle requests from the application’s clients.
• Services must sometimes collaborate to handle those requests.
• They must use an inter-process communication protocol.
32

5.1 Remote Procedure Invocation (RPI)
Solutions:
• Use RPI for inter-service communication.
• Client uses a request/reply-based protocol to make requests to a service.
Example:
• Some RPI technologies:
• REST
• gRPC
• Apache Thrift
Benefits:
• Simple and familiar, no intermediate broker
• Request / reply is easy
Drawbacks:
• Usually only supports request/reply and not other interaction patterns (e.g. notifications, request/async response, publish/subscribe, publish/async response)
• Reduced availability since the client and the service must be available for the duration of the interaction
• Client needs to discover locations of service instances
33

5.1 Remote Procedure Invocation (RPI) - Continued
REST style:
34

5.2 Messaging
Solutions:
• Use asynchronous messaging for inter-service communication.
• Services communicating by exchanging messages over messaging channels.
Example:
• Apache Kafka
• RabbitMQ
Benefits:
• Loose coupling since it decouples client from services.
• Improved availability since the message broker buffers messages until the consumer is able to process them.
• Supports a variety of communication patterns including:
• request/reply, notifications,
• request/async response,
• publish/subscribe,
• publish/async response
Drawbacks:
• Additional complexity of message broker, which must be highly available.
• Request/reply-style communication is more complex
• Client needs to discover location of message broker
35

5.2 Messaging
Apache Kafka:
36

5.3 Domain-specific protocol
Solutions:
• Use a domain-specific protocol for inter-service communication.
Example:
• Some domain-specific protocols including:
• Email protocols such as SMTP and IMAP
• Media streaming protocols such as RTMP, HLS, and HDS
37

6. External API Patterns
• API gateway.
• Backend for front-end.
38

6. External API
Context:
• Building an online store, you need to develop multiple versions of the product details user interface, for each client type.
• Online store must expose product details via a REST API for use by 3rd party applications.
• Code that displays the product details needs to fetch information from all of these services.
Problem:
• How do the clients of a Microservices-based application access the individual services?
Forces:
• Granularity of APIs provided by microservices is often different than what a client needs.
• Microservices typically provide fine-grained APIs.
• Different clients need different data.
• Network performance is different for different types of clients.
• The number of service instances and their locations (host + port) changes dynamically.
• Partitioning into services can change over time and should be hidden from clients.
• Services might use a diverse set of protocols.
39

6. External API
6.1 API gateway
Solutions:
• Implement an API gateway that is the single entry point for all clients.
• API gateway handles requests in one of two ways:
• Some requests routed to the appropriate service.
• It handles other requests by fanning out to multiple services.
• API gateway can expose a different API for each client (e.g. Netflix).
• Implement security, e.g. verify that the client is authorized to perform the request.
40

6. External API
6.1 API gateway
Example:
• Netflix API gateway
• AWS API Gateway
Benefits:
• Insulates the clients from how the application is partitioned into microservices.
• Insulates the clients from the problem of determining the locations of service instances.
• Provides the optimal API for each client.
• Reduces the number of requests/roundtrips.
• Simplifies the client by moving logic for calling multiple services from the client to API gateway.
• Translates from a “standard” public web-friendly API protocol to whatever protocols are used internally.
Drawbacks:
• Increased complexity – yet another moving part.
• Increased response time due to the additional network hop through the API gateway.
41

6. External API
6. 2 Backend for front-end
Solutions:
• Defines a separate API gateway for each kind of client.
42

7. Service Discovery Patterns
• Client-side discovery
• Server-side discovery
• Service registry
• Self registration
• 3rd party registration
43

7. Service Discovery
7.1 Client-side Service Discovery
Context:
• Services typically need to call one another.
• Microservice-based application typically runs in a virtualized or containerized environments where the number of instances of a
service and their locations changes dynamically.
Problem:
• How does the client of a service discover the location of a service instance?
Forces:
• Each instance of a service exposes a remote API at a particular location (host and port)
• The number of services instances and their locations changes dynamically.
• Virtual machines and containers are usually assigned dynamic IP addresses.
• The number of services instances might vary dynamically.
44

7.1 Client-side Service Discovery - Continued
Solution:
• When making a request to a service, the client obtains the location of a service instance by querying a Service Registry.
• Service Registry knows the locations of all service instances.
45

7.1 Client-side Service Discovery - Continued
Example:
Netflix Eureka:
@Configuration
@EnableEurekaClient
@Profile(Array("enableEureka"))
class EurekaClientConfiguration {
Benefits:
• Fewer moving parts and network hops compared to Server-side Discovery.
Drawbacks:
• Couples the client to the Service Registry.
• Need to implement client-side service discovery logic for each programming language/framework used by your application.
46

7.2 Server-side Service Discovery
Solutions:
• When making a request to a service, the client makes a request via a router (a.k.a load balancer) that runs at a well known
location.
• The router queries a service registry, which might be built into the router, and forwards the request to an available service
instance.
47

7.2 Server-side Service Discovery - Continued
Example:
• AWS Elastic Load Balancer (ELB), which also function as a Service Registry.
Benefits:
• Compared to client-side discovery, the client code is simpler since it does not have to deal with discovery. Instead, a client
simply makes a request to the router.
Drawbacks:
• Unless it’s part of the cloud environment, the router is another system component that must be installed and configured.
• Router needs to be replicated for availability and capacity.
• The router must support the necessary communication protocols (e.g. HTTP, gRPC, Thrift) unless it is TCP-based router.
• More network hops are required than when using Client Side Discovery.
48

7.3 Service Registry
Context:
• Clients of a service use either Client-side discovery or Server-side discovery to determine the location of a service instance
to which to send requests.
Problem:
• How do clients of a service and/or routers know about the available instances of a service?
Forces:
• Each instance of a service exposes a remote API at a particular location (host and port)
• The number of services instances and their locations changes dynamically.
Solutions:
• Implement a service registry, which is a database of services, their instances and their locations.
• Service instances are registered with the service registry on start-up and deregistered on shutdown.
• Client of the service and/or routers query the service registry to find the available instances of a service.
49

7.3 Service Registry - Continued
Example:
• Netflix Eureka service registry
• Apache Zookeeper
• Consul
• Etcd
Benefits:
• Client of the service and/or routers can discover the location of service instances.
Drawbacks:
• Yet another infrastructure component that must be setup, configured and managed.
• Service registry instances must be deployed on fixed and well known IP addresses.
• Clients of the service registry need to know the location(s) of the service registry instances.
50

7.4 Self Registration
Context:
• Service instances must be registered with the service registry on start-up so that they can be discovered,
• and unregistered on shutdown.
Problem:
• How are service instances registered with and unregistered from the service registry?
Forces:
• Service instances must be registered with the service registry on start-up and unregistered on shutdown
• Service instances that crash must be unregistered from the service registry
• Service instances that are running but incapable of handling requests must be unregistered from the service registry
Solutions:
• A service instance is responsible for registering itself with the service registry.
• The client must typically periodically renew its registration so that the registry knows it is still alive.
• On shutdown, the service instance unregisters itself from the service registry.
51

7.4 Self Registration - Continued
Example:
• Eureka Service Registry
Benefits:
• Service instance knows its own state so can implement a state model that’s more complex than UP/DOWN, e.g. STARTING,
AVAILABLE.
Drawbacks:
• Couples the service to the Service Registry
• Must implement service registration logic in each programming language/framework that you use to write your services.
• A service instance that is running yet unable to handle requests will often lack the self-awareness to unregister itself from the service
registry.
52

7.5 3rd Party Registration
Context:
• Service instances must be registered with the service registry on start-up so that they can be discovered and unregistered on
shutdown.
Problem:
• How are service instances registered with and unregistered from the service registry?
Forces:
• Service instances must be registered with the service registry on start-up and unregistered on shutdown.
• Service instances that crash must be unregistered from the service registry.
• Service instances that are running but incapable of handling requests must be unregistered from the service registry.
Solutions:
• A 3rd party registrar is responsible for registering and unregistering a service instance with the service registry.
• When the service instance starts up, the registrar registers the service instance with the service registry.
• When the service instance shuts downs, the registrar unregisters the service instance from the service registry.
53

7.5 3rd Party Registration - Continued
Example:
• AWS Autoscaling Groups automatically (un)registers EC2 instances with Elastic Load Balancer
• Clustering frameworks such as Kubernetes (un)register service instances with the built-in/implicit registry
Benefits:
• The service code is less complex since it is not responsible for registering itself.
• The registrar can perform health checks on a service instance and register/unregister the instance based the health check.
Drawbacks:
• The 3rd party registrar might only have superficial knowledge of the state of the service instance, e.g. RUNNING or NOT
RUNNING and so might not know whether it can handle requests.
• Unless the registrar is part of the infrastructure it’s another component that must be installed, configured and maintained.
54

7.5 3rd Party Registration - Continued
Kubernetes with its registry:
55

8. Reliability Patterns
• Circuit Breaker
56

8. Reliability
8.1 Circuit Breaker
Context:
• When one service synchronously invokes another there is always the possibility that the other service is unavailable or is
exhibiting such high latency it is essentially unusable.
• Previous resources such as threads might be consumed in the caller while waiting for the other service to respond.
• This might lead to resource exhaustion, which would make the calling service unable to handle other requests.
• The failure of one service can potentially cascade to other services throughout the application.
Problem:
• How to prevent a network or service failure from cascading to other services?
57

8. Reliability
8.1 Circuit Breaker - Continued
Solutions:
• A service client should invoke a remote service via a proxy that functions in a similar fashion to an electrical circuit breaker.
• When the number of consecutive failures crosses a threshold, the circuit breaker trips, and for the duration of a timeout period
all attempts to invoke the remote service will fail immediately.
• After the timeout expires the circuit breaker allows a limited number of test requests to pass through.
• If those requests succeed the circuit breaker resumes normal operation.
• Otherwise, if there is a failure, the timeout period begins again.
58

8. Reliability
Benefits:
• Services handle the failure of the services that they invoke
Drawbacks:
• Challenging to choose timeout values without creating false positives or introducing excessive latency.
59

8. Reliability
Example:
OSS Netflix Hystrix provides an implementation (@HystrixCommand annotation) .
@HystrixCommand(fallbackMethod = "reliable")
public String readingList() {
URI uri = URI.create("http://localhost:8090/recommended");
return this.restTemplate.getForObject(uri, String.class);
}
public String reliable() {
return "Code Etique Alithya";
}
60

9. Data Management Patterns
• Database per Service.
• Shared database.
• Saga.
• API Composition.
• CQRS.
• Event sourcing.
• Transaction log tailing.
• Database triggers.
• Application events.
61

9. Data Management
9.1 Database per Service
Context:
• Most services need to persist data in some kind of database.
Problem:
• What’s the database architecture in a microservices application?
Forces:
• Services must be loosely coupled so that they can be developed, deployed and scaled independently.
• Some business transactions must enforce invariants that span multiple services.
• Other business transactions, must update data owned by multiple services.
• Some queries must join data that is owned by multiple services.
• Databases must sometimes be replicated and sharded in order to scale.
• Different services have different data storage requirements (relational, NoSQL).
62

9. Data Management
Solution:
• Keep each microservice’s persistent data private to that service and accessible only via its API.
• For a relational database then the options are:
• Private-tables-per-service
• Schema-per-service
• Database-server-per-service
63

9. Data Management
Benefits:
• Helps ensure that the services are loosely coupled.
• Each service can use the type of database that is best suited to its needs.
Drawbacks:
• Implementing business transactions that span multiple services is not straightforward.
• Services publish events when they update data. Other services subscribe to events and update their data in response.
• Complexity of managing multiple SQL and NoSQL databases
64

9. Data Management
9.2 Shared database
Problem:
• What’s the database architecture in a microservices application?
Solutions:
• Use a single database that is shared by multiple services.
• Each service freely accesses data owned by other services using local ACID transactions.
Benefits:
• A developer uses familiar and straightforward ACID transactions to enforce data consistency
• A single database is simpler to operate
Drawbacks:
• Development time coupling
• Runtime coupling
• Single database might not satisfy the data storage and access requirements of all services.
65

9. Data Management
9.3 Saga
Context:
• Need a mechanism to ensure data consistency across services.
Problem:
• How to maintain data consistency across services?
Forces:
• 2 Phase Commit is not an option.
Solution:
• Implement each business transaction that spans multiple services as a saga. A saga is a sequence of local transactions. Each
local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a
local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that
undo the changes that were made by the preceding local transactions.
66

9. Data Management
9.3 Saga
Example:
An e-commerce application that uses this approach would create an order using a saga that consists of the following local transactions:
• The Order Service creates an Order in a pending state
• The Customer Service attempts to reserve credit for that Order.
• The Order Service changes the state of the order to either approved or cancelled
• Each local transaction publishes an event or message that triggers the next step. For example, in the CreateOrder saga, the OrderService
publishes an OrderCreated event and the Customer Service publishes either a Credit Reserved event or a CreditLimitExceeded event.
67

9. Data Management
9.3 Saga – (3 of 3)
Benefits:
• It enables an application to maintain data consistency across multiple services without using distributed transactions
Drawbacks:
• The programming model is more complex (e.g. design compensating transaction).
68

9. Data Management
9.4 API Composition
Context:
• Database per service pattern
• No longer straightforward to implement queries that join data from multiple services.
Problem:
• How to implement queries in a microservice architecture?
Solutions:
• Implement a query by defining an API Composer, which invoke the services that own the data and performs an in-memory join of the results.
69

9. Data Management
9.4 API Composition
Example:
• An API Gateway often does API composition.
Benefits:
• It a simple way to query data in a microservice architecture
Drawbacks:
• Some queries would result in inefficient, in-memory joins of large datasets.
70

9. Data Management
9.5 CQRS
Context:
• Database per service pattern
• No longer straightforward to implement queries that join data from multiple services.
Problem:
• How to implement queries in a microservice architecture?
Solutions:
• Split the application into two parts: the command-side and the query-side.
• Command-side handles create, update, and delete requests and emits events when data changes.
• Query-side handles queries by executing them against one or more materialized views that are kept up to date by subscribing
to the stream of events emitted when data changes.
71

9. Data Management
9.5 CQRS
Benefits:
• Necessary in an event sourced architecture.
• Improved separation of concerns = simpler command and query models.
• Supports multiple denormalized views that are scalable and performant.
Drawbacks:
• Increased complexity
• Potential code duplication
• Replication lag/eventually consistent views
72

9. Data Management
9.6 Event sourcing
Context:
• In order to be reliable, services must atomically publish events whenever their state changes.
• It is not viable to use a distributed transaction that spans the database and the message broker.
Problem:
• How to reliably/atomically publish events whenever state changes?
Forces:
• 2PC is not an option
Solutions:
• Event store is difficult to query since it requires typical queries to reconstruct the state of the business entities.
• Whenever the state of a business entity changes, a new event is appended to the list of events.
• The application reconstructs an entity’s current state by replaying the events.
• Applications persist events in an event store, which is a database of events.
• The event store also behaves like a message broker.
• It provides an API that enables services to subscribe to events.
• When a service saves an event in the event store, it is delivered to all interested subscribers.
73

9. Data Management
9.6 Event sourcing
Example:
• Instead of simply storing the current state of each order as a row in an ORDERS table, the application persists each Order as a
sequence of events. The CustomerService can subscribe to the order events and update its own state.
74

9. Data Management
9.6 Event sourcing
Benefits:
• It solves one of the key problems in implementing an event-driven architecture.
• Makes it possible to reliably publish events whenever state changes.
• Because it persists events rather than domain objects, it mostly avoids the object-relational impedance mismatch problem.
• It provides a 100% reliable audit log of the changes made to a business entity.
• Makes it possible to implement temporal queries that determine the state of an entity at any point in time.
• Event sourcing-based business logic consists of loosely coupled business entities that exchange events.
Drawbacks:
• Learning curve because It is a different and unfamiliar programming style.
• Event store is difficult to query since it requires typical queries to reconstruct the state of the business entities.
75

9. Data Management
9.7 Transaction log tailing
Context:
•
Problem:
Forces:
Solutions:
• Tail the database transaction log and publish each change as an event.
Example:
• Eventuate Local
76

9. Data Management
9.7 Transaction log tailing
Benefits:
• No 2PC
• No application changes required
• Guaranteed to be accurate
Drawbacks:
• Relatively obscure
• Database specific solutions
• Tricky to avoid duplicate publishing
• Low level DB changes make difficult to determine the business level events
77

9. Data Management
9.8 Database triggers
Context:
Problem:
Forces:
Solutions:
• One or more database triggers insert events into an EVENTS table, which is polled by a separate process that publishes the
events.
78

9. Data Management
9.9 Application events
Context:
Problem:
Forces:
Solutions:
• The application insert events into an EVENTS table as part of the local transaction. A separate process polls the EVENTS table and publishes the events to a message
broker.
Benefits:
• High level domain events
• No 2PC
Drawbacks:
• Requires changes to the application, which are potentially error prone.
• Tricky to avoid duplicate publishing
• Tricky to publish events in order
• Only works for SQL and some NoSQL databases
79

10. Security Patterns
• Access Token
80

10. Security
10.1 Access Token
Context:
• The application consists of numerous services.
• The API gateway is the single entry point for client requests.
• It authenticates requests, and forwards them to other services, which might in turn invoke other services.
Problem:
• How to communicate the identity of the requestor to the services that handle the request?
Forces:
• Services often need to verify that a user is authorized to perform an operation
Solutions:
1. The API Gateway authenticates the request,
2. Passes an access token (e.g. JSON Web Token) that securely identifies the requestor in each request to the services.
3. Service can include the access token when makes requests to other services.
81

10. Security
10.1 Access Token
Example:
• JSON Web Token (JWT)
Benefits:
• The identity of the requestor is securely passed around the system.
• Services can verify that the requestor is authorized to perform an operation.
82

11. Testing
Problem:
• How to easily test that a service provides an API that its clients expect?
Forces:
• End to end testing (i.e. tests that launch multiple services) is difficult, slow, brittle, and expensive.
83

11. Testing
Five Layers of Microservice Testing
84

11. Testing Patterns
• Service Component Test.
• Service Integration Contract Test.
85

11. Testing
11.1 Service Component Test
Solution:
• A test suite that tests a service in isolation using test doubles for any services that it invokes.
Example:
• Spring Cloud Contract (Consumer Driven Contracts)
Benefits:
• Testing a service in isolation is easier, faster, more reliable and cheap.
Drawbacks:
• Tests might pass but the application may fail in production.
• How to ensure that the test doubles always correctly emulate the behavior of the invoked services?
86

11. Testing
11.1 Service Component Test
87

11. Testing
11.2 Service Integration Contract Test
Problem:
• How to easily test that a service provides an API that its clients expect?
Solution:
• A test suite for a service that is written by the developers of another service that consumes it.
• The test suite verifies that the service meets the consuming service’s expectations.
Example:
• Spring Cloud Contract
Benefit:
• Testing a service in isolation is easier, faster, more reliable and cheap
Drawbacks:
• Tests might pass but the application may fail in production.
• How to ensure that the consumer provided tests match what the consumer actually requires?
88

12. Observability Patterns
• Log aggregation.
• Application metrics.
• Audit logging.
• Distributed tracing.
• Exception tracking.
• Health check API.
• Log deployments and changes.
89

12. Observability
12.1 Log aggregation
Context:
• The application consists of multiple services and service instances that are running on multiple machines.
• Requests often span multiple service instances.
• Each service instance generates writes information about what it is doing to a log file.
Problem:
• How to understand the behavior of an application and troubleshoot problems?
Forces:
• Any solution should have minimal runtime overhead
Solutions:
• Use a centralized logging service that aggregates logs from each service instance.
• The users can search and analyze the logs.
• They can configure alerts that are triggered when certain messages appear in the logs.
Examples:
• AWS Cloud Watch, Logstash (ELK)
Drawbacks:
• Handling a large volume of logs requires substantial infrastructure.
90

12. Observability
12.1 Log aggregation using ELK
91

12. Observability
12.2 Application metrics
Context:
• Applied the Microservice architecture pattern.
Problem:
Forces:
• Any solution should have minimal runtime overhead
Solutions:
• Instrument a service to gather statistics about individual operations. Aggregate metrics in centralized metrics service, which provides reporting and alerting.
Examples:
Java Metrics Library (Dropwizard)
Prometheus
AWS Cloud Watch
Benefits:
• provides deep insight into application behavior.
Drawbacks:
• Metrics code is intertwined with business logic making it more complicated.
• Aggregating metrics can require significant infrastructure.
92

12. Observability
12.2 Application metrics
93

12. Observability
12.3 Audit logging
Context:
Problem:
• How to understand the behavior of users and the application and troubleshoot problems?
Forces:
• It is useful to know what actions a user has recently performed: customer support, compliance, security, etc.
Solutions:
• Record user activity in a database.
Benefits:
• Provides a record of user actions
Drawbacks:
• auditing code is intertwined with the business logic, which makes the business logic more complicated
94

12. Observability
12.4 Distributed tracing
Context:
Problem:
Forces:
• External monitoring only tells you the overall response time and number of invocations
• No insight into the individual operations
Solutions:
Instrument services with code that
Assigns each external request a unique external request id
Passes the external request id to all services that are involved in handling the request
Includes the external request id in all log messages
Records information (e.g. start time, end time) about the requests and operations performed when handling a external
request in a centralized service
95

12. Observability
12.4 Distributed tracing
Examples:
• Spring Cloud Sleuth instruments Spring components to gather trace information and can delivers it to a Zipkin Server, which gathers and
displays traces.
Benefits:
• provides useful insight into the behavior of the system including the sources of latency.
• Enable the developers to search logs using external request id.
Drawbacks:
Aggregating and storing traces can require significant infrastructure.
96

12. Observability
12.5 Exception tracking
Context:
• Errors sometimes occur when handling requests.
• When an error occurs, a service instance throws an exception, which contains an error message and a stack trace.
Problem:
Forces:
• Exceptions must be de-duplicated, recorded, investigated by developers and the underlying issue resolved
Solutions:
• Report all exceptions to a centralized exception tracking service that aggregates and tracks exceptions and notifies
developers.
97

12. Observability
12.5 Exception tracking
Examples:
• Sentry Datadog
• PagerDuty
Benefits:
• Easier to view exceptions and track their resolution
Drawbacks:
• Exception tracking service is additional infrastructure
98

12. Observability
12.6 Health check API
Context:
• Service instance can be incapable of handling requests yet still be running.
• monitoring system should generate a alert.
• Load balancer or service registry should not route requests to the failed service instance.
Problem:
• How to detect that a running service instance is unable to handle requests
Forces:
• Alert should be generated when a service instance fails.
• Requests should be routed to working service instances
Solutions:
• Service has an health check API endpoint (e.g. HTTP /health) that returns the health of the service.
• A health check client (monitoring service, service registry or load balancer) periodically invokes the endpoint to check the health of the service instance.
Examples:
Spring Boot Actuator configures “/health” HTTP endpoint that invokes helth check logic.
Benefits:
99

12. Observability
12.6 Health Check API
Examples:
• Spring Boot Actuator configures “/health” HTTP endpoint that invokes health check logic.
Benefits:
• Enables the health of a service instance to be periodically tested.
Drawbacks:
• Service instance might fail between health checks and so requests might still be routed to a failed service instance
10
0

12. Observability
12.7 Log deployments and changes
Context:
Problem:
Forces:
• It useful to see when deployments and other changes occur since issues usually occur immediately after a change
Solutions:
• A deployment tool can publish a pseudo-metric whenever it deploys a new version of a service.
Examples:
• AWS Cloud Trail provides logs of AWS API calls.
Benefits:
• Enables deployments and changes to be easily correlated with issues leading to faster resolution.
10
1

13. UI Patterns
Context:
• Microservices are developed by business capability/subdomain-oriented teams that are also responsible for the user
experience.
• Some UI screens/pages display data from multiple service.
• Each data item corresponds to a separate service and the different teams are responsible for how it is displayed.
Problem:
• How to implement a UI screen or page that displays data from multiple services?
10
2

13. UI Patterns
This Amazon.ca web page is composed of 11 microservices
10
3

13. UI Patterns
• Server-side page fragment composition.
• Client-side UI composition
10
4

13. UI Patterns
13.1 Server-side page fragment composition
Solution:
• Each team develops a web application that implements the region of the page for their service.
• A UI team implements the page templates that build pages by performing server-side aggregation (e.g. server-side include
style) of the service-specific HTML fragments.
10
5

13. UI Patterns
13.2 Client-side UI composition
Solution:
• Each team develops a client-side UI component that implements the region of page for their service.
• A UI team implements the page skeletons that build pages/screens by composing multiple, service-specific UI
components.
10
6

References
• Microservice Patterns book, Chris Richardson, 2018
• A Pattern Language for Microservices (http://microservices.io), Chris Richardson
• Example applications, http://eventuate.io/exampleapps.html
• FTGO Sample application: https://github.com/microservice-patterns/ftgo-application
10
7

Microservice Architecture Patterns, by Richard Langlois P. Eng.

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Microservice Architecture Patterns, by Richard Langlois P. Eng.

Similar to Microservice Architecture Patterns, by Richard Langlois P. Eng. (20)

More from Richard Langlois P. Eng.

More from Richard Langlois P. Eng. (7)

Recently uploaded

Recently uploaded (20)

Microservice Architecture Patterns, by Richard Langlois P. Eng.