The document discusses how microservices, events, and Docker form a perfect trio for developing and deploying applications. It covers how microservices can help accelerate development but also introduce complexity that events and Docker can help address. Events are proposed to maintain data consistency between microservices rather than distributed transactions. Docker is presented as a way to containerize microservices for improved isolation, manageability and efficient deployment and scaling. The overall presentation argues this combination of techniques can help build large, complex applications.

2. @crichardson
Microservices + Events +
Docker = A Perfect Trio
Chris Richardson
Founder of Eventuate.io
Founder of the original CloudFoundry.com
Author of POJOs in Action
@crichardson
chris@chrisrichardson.net
http://eventuate.io

3. @crichardson
Presentation goal
http://muppet.wikia.com/wiki/Brought_to_You_by_the_Number_3
Microservices,
Events, and
Docker are a great
way to develop and
deploy applications

4. @crichardson
About Chris

5. @crichardson
About Chris
Consultant and trainer
focusing on modern
application architectures
including microservices
(http://www.chrisrichardson.net/)

6. @crichardson
About Chris
Founder of a startup that is creating
a platform that makes it easier for
developers to write transactional
microservices
(http://eventuate.io)

7. @crichardson
For more information
http://learnmicroservices.io

8. @crichardson
Agenda
Monolith vs. microservices
Event-driven microservices
Developing and deploying microservices using Docker

9. @crichardson
Let’s imagine you are building
a large, complex application,
e.g. an online store

10. @crichardson
Successful software development
Architecture
Process Organization
Agile
Continuous delivery
…
Small,
autonomous,
teams
3

11. @crichardson
?Architecture

12. @crichardson

13. @crichardson
The monolithic architecture
Tomcat
Browser
WAR
SQL
database
HTML
REST/JSON
Client
App
Simple to ….
Develop
Test
Deploy
Scale
Catalog
Module
Reviews
Module
Orders
Module
StoreFront UI
Module

14. @crichardson
But
successful
applications
keep
growing
….

15. @crichardson
Monolithic architecture
Process Organization
Agile
Continuous delivery
…
Small,
autonomous,
teams

16. @crichardson
Apply functional
decomposition
X axis
- horizontal duplication
Z
axis
-data
partitioning
Y axis -
functional
decomposition
Scale
by
splitting
sim
ilar
things
Scale by
splitting
different things
3

17. @crichardson
Microservice architecture
Browser
Mobile
Device
Store
Front UI
API
Gateway
Catalog
Service
Review
Service
Order
Service
…
Service
Catalog
Database
Review
Database
Order
Database
…
Database
HTML
REST
REST
REST

18. @crichardson
Microservice architecture
Process Organization
Agile
Continuous delivery
…
Small,
autonomous,
teams✔ ✔

19. @crichardson
Drawbacks
Complexity

20. @crichardson
Drawbacks
Complexity of developing a distributed system
Implementing inter-process communication
Handling partial failures
Complexity of implementing business transactions that span multiple
databases (without 2PC)
Complexity of testing a distributed system
Complexity of deploying and operating a distributed system
Managing the development and deployment of features that span
multiple services
Fortunately solutions exists

21. @crichardson
The benefits typically
outweigh the drawbacks
for
large, complex applications

22. @crichardson
Issues to address
How to deploy the services?
How do the services communicate?
How do clients of the application communicate with the
services?
How to partition the system into services?
How to deal with distributed data management problems?
….

23. @crichardson
Agenda
Monolith vs. microservices
Event-driven microservices
Developing and deploying microservices using Docker

24. Data management patterns

25. @crichardson
The Database
Shared database
Order Service Customer Service … Service
Order table
Customer
table
…
orderTotal creditLimit
Tight coupling
Simple and
ACID

26. @crichardson
Database per service
Order Service Customer Service
Order Database Customer Database
Order table
Customer
table
orderTotal creditLimit
Loose coupling " but more complex # and….

27. @crichardson
2PC (aka. distributed
transactions)
is not viable choice
for most modern applications

28. @crichardson
Customer management
How to maintain data consistency
without 2PC?
Order management
Order Service
placeOrder()
Customer Service
updateCreditLimit()
Customer
creditLimit
...
has ordersbelongs toOrder
total
Invariant:
sum(open order.total) <= customer.creditLimit
?

29. @crichardson
Event-driven architecture

30. @crichardson
Use event-driven, eventually
consistent order processing
Order
Service
Customer
Service
Order created
Credit Reserved
Credit Check Failed
Place Order
OR

31. @crichardson
How atomically update
database and publish an event
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEvent
dual write problem
?

32. @crichardson
Reliably publish events when
state changes

33. @crichardson
Use event-sourcing
Event table
Aggregate
type
Event
id
Aggregate
id
Event
data
Order 902101 …OrderApproved
Order 903101 …OrderShipped
Event
type
Order 901101 …OrderCreated

34. @crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodically snapshot to avoid loading all events

35. @crichardson
Benefits of event sourcing
Solves data consistency issues in a Microservice/NoSQL based
architecture
Reliable event publishing: publishes events needed by predictive
analytics etc, user notifications,…
Eliminates O/R mapping problem (mostly)
Reifies state changes:
Built in, reliable audit log
temporal queries
Preserved history More easily implement future requirements

36. @crichardson
Drawbacks of event sourcing
Requires application rewrite
Weird and unfamiliar style of programming
Events = a historical record of your bad design decisions
Must handle duplicate events: idempotent handlers or
duplicate detection
Querying the event store can be challenging

37. But what about queries?

38. @crichardson
Find recent, valuable
customers
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service
What if event
sourcing is
used?…. is no longer easy

39. @crichardson
Command Query Responsibility
Segregation (CQRS)
Command side
Commands
Aggregate
Event Store
Events
Query side
Queries
Materialized
View
Events
POST
PUT
DELETE
GET
MongoDB
Redis
Neo4j
SQL
ElasticSearch
…
More complex # but high performance, scalable views "

40. @crichardson
Agenda
Monolith vs. microservices
Event-driven microservices
Developing and deploying microservices using Docker

41. @crichardson
We have applied the microservices pattern:
How to deploy the 10s or 100s of services?

42. @crichardson
Forces
Services are written using a variety of languages, frameworks,
and framework versions
Each service consists of multiple service instances for
throughput and availability
Building and deploying a service must be fast
Services must be deployed and scaled independently
Service instances need to be isolated
Deployment must be reliable and cost-effective

43. @crichardson

44. @crichardson

45. @crichardson
VM
VM
Pattern: Service per Container
host
Service
Container
image
Container
Service
Container
Service
Container
Service
packaged as
deployed as

46. @crichardson
Benefits of containers
Great isolation
Great manageability
Container encapsulates implementation technology
Efficient resource utilization
Fast deployment

47. @crichardson
Docker
(Compose)
also simplifies
development

48. Running infrastructure services
on development machines
Typical services needs a
database, message
broker, …
Making sure every
developer installs the
correctly version = painful
rabbitmq:
image: rabbitmq:3.5.3
ports:
- "5672:5672"
- "15672:15672"
mongodb:
image: mongo:3.0.4
ports:
- "27017:27017"
command: mongod --smallfiles

49. @crichardson
Deploying microservices for
end-to-end testing
restfulservice:
image: java:openjdk-8u45-jdk
working_dir: /app
volumes:
- ./spring-boot-restful-service/build/libs:/app
command: java -jar /app/spring-boot-restful-service.jar
ports:
- "8081:8080"
links:
- rabbitmq
- mongodb
environment:
SPRING_DATA_MONGODB_URI: mongodb://mongodb/userregistration
SPRING_RABBITMQ_HOST: rabbitmq

50. @crichardson
Jenkins-based deployment
pipeline
Build & Test
microservice
Build & Test
Docker
image
Deploy
Docker
image
to registry
One pipeline per microservice

51. @crichardson
Smoke testing docker images
Smoke test
Docker
daemon
Service
containerGET /health
POST /containers/create
creates
POST /containers/{id}/start
Docker daemon must listen on
TCP port

52. @crichardson
Running on Docker!
EC2 Instance
Jenkins
Container
Artifactory
container
EBS volume
/jenkins-
home
/gradle-home
/artifactory-
home

53. @crichardson
Summary
Use microservices to accelerate development
Use an event-driven architecture to maintain data consistency
Use Docker to simplify development and deployment

54. @crichardson
@crichardson chris@chrisrichardson.net
http://learnmicroservices.io
Questions?