JAX London Presentation 2014

1. @crichardson
Developing applications
with a microservice
architecture
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com

2. Presentation goal
How decomposing applications into
@crichardson
microservices
improves deployability and scalability
and
simplifies the adoption of new
technologies

3. @crichardson
About Chris

4. @crichardson
About Chris

5. @crichardson
About Chris
Founder of a buzzword compliant (stealthy, social, mobile,
big data, machine learning, ...) startup
Consultant helping organizations improve how they
architect and deploy applications using cloud computing,
micro services, polyglot applications, NoSQL, ...

6. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Using an API gateway
Inter-service communication mechanisms

7. @crichardson
Let’s imagine you are
building an online store

8. @crichardson
Traditional application
architecture
StoreFrontUI
Product Info
Service
Recommendation
Service
Tomcat
Browser/
Client
WAR/EAR
MySQL
Database
Review Service
Simple to
develop
test
deploy
Load
balancer
scale
Spring MVC
Spring
Hibernate
Order Service
HTML
REST/JSON

9. @crichardson
But big, complex, monolithic
applications
⇒
big problems

10. @crichardson
Intimidates developers

11. @crichardson
Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in
one basket

12. @crichardson
Overloads your IDE and
container
Slows down development

13. @crichardson
Obstacle to scaling
development
I want
to update the UI
But
the backend is not working
yet!
Lots of coordination and
communication required

14. Requires long-term commitment
to a technology stack
@crichardson

15. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Using an API gateway
Inter-service communication mechanisms

16. @crichardson

17. @crichardson
The scale cube
X axis
- horizontal duplication
Z axis - data partitioning
Y axis -
functional
decomposition
similar
splitting by things
Scale Scale by
splitting
different things

18. @crichardson
Y-axis scaling - application level
WAR
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

19. @crichardson
Y-axis scaling - application level
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

20. @crichardson
Y-axis scaling - application level
Product Info
Service
Product Info
Recommendation
Service
Review
Service
Order
Service
Browse Products
UI
Checkout UI
Order management
UI
Account
management UI
Apply X-axis and Z-axis scaling
to each service independently

21. Service deployment options
@crichardson
Isolation, manageability
VM or Physical Machine
Docker/Linux container
JVM
JAR/WAR/OSGI bundle/...
Density/efficiency

22. @crichardson
Partitioning strategies...
Partition by noun, e.g. product info service
Partition by verb, e.g. Checkout UI
Single Responsibility Principle
Unix utilities - do one focussed thing well

23. @crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art

24. @crichardson
Example micro-service
require 'sinatra'
post '/' do
phone_number = params[:From]
registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}"
<<-eof
<Response>
<Sms>To complete registration please go to #{registration_url}</Sms>
</Response>
eof
end
Responds to incoming SMS messages
via Twilio

25. @crichardson
More service, less micro
But more realistically...
Focus on building services that make
development and deployment easier
- not just tiny services

26. @crichardson
Real world examples
http://techblog.netflix.com/
~600 services
http://highscalability.com/amazon-architecture
100-150 services to build a page
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128

27. @crichardson
There are many benefits

28. @crichardson
Smaller, simpler apps
Easier to understand and develop
Less jar/classpath hell - who needs OSGI?
Faster to build and deploy
Reduced startup time - important for GAE

29. Scales development:
develop, deploy and scale
each service independently
@crichardson

30. @crichardson
Improves fault isolation

31. Eliminates long-term commitment
to a single technology stack
@crichardson
Modular, polyglot, multi-framework
applications

32. @crichardson
Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving

33. Easily try other technologies
@crichardson
... and fail safely

34. But there are drawbacks
@crichardson

35. @crichardson
Complexity

36. Complexity of developing a
@crichardson
distributed system
http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

37. @crichardson
Multiple databases
&
Transaction management
e.g. Fun with eventual consistency
Come to my 11.30 am talk

38. @crichardson
Complexity of testing a
distributed system
http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

39. Complexity of deploying and
operating a distributed
@crichardson
system
http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
You need a lot of automation

40. @crichardson
Developing and deploying
features that span multiple
services requires careful
coordination

41. @crichardson
When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful

42. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Using an API gateway
Inter-service communication mechanisms

43. Let’s imagine that you want to
display a product’s details...
Product
Info Reviews
Recommendations
@crichardson

44. Directly connecting the front-end to the backend
@crichardson
View Controller
Model
Product Info
service
Recommendation
Service
Review
service
REST
REST
Thrift
Traditional server-side
web application
View Controller
Model
Browser/Native App
Chatty API
Web unfriendly
protocols

45. @crichardson
Use an API gateway
View Controller
Model
Product Info
service
Recommendation
Service
Review
service
REST
REST
Thrift
API
Gateway
View Controller
Model
Browser/Native App
Single entry point
Client
specific APIs
Protocol
translation
Traditional server-side
web application

46. @crichardson
Optimized client-specific
APIs
Web
application
Mobile
App
NodeJS
API
Gateway
REST
proxy
Event
publishing
Product Info
service
Recommendation
Service
Review
service
REST
REST
Thrift
getProductInfo()
getRecomm...()
getReviews()
getProductDetails()

47. @crichardson
Netflix API Gateway
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific
end points

48. @crichardson
API gateway design
challenges
Performance and scalability
Non-blocking I/O
Asynchronous, concurrent code
Handling partial failures
....
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

49. @crichardson
Useful frameworks for
building an API gateway
JVM:
Netty, Vertex
Netflix Hystrix
...
Other:
NodeJS

50. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Using an API gateway
Inter-service communication mechanisms

51. Inter-service communication
options
@crichardson
Synchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Asynchronous is preferred
JSON is fashionable but binary format
is more efficient

52. Pros and cons of messaging
Pros
Cons
Decouples client from
Additional complexity of
server
message broker
Message broker buffers
Request/reply-style
messages
communication is more
Supports a variety of
complex
communication patterns

53. Pros and cons of HTTP
Pros
Simple and familiar
Request/reply is easy
Firewall friendly
No intermediate broker
Cons
Only supports request/
reply
Server must be
available
Client needs to
discover URL(s) of
server(s)

54. @crichardson
Discovery option #1:
Internal load balancer
Load
Balancer
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client/
API gateway
Services register
with load balancer
Client talks to
load balancer
Has a well-known
location
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/USVPC_creating_basic_lb.html

55. @crichardson
Discovery option #2:
client-side load balancing
REST
Client
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client
Service
Registry
Services register
with registry
Client polls
registry
http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html
http://techblog.netflix.com/2012/09/eureka.html

56. @crichardson
Lots of moving parts!
PProrodducutc tIn Ifnofo
Service
Recommendation
Service
Review
Service
Order
Service
Browse Products UI
Checkout UI
Order management
UI
Account
management UI
API
Gate
way
Service registry
Content
Router
HTML
Browser
REST
Client
Ext.
LB
Ext.
LB

57. @crichardson
Summary

58. Monolithic applications are
simple to develop and deploy
@crichardson
BUT have significant
drawbacks

59. @crichardson
Apply the scale cube
Modular, polyglot, and
scalable applications
Services developed,
deployed and scaled
independently

60. Use a modular, polyglot architecture
View Controller Product Info
@crichardson
Model
service
Recommendation
Service
Review
service
REST
REST
AMQP
API
Gateway
Server-side web
application
View Controller
Model
Browser/Native
application

61. @crichardson
Start refactoring your
monolith
Monolith Anti-corruption Service
layer
Glue code
Pristine

62. Come to my 11.30 am talk
to learn more....
Building microservices
with Scala, functional
domain models and
@crichardson
Spring Boot

63. @crichardson chris@chrisrichardson.net
@crichardson
Questions?
http://plainoldobjects.com http://microservices.io