The presentation supported the speech "Think differently – Stream-based Microservice Architecture for Next-Generation Applications" by Fabian Wilckens (EMEA Solutions Architect, MapR Technologies Inc.) at the HUG Italy meet-up supported by Engineering Group's SpagoBI Labs, which took place in Milan, Italy on March 17th, 2016. Read more: http://bit.ly/1UydNuz

1. © 2016 MapR Technologies 1© 2016 MapR Technologies 1MapR Confidential
© 2016 MapR Technologies
Fabian Wilckens
EMEA Solutions Architect
THINK DIFFERENTLY
Stream-based Microservice Architecture for
Next-Generation Applications

2. © 2016 MapR Technologies 2© 2016 MapR Technologies 2MapR Confidential

3. © 2016 MapR Technologies 3© 2016 MapR Technologies 3MapR Confidential

4. © 2016 MapR Technologies 4© 2016 MapR Technologies 4MapR Confidential

5. © 2016 MapR Technologies 5© 2016 MapR Technologies 5MapR Confidential
The microservice architectural style is an approach to
developing a single application as a suite of small services,
each running in its own process and communicating with
lightweight mechanisms, often an HTTP resource API. These
services are built around business capabilities and
independently deployable by fully automated deployment
machinery. There is a bare minimum of centralized
management of these services, which may be written in
different programming languages and use different data
storage technologies.
- Martin Fowler
MICROSERVICE ARCHITECTURAL STYLE
SINGLE APPLICATION
SUITE OF SMALL SERVICES
COMMUNICATING WITH LIGHTWEIGHT MECHANISMS
BUILT AROUND BUSINESS
CAPABILITIESFULLY AUTOMATED
DEPLOYMENTBARE MINIMUM OF CENTRALIZED MANAGEMENT
WRITTEN IN DIFFERENT PROGRAMMING LANGUAGES
DIFFERENT DATA STORAGE
TECHNOLOGIES

6. © 2016 MapR Technologies 6© 2016 MapR Technologies 6MapR Confidential
WAIT A SECOND
SOA?SERVICE ORIENTED ARCHITECTURE
IS THERE A DIFFERENCE?

7. © 2016 MapR Technologies 7© 2016 MapR Technologies 7MapR Confidential
Integrates different
applications as a set of
services
Architect a single application as
a set of services
APP 1 APP 2
APP 4 APP 5
ENTERPRISE SERVICE BUS
APP 2
LOGISTICS
SERVICE
CUSTOMER
SERVICE
BILLING
SERVICE
USER INTERFACE SERVICE
MICROSERVICE-STYLE
CLASSIC SOA-STYLE

8. © 2016 MapR Technologies 8© 2016 MapR Technologies 8MapR Confidential
A few years back …

9. © 2016 MapR Technologies 9© 2016 MapR Technologies 9MapR Confidential
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
Easy to develop? Easy to deploy? Easy to scale?

10. © 2016 MapR Technologies 10© 2016 MapR Technologies 10MapR Confidential
CHALLENGES AHEAD

11. © 2016 MapR Technologies 11© 2016 MapR Technologies 11MapR Confidential
LARGE CODE & BIG DATA
HARD TO UNDERSTAND & CHANGE
OVERLOADED IDE
OVERLOADED WEB CONTAINERS
DEVELOPMENT SLOWS DOWN
EVEN SMALL APPLICATION CHANGES REQUIRE FULL
REBUILD, TEST AND DEPLOYMENT
IMPACT ANALYSIS IS A HUGE EFFORT AND TAKES TIME
DIFFICULT TO FREQUENTLY CHANGE & DEPLOY

12. © 2016 MapR Technologies 12© 2016 MapR Technologies 12MapR Confidential
OTHER CHALLENGES
LONG-TERM COMMITMENT TO TECHNOLOGY STACK
RE-WRITE = COMPLETE RE-WRITE
MODULE DEPENDENCIES FORCE RE-WRITE
FAILURE IN MONOLITHIC APPLICATIONS BRINGS IT DOWN

13. © 2016 MapR Technologies 13© 2016 MapR Technologies 13MapR Confidential
HORIZONTAL SCALING
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface

14. © 2016 MapR Technologies 14© 2016 MapR Technologies 14MapR Confidential
VERTICAL SCALING
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface

15. © 2016 MapR Technologies 15© 2016 MapR Technologies 15MapR Confidential
DATA SCALING
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface
App Server
WAR/EAR
Backend
Logistics
Ordering
User Interface

16. © 2016 MapR Technologies 16© 2016 MapR Technologies 16MapR Confidential

17. © 2016 MapR Technologies 17© 2016 MapR Technologies 17MapR Confidential
MICROSERVICES SCALING
Container
Ordering
Container
Ordering
Container
Ordering
Ordering
Database
Container
User Interface
Container
Logistics
Logistics
Database
Container
Backend
Backend
Database

18. © 2016 MapR Technologies 18© 2016 MapR Technologies 18MapR Confidential
MICROSERVICES TEAM SCALING
Container
Ordering
Container
Logistics
Container
Backend

19. © 2016 MapR Technologies 19© 2016 MapR Technologies 19MapR Confidential
LET’S TALK BENEFITS (PROS)
ONLY MAKE APIS TALK TO EACH OTHER
ADD MODULES BASED UPON BUSINESS REQUIREMENTS
CHANGE MODULES INDEPENDENTLY
DIFFERENT TEAMS WORK ON THEIR OWN PROBLEMS
FASTER TIME-TO-MARKET FOR NEW SERVICES
CAN STILL INTEGRATE WITH SOA
SERVICES/TEAMS SCALE INDEPENDENTLY
…

20. © 2016 MapR Technologies 20© 2016 MapR Technologies 20MapR Confidential

21. © 2016 MapR Technologies 21© 2016 MapR Technologies 21MapR Confidential

22. © 2016 MapR Technologies 22© 2016 MapR Technologies 22MapR Confidential
The Challenge with Data Pipelines
Filtering &
Aggregation
Alerting Processing

23. © 2016 MapR Technologies 23© 2016 MapR Technologies 23MapR Confidential
Streams Simplify Data Movement
Filtering &
Aggregation
Alerting Processing
Streams
Reliable publish/subscribe
transport between sources
and destinations.

24. © 2016 MapR Technologies 24© 2016 MapR Technologies 24MapR Confidential
Legacy Systems: Message Queues
IBM MQ, TIBCO, RabbitMQ
OrdersFront End
Order Processing
Order Processing
Usage/Requirements
●Tight, transactional
conversations between systems
●1:1 or Few:Few
●Low data rates
●Mission-critical delivery
Approach
●Queue-oriented design
●Each message replicated to N output
queues
●Messages popped when read
●Scale-up, master/slave
Doesn’t Do
●High message rates (>100K/s)
●Slow consumers
●Queue replay/rewind

25. © 2016 MapR Technologies 25© 2016 MapR Technologies 25MapR Confidential
Evolving “big data” Event Streams: Distributed Logs
Kafka, Hydra, DistributedLog
Usage/Requirements
●High throughput data transferred
from
decoupled systems
●Many->1
●1->Many
●Different speeds
Approach
●Log-oriented design
●Write messages to log files
●Consumers pull messages at
their own pace
●Scale-out
Doesn’t Do
●Global applications
●Message persistence
●Integrated analytics
(data movement required)
DB_Changes
Stream Processing
Search/
EDW
DB

26. © 2016 MapR Technologies 26© 2016 MapR Technologies 26MapR Confidential
Introducing MapR Streams
Global Publish/Subscribe Event Streaming
Producers
Publish Billions of
messages/sec to a topic
Consumers
Reliable delivery to all
consumers. Immediately
Global
Tie together geo-dispersed
clusters. Worldwide

27. © 2016 MapR Technologies 27© 2016 MapR Technologies 27MapR Confidential
MapR Converged Data Platform
Tenant 2 Tenant 3
Tenant 1

28. © 2016 MapR Technologies 28© 2016 MapR Technologies 28MapR Confidential
Container
Ordering
Container
User Interface
Container
Logistics
Container
Backend

29. © 2016 MapR Technologies 29© 2016 MapR Technologies 29MapR Confidential
Global Applications
Container
Ordering
Container
Backend
Container
Logistics
Container
User Interface
Container
User Interface
Container
User Interface

30. © 2016 MapR Technologies 30© 2016 MapR Technologies 30MapR Confidential
Are Microservices here to stay?
We don’t know …
Monolithic Apps are dead

31. © 2016 MapR Technologies 31© 2016 MapR Technologies 31MapR Confidential
Content & Pictures
• http://enterprise.vnews.com/wp-content/uploads/2015/07/EP-SeanSmithCover-ns-vn-0728151.jpeg
• http://www.rodmatthews.com.au/_assets/images/blog/Positional%20Images/Complexity%20Small%20
Version.jpg
• http://www.coliquio-insights.de/wp-content/uploads/2015/02/5141328136_16d8f73ee7_b.jpg