This document presents a research about microservices architectures in the enterprise. The document explores some of the key patterns and technologies relevant to implement microservices solutions in enterprise environments.

1. Microservices Architecture in the
Enterprise: A Research Study and
Reference Architecture
By Tellago, Inc

2. Contents
Overview.......................................................................................................................................................2
What are Microservices? ..............................................................................................................................2
Microservices in the Enterprise: The SOA Battle..........................................................................................3
Capabilities of Enterprise Microservices Architecture..................................................................................4
Service Discovery and Service Registry.....................................................................................................5
Microservices Description ........................................................................................................................7
Lightweight Middleware...........................................................................................................................9
Containerization of Microservices .........................................................................................................12
API Gateway............................................................................................................................................14
Inter-Service Communication .................................................................................................................15
One Microservice per Data Source Strategy...........................................................................................17
One Microservice per Use Case Strategy ...............................................................................................19
Microservices Monitoring.......................................................................................................................21
Other Relevant Capabilities of Enterprise Microservices Architecture ......................................................23
Summary.....................................................................................................................................................23

3. Overview
Microservices is one of the hottest trends in modern enterprise software. Created to
address some of the limitations of traditional service oriented architecture (SOA),
microservices proposes an architecture style of decomposing monolithic applications
into a series of smaller services that can accomplish atomic tasks. Despite its
undoubted popularity, most enterprises struggle with taking even basic steps
towards implementing microservices architecture. These challenges are a result of a
combination of the new architecture models, as well as the fundamental changes in
the software development processes introduced by microservices.
This paper explores some of the key features of enterprise microservices
architecture. The paper reviews a reference architecture based some of the lessons
we have learned implementing dozens of microservices solutions for enterprise
customers. From the technical standpoint, the paper reviews a series of new and
interesting technologies that can enable the foundation blocks of microservices
architecture.
What are Microservices?
Like any other hot technology trend, microservices has been surrounded with more
definitions than what we can cover in this article. However, in the context of
enterprise software solutions, there is a definition we think describes the
capabilities of microservices:
“Microservices are loosely coupled service oriented architecture with bounded
contexts.” (Adrian Cockcroft, Netflix)
The reason this definition of microservices is very relevant in the enterprise is
because it directly ties to SOA which is so common in corporate environments.
By loosely coupled SOAs, we refer to the principle that microservices should be very
autonomous and not highly dependent on other microservices. From that
perspective, microservices should avoid relying on common components or

4. infrastructure building blocks, like databases, that end up introducing direct
dependencies between microservices.
Bounded context is a concept from domain driven design (DDD) theory that
provides a methodology for dividing large architecture models into specific bounded
context and expressing the explicit relationships between them. In the context of
microservices, bounded context should allow a team to evolve a microservice
without having explicit knowledge of other microservices in the organization.
Microservices in the Enterprise: The SOA Battle
Microservices can very well become a fundamental building block of how enterprise
software will be written in the next few years. From a conceptual standpoint,
microservices offers models that enable levels of innovation and agility that have
been impossible with earlier architecture styles. However, the adoption of
microservices in the enterprise is also full of challenges.
One of the biggest challenges of implementing microservices solutions in the
enterprise is that organizations can’t buy a product or platform that provides a
microservices infrastructure. Instead, microservices architecture is typically a
combination of different technologies to power its different capabilities. This
approach contrasts with traditional SOA scenarios in which most requirements could
be addressed using the SOA platforms provided by enterprise software vendors.
Another obstacle encountered by microservices solutions in the enterprise is the
friction introduced by SOA frameworks and development processes. To some
extent, many organizations still struggle trying to find clear differentiators between
the traditional SOA approach and the new microservices model. To help with this
issue, we have listed some of the key differentiators between SOA and
microservices we consider relevant in enterprise solutions:
Capability SOA Microservices

5. Middleware Centralized ESB Lightweight, federated
Bus
Services Complex business services Small, atomic services
focus on individual use
cases
Protocols SOAP, WS-* REST and Lightweight RPC
Deployment Model Centralized Host Isolated container
infrastructure
Governance Centralized governance
based on a service
registry, repository
Federated governance
based on lightweight
discovery protocols
Development process Top-down (architect to
developer)
Bottom-up
Focus Control Speed, agility
While microservices architecture has evolved as an alternative to the limitations of
traditional SOAs, they still don’t enjoy the same level of enterprise tooling and
support as its predecessors. As a result, many organizations are still hesitant to
embrace microservices solutions. However, when implemented following best
practices microservices solutions deliver levels of agility and flexibility largely
exceeding what could be accomplished with SOA solutions.
Having implemented dozens of microservices solutions for Global 2000 companies,
we believe there are a series of capabilities and best practices that are foundational
to implement enterprise-ready microservices architecture. The next section
explores some of the key capabilities of enterprise microservices solutions.
Capabilities of Enterprise Microservices Architecture

6. While there is no well-established architecture reference for microservices solutions,
there are very well defined capabilities that are essential to enable these types of
solutions in the enterprise. Correctly assembling and combining these capabilities
represents a robust foundation to build enterprise-ready microservices solutions.
From the large sets of features attributed to microservices architecture, we believe
the following group is essential in any enterprise microservices solution:
 Service Discovery
 Service Description
 Lightweight Middleware
 API Gateway
 Containerization
 Data Source Partition
 Verb Partition
Service Discovery and Service Registry
Dynamically discovering services and capabilities is an essential element of
enterprise microservices architecture. Traditional SOA solutions typically rely on
centralized service registries for cataloging the services in an enterprise.
Unfortunately, the centralized service registry pattern is completely inapplicable for
environments in which tens or hundreds of microservices are constantly being
deployed across different environments.
In microservices architecture, service discovery requires decentralized, dynamic
and highly scalable models that can be adapted to different runtime topologies. In
this sense, microservices discovery should play a very similar role to DNS in
networking solutions. The following diagram illustrates this concept:

7. In the microservices world, the role of a service registry changes from a centralized
to a completely federated entity. The evolution of microservices architecture has
caused the evolution of different technologies that can enable service discovery at
an enterprise level.
Technologies to Enable Microservices Discovery
 Consul (https://consul.io/): Consul is a peer to peer, strongly consistent
data store that uses a gossip protocol to communicate and form dynamic
clusters. It provides a hierarchical key-value store that you can place data in
and register watches against to be notified when something changes within a
particular key space. Consul implements a full service discovery system in
the library, so you don’t need to implement your own or use a third-party
library. This includes health checks on both nodes and services. It
implements a DNS server interface, allowing you to perform service lookups
using the DNS protocol. It also allows “clients” to run as independent
processes and register/monitor services on their behalf. This removes the
need to add explicit Consul support into your applications.

8.  Eureka (https://github.com/Netflix/eureka): Eureka is a “mid-tier load
balancer” built by Netflix and released as open source. It is designed to allow
services to be able to register with a Eureka server and then locate each
other via that server. Eureka contains a built in load balancer which extends
its discovery capabilities to highly distributed environments. Eureka is
currently only practical in AWS environments.
 Zookeeper (https://zookeeper.apache.org/): ZooKeeper is an Apache
project providing a distributed, eventually consistent hierarchical
configuration store. ZooKeeper originated out of the world of Hadoop, where
it was built to help in the maintenance of the various components in a
Hadoop cluster. It is not a service discovery system per se, but is instead a
distributed configuration store that provides notifications to registered
clients. With this, it is possible to build a service discovery infrastructure;
however, every service must explicitly register with ZooKeeper, and the
clients must then check on the configuration.
 etcd (https://github.com/coreos/etcd): etcd is an HTTP accessible key-value
store. In this, it is similar in concept to ZooKeeper and the K/V portion of
Consul. It functions as a distributed, hierarchical configuration system, and
can be used to build a Service Discovery system. It originally grew out of the
CoreOS project, is maintained by them, and recently achieved a stable major
release. From the functional standpoint, etcd is dependent on HTTP and,
consequently, is only practical is environments in which all services are
leveraging that protocol.
Microservices Description
Understanding the capabilities of microservices is essential to streamline its
adoption in the enterprise. Like many of its distributed programming predecessors,
microservices architecture relies on description languages that illustrate the
capabilities of microservices so that they can be used by client applications. In the
case of microservices, the usage of description languages goes beyond modeling

9. the features of a specific microservice and includes contextual elements, such as
dependencies on other microservices, SLAs, etc.
Description languages have been the subject of passionate debates during the
evolution of movements like SOA and REST. While description languages are
undeniably effective for describing the capabilities of a service endpoint, they can
introduce unnecessary complexities when used as a proxy generation mechanism
as interoperability between different stacks becomes challenging. To avoid these
challenges, microservices architecture should leverage lightweight contracts that
describe the core capabilities of services. The following figure illustrates this
concept:

10. In microservices environments it is very common to have tens or hundreds of
services interacting between each other. In those scenarios, understanding which
features of a microservice are required by other microservices is essential to
guarantee the continuous evolution of a solution. To address that challenge,
microservices architecture can leverage techniques like consumer driven contracts
in which client applications describe which capabilities they require from a specific
microservice.
Technologies to Implement Microservices Description
 Swagger (http://swagger.io/): Swagger defines a standard, language-
agnostic interface to REST APIs which allows both humans and computers to
discover and understand the capabilities of the service without access to
source code, documentation, or through network traffic inspection. Swagger
has experienced a great level of adoption in the API platform space becoming
the standard of choice for describing RESTful endpoints.
 API Blueprint (https://apiblueprint.org/): API Blueprint is a documentation-
oriented API description language that can be considered the most popular
alternative to Swagger. API Blueprint relies on Markdown as the underlying
language which makes it semantically rich compared to alternatives. Even
though the project is mainly maintained by Apiary, API Blueprint enjoys a
very vibrant ecosystem of tools and extensions.
 gRPC IDL (https://thrift.apache.org/): gRPC is a next generation remote
procedure call (RPC) platform developed by Google. As part of its technology
stack, gRPC uses protocol buffers as Interface Definition Language (IDL) for
describing both the service interface and the structure of the payload
messages. It is possible to use other alternatives if desired. The gRPC IDL
provides seamless interoperability between services and clients created in
many programming languages including Java, Python, C#, Node.js, Go, etc.
Lightweight Middleware

11. Microservices are not exempted from the middleware requirements so common in
the enterprise. SOAs produced an entire generation of enterprise service bus (ESB)
platforms designed to broker the interaction between different service endpoints.
From the practical standpoint, ESBs aren’t only adopted to expand the
infrastructure capabilities (routing, persistence, transformation, etc) of SOA
solutions, but they also implement significant levels of logic related to specific SOA
processes. From this perspective, ESBs can be seen as the antithesis of
microservices as they become a centralized business logic broker to many services.
However, many of the middleware capabilities of traditional ESBs are still required
in microservices infrastructures and the trick becomes how to leverage these
capabilities without crossing the line into a monolithic architecture.
As an architecture style, microservices promotes the notion of “smart endpoints and
dump pipes,” or, as we preferred to call it, lightweight middleware. This type of
middleware expands the capabilities of microservices with infrastructure
components such as message persistence, message transformation, message
routing, or publish-subscribe communication, etc., while, at the same time,
maintains the business process logic in the microservices. This model keeps the
middleware infrastructure relatively lightweight and flexible enough to scale to
complex topologies. The following figure illustrates the lightweight middleware
patter in the context of microservices:

12. Technologies to Implement a Lightweight Middleware
From a practical standpoint, there are several technologies that can be used to
implement a lightweight microservices middleware. In our experience, the second
generation of enterprise messaging platforms, such as RabbitMQ, Kafka, or
ZeroMQ, seem to be the preferred options by most of our clients. Let’s explore
some of these options:
 RabbitMQ (https://www.rabbitmq.com/): RabbitMQ is a messaging broker
that serves as an intermediary for messaging. It gives your applications a
common platform to send and receive messages, and your messages a safe
place to live until received. Emerging as the de facto standard for cloud
messaging, RabbitMQ is used for efficient communication between servers,
applications, and devices.
 ZeroMQ (http://zeromq.org/): ZeroMQ is a high-performance asynchronous
messaging library, aimed at use in distributed or concurrent applications. It
provides a message queue, but, unlike message-oriented middleware, a

13. ZeroMQ system can run without a dedicated message broker. The library's
API is designed to resemble that of Berkeley sockets.
 Kafka (http://kafka.apache.org/): Created by LinkedIn, Apache Kafka is a
high performance publish-subscribe messaging platform. Kafka is designed to
allow a single cluster to serve as the central data backbone for a large
organization. It can be elastically and transparently expanded without
downtime.
Containerization of Microservices
The rapid evolution of container platforms like Docker have been one of the main
catalysts to the evolution of microservices solutions. Being able to package
microservices in portable containers simplifies the management and
operationalization requirements of microservices infrastructures. From the
functional standpoint, containers enable isolation of the infrastructure of different
microservices allowing these to evolve relatively independently. The following
diagram illustrates this concept:

14. The infrastructure isolation provided by containers also enables other essential
capabilities of microservices architecture such as continuous deployment and
seamless scalability. From the deployment perspective, containers allow individual
microservices to be deployed and tested without affecting other microservices.
Additionally, the isolated infrastructure provided by containers allows microservices
to be scaled independently.
Technologies for Containerizing Microservices
 Docker (https://www.docker.com ): Docker is the lead platform in the
market providing a model that enables the deployment and management of
applications inside containers. Docker enjoys a very robust ecosystem of
tools and solutions and is supported by a large number of enterprise
platforms.

15.  rkt (https://github.com/coreos/rkt): rkt is an alternative to the Docker
runtime, designed for server environments with the most rigorous security
and production requirements. rkt is oriented around the App Container
specification, a new set of simple and open specifications for a portable
container format.
 Kubernetes (http://kubernetes.io/): Created by Google, Kubernetes is an
open-source system for managing containerized applications across multiple
hosts in a cluster. Kubernetes is intended to make deploying
containerized/microservice-based applications easy but powerful.
API Gateway
The small granularity of microservices introduces a high degree of latency and
chattiness for client applications. For instance, a simple web or mobile application
could end up calling dozens of microservices to access the information required by a
single screen. As microservices evolve and new versions are created, the large
number of dependencies becomes incredibly fragile and challenging to maintain.
An API gateway is a server-side technology pattern that aggregates multiple
microservices and coarse level services tailored to specific client applications. From
this perspective, an API gateway can expose abstract services which reveal the
capabilities required by consumer applications. In addition to the aggregation of
microservices, the API gateway extends microservices with infrastructure
capabilities, such as data caching or instrumentation, so they can be consistently
applied across all endpoints. We can see the API Gateway pattern in the following
diagram:

16. Technologies to Implement a Microservices API Gateway
 Mashape’s Kong (https://getkong.org/): Kong is an open source
management platform for APIs and microservices. Kong is the engine behind
Mashape’s popular API Gateway.
 Apigee (http://apigee.com): Apigee can be considered the market leader in
API management. The current version of the Apigee Edge platform provides
many enterprise-ready capabilities relevant in microservices scenarios.
 Azure API Gateway (https://azure.microsoft.com/en-us/services/api-
management/): The Azure API Gateway is a native service in the Azure
platform to enable the publishing, security, and management of APIs and
microservices both on-premise and in the cloud.
 AWS API Gateway (https://aws.amazon.com/api-gateway/): AWS API
Gateway is a native service for API management on the AWS platform. The
service is tightly integrated with other services on the AWS platform making
it a great solution for managing microservices powered by AWS.
Inter-Service Communication
Microservices architecture exponentially increases the levels of inter-process
communication (IPC) required to accomplish specific business tasks. While REST-

17. based models are the most popular mechanisms for enabling microservices
communication, they are often impractical overall for internal IPC communications.
As a result, microservices architecture leverages next generation remote procedure
call (RPC) models to enable internal IPC communications. The following diagram
illustrates an example of IPC microservices communication.
While there are very well known challenges with traditional RPC architecture,
modern RPC models have made tremendous progress to avoid the pitfalls. The
complexity of IPC communication in microservices architecture expands beyond the
protocol of choice. IPC patterns such as synchronous, asynchronous, streaming,
etc., are common in large scale enterprise microservices solutions and should be
served under the same architecture.

18. Technologies to Enable IPC Communications
 gRPC (http://grpc.io ): Created by Google, gRPC is a high performance RPC
platform. In gRPC a client application can directly call methods on
a server application on a different machine as if it were a local object,
making it easier for you to create distributed applications and services. As in
many RPC systems, gRPC is based around the idea of defining a service,
specifying the methods that can be called remotely with their parameters and
return types. On the server side, the server implements this interface and
runs a gRPC server to handle client calls.
 Akka (http://akka.io): Akka is an open-source toolkit and runtime
simplifying the construction of concurrent and distributed applications on the
JVM. Akka supports multiple programming models for concurrency, but it
emphasizes actor-based concurrency, with inspiration drawn from Erlang.
 Apache Thrift (https://thrift.apache.org/): Thrift is an interface definition
language and binary communication protocol used to define and create
services for numerous languages. It is used as a remote procedure call (RPC)
framework and was developed at Facebook for "scalable cross-language
services development.” It combines a software stack with a code generation
engine to build services that work efficiently across a large number of
languages.
One Microservice per Data Source Strategy
Data access is one of the fundamental capabilities of microservices in the
enterprise. From this perspective, determining the right granularity of data access
capabilities of a microservice is a permanent debate when designing this type of
architecture. Typically, data access operations in a microservice are linked to a
specific data source with some relevant business context, such as contacts or
accounts. Using the data source entity as the partitioning unit for data access
microservices is a recommended practice in enterprise architecture. The following
diagram illustrates this strategy:

19. Architecting data access microservices around isolated data sources guarantees a
level of independence between the different services in enterprise solutions. This
model allows microservices to evolve independently around contextually bounded
concepts such as business data sources. Microservice purists will argue that each
microservice should use an independent database. While this principle is impractical
in many enterprise architectures, partitioning at the data source level offers a
simple and practical model to enable large scale data access in microservices
architecture.

20. Technologies to Enable the One Microservice per Data Source Strategy
 GraphQL (http://graphql.org/): Designed by Facebook, GraphQL is a query
language designed to build client applications by providing an intuitive and
flexible syntax and system for describing their data requirements and
interactions.
 OData (http://www.odata.org/): Open Data Protocol (OData) is an open
protocol which allows the creation and consumption of queryable and
interoperable RESTful APIs in a simple and standard way. OData is widely
implemented by many enterprise software platforms and is the default data
access model used by many Microsoft products.
 Falcor (http://netflix.github.io/): Created by Netflix, Falcor is a framework
for data fetching JSON resources. Falcor allows you to model all your
backend data as a single Virtual JSON object on your Node server. On the
client you work with your remote JSON object using familiar JavaScript
operations like get, set, and call. If you know your data, you know your API
for any JSON resource.
One Microservice per Use Case Strategy
In addition to data access, microservices architecture needs to enable functional
capabilities in the form of service endpoints. By functional we refer to microservices
that execute business logic as part of its operations. Typically, these types of
microservices are associated with functional use cases relevant to specific business
scenarios. From this perspective, partitioning per specific use case provides a very
pragmatic model to associate microservices with atomic business capabilities. This
concept is shown in the following figure:

21. Building microservices around individual use cases is an intuitive methodology for
organizing a microservices topology. Additionally, having microservices that
abstract atomic business use cases facilitates the composition of microservices to
enable more sophisticated business scenarios. Other capabilities like functional and
integration testing clearly benefit from this partitioning strategy.
Technologies to Enable the One Microservice per Use Case Strategy
 gRPC (http://grpc.io): Created by Google, gRPC is a high performance RPC
platform. In gRPC a client application can directly call methods on
a server application on a different machine as if it were a local object,
making it easier for you to create distributed applications and services. As in
many RPC systems, gRPC is based around the idea of defining a service,
specifying the methods that can be called remotely with their parameters and
return types. On the server side, the server implements this interface and
runs a gRPC server to handle client calls.

22.  Akka (http://akka.io): Akka is an open-source toolkit and runtime
simplifying the construction of concurrent and distributed applications on the
JVM. Akka supports multiple programming models for concurrency, but it
emphasizes actor-based concurrency, with inspiration drawn from Erlang.
 Apache Thrift (https://thrift.apache.org/): Thrift is an interface definition
language and binary communication protocol used to define and create
services for numerous languages. It is used as a remote procedure call (RPC)
framework and was developed at Facebook for "scalable cross-language
services development.” It combines a software stack with a code generation
engine to build services that work efficiently across a large number of
languages.
Microservices Monitoring
Instrumenting and monitoring microservices is an essential capability to
operationalize these solutions in the enterprise. While application performance
monitoring (APM) platforms provide the required capabilities to instrument
individual microservices, they missed some important elements needed to monitor
complete microservices architecture.
The complexity of microservices topologies can become an interesting challenge
from the observability and instrumentation perspective. Efficiently monitoring
microservices architecture requires providing some sophisticated capabilities, such
as tracing requests across microservices, and mapping requests to microservices
topologies, among others. Like other enterprise monitoring platforms, microservices
instrumentation requires a centralized endpoint that collects the runtime
information from the different microservices endpoints. The following figure
illustrates this concept:

23. Technologies to Enable Microservices Monitoring
 Netflix’s Atlas (https://github.com/Netflix/atlas/wiki): Atlas was
developed by Netflix to manage dimensional time series data for near real-
time operational insight. Atlas features in-memory data storage, allowing it
to gather and report very large numbers of metrics, very quickly.
 Netflix’s Hystrix (https://github.com/Netflix/Hystrix): Hystrix is a latency
and fault tolerance library designed to isolate points of access to remote
systems, services, and third-party libraries, stop cascading failure, and
enable resilience in complex distributed systems where failure is inevitable.
 Twitter’s Zipkin (https://twitter.github.io/zipkin): Created by Twitter,
Zipkin is a distributed tracing system. It helps gather timing data needed to
troubleshoot latency problems in microservice architecture. The front end is
a "waterfall" style graph of service calls showing call durations as horizontal
bars.
 Trace (http://trace.risingstack.com/): Trace is a microservice monitoring
and debugging tool that empowers you to get all the metrics you need
when operating microservices. Trace comes as both a free, open source
tool and as a hosted service.

24. Other Relevant Capabilities of Enterprise Microservices Architecture
Enterprise microservices architecture is relatively complex compared to some of its
predecessors in the distributed systems space. In this sense, there are many other
key capabilities that could be considered relevant to enable microservices at an
enterprise scale. In addition to the capabilities listed in the previous sections,
enterprises should consider some of the following features:
 Design for Failure: Assume that microservices solutions are going to fail all
the time and architect for an extreme level of resiliency.
 Failure Isolation: Prevent individual failures from cascading through a
microservices solution.
 Microservices Testing: Provide models to enable the functional and
integration test of microservices.
 Configuration Management: Enable highly available models to manage
configuration for microservices.
 Security: Data privacy, authentication, and access control are always
relevant capabilities of microservices.
Summary
Microservices is becoming one of the most important trends in modern enterprise
software solutions. Conceptually, microservices extend SOA with more efficient
architecture models. Implementing microservices require a cultural change in the
way that most enterprises approach software development.
This paper covered some of the best practices, technologies and patters of
enterprise microservices architectures. The paper in the fundamental building
blocks that should be considered by organizations to efficiently leverage
microservices models. The patterns and technology recommendations included in
this paper are based on our experience implementing this type of solutions for our
customers and not on any commercial affiliation with the vendors.