Serverless Architectures - Where have all the servers gone?

Serverless
Architectures
Where have all the servers gone?
Nane Kratzke

TL;DR
• Serverless architectures refer to
applications that depend substantially on
3rd party services (Backend as a Service,
BaaS)
• or on custom code that is run in
ephemeral containers (Function as a
Service, FaaS).
• By moving much behavior to the front
end, such architectures reduce the need
for ‚always on‘ servers.
• Therefore, such systems can reduce
operational cost and shift operational
complexity to BaaS service providers
• at cost of vendor dependencies and (still)
immaturity of supporting services and
tools.
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
2

Outline
o What is Serverless?
o How is Serverless changing
architectures?
o Benefits and drawbacks
o (In-)appropriate use cases
o Open issues
o Serverless frameworks
3

4
What is Serverless?

What is Serverless?
5
The term Serverless application is used to describe
(rich client like) applications that significantly depend
on 3rd party (cloud) services.
• These (3rd party) services are sometimes called
Backend as a Service (BaaS).
• A good example is the Single Sign On Service
(Auth0, https://auth0.com)
However, BaaS service logic can be implemented
serverless as well.
• This logic is run in stateless deployment units
(often containers) that are event-triggered,
ephemeral and fully managed by a 3rd party.
• This logic is called Functions as a Service (FaaS).
• The most popular service provider for FaaS is
currently and probably AWS Lambda.
BaaS is about using
services in Serverless
architectures.
FaaS is about realizing
services in a Serverlessway.

Bare Metal Server
The Serverless Evolution
Where have all the servers gone?
6
VM
Bare Metal Server Bare Metal Server
A
VM
A B
B
Bare Metal Server
VM VM
Container
Engine
A B
Bare Metal Server
VM VM
Container
Engine
FaaS Runtime
A
B
...
...
A
...
Virtualization
Containerization
Time-
Sharing
Dedicated Server
„In recent times“ applications have been
deployed to dedicated servers. In consequence, the
servers were often overdimensioned and had very
inefficient utilization rates. The question arised
how to increase application density without
touching the application design itself.
Machine virtualization is mainly used to
consolidate and isolate applications that
otherwise would have been deployed to
dedicated servers. This increases the
application density on bare metal servers but
the virtual machine images (deployment
unit) are very large. However, the application
can stay untouched. If this model is provided
as a cloud service it is called Infrastructure as
a Service (IaaS).
To pragmatically operate more than one
application per virtual machine, containerization
established as a trend (Docker). A container starts
faster than a virtual machine and shares the
operating system with other containers, thus
reducing deployment unit sizes and increasing
application density per virtual machine. But the
application should follow cloud architectural
styles (like 12-factor-app, microservices) to fully
leverage the opportunities.
But a container still requests a share of CPU, memory,
and storage – even if the provided service is hardly
requested. It would be more ressource efficient, if
services would consume ressources only if there are
incoming requests. This is what FaaS runtime
environments provide. So, services can timeshare a
host, thus further increasing application density per
host. However, this involves to follow a more rich client
architecture model for the user-interface and a service-
composed-of-functions approach for services.
1
2 3 4
Serverless

Is Serverless PaaS?
Are Serverless applications just
another form of Platform as a Service
(PaaS) like Heroku?
• The key operational difference between
FaaS and PaaS is scaling.
• With most PaaS you still need to think
about scale on a level of execution
instances (dynos, machines, containers,
etc.).
• FaaS scaling is fine grained and
completely transparent. It works on a
level of individual service requests.
7
Adrian
Cockcroft
VP Cloud Architecture
Strategy at AWS
„If your PaaS can
efficiently start instances
in 20ms that run for half
a second, then call it
Serverless.“

OK, what is about containers?
8
The argument made for PaaS
holds with containers.
• FaaS scaling is fine grained and
completely transparent. It works
on a level of individual service
requests.
• Most container platforms do not
offer such solutions (although
Kubernetes is tending towards this
level providing concepts like
Horizontal Pod Autoscaling).
• FaaS vs. Containers? It is more a
question about when to use what?
Mike Roberts
Serverless Expert, Founder, and
Co-Author of What is
Serverless?, O´Reilly
„[…] FaaS is seen as a better choice
for event-driven style […]
and containers are seen as better
choice for synchronous-request
driven components […]“

Function as a Service (FaaS)
• FaaS is about running backend code without
managing own servers.
• FaaS offerings do not require coding to a specific
framework or library. FaaS functions can be
mostly implemented as „first class“ programs in
JavaScript, Python, any JVM language (Java,
Clojure, Scala, …), and more languages.
• Deployment is different compared with traditional
systems - just upload the code to a FaaS provider
and it does anything else.
• Horizontal scaling is completely automatic, elastic
and managed by the provider.
• Functions in FaaS are triggered by event types
defined by the provider, this might be file
updates, scheduled events (time), messages on a
message bus, or simply HTTP requests.
9

Function as a Service
We have to consider STATE and DURATION
FaaS functions are stateless
• They provide pure functional
transformations of their input
• or they have to use of a database,
cross-application cache (e.g.
Redis), or object storage (e.g. S3)
to store state across requests.
10
FaaS functions have timeouts
• E.g. AWS Lambda allows no functions
to execute longer than 5 minutes.
• So, certain long lived tasks are not
suited to FaaS functions.

Function as a Service
We have to consider STARTUP LATENCY
FaaS function respond times depend on a number of
factors
• and maybe anywhere between milliseconds and minutes
(in very worst cases).
• You should consider the overhead of starting a potential
necessary runtime environment for your function code.
• E.g.: JavaScript and Python are known to spun up faster
than a JVM.
• Latencies can get longer, especially if
• a function processes events infrequently (e.g. more than
10 minutes between invocations)
• or sudden spikes in traffic occur (normally 10 requests per
second, but suddenly 1000 reqs/sec).
• It is likely that in these cases the FaaS provider must start
additional (or the first) container instances which involves
longer startup times.
11
So, a low-latency trading application or
a missile control system might be no
appropriate use case for Serverless!!!

How is Serverless Changing Architectures?
12

Serverless by Example
Let us start with a traditional non-serverless architecture
13
Client (Browser) Application Server Relational Database
• Think about a traditional 3-
tier client-oriented system
with server-side logic.
• A good example is a typical
ecommerce app.
• Using such an architecture
the client can be relatively
unintelligent.
• Most of the logic will be
based in the application
server.
So, how will Serverless
change this architecture?

Serverless by Example
Now, let us make it a Serverless architecture
14
Purchase
Function
Search
FunctionAPI Gateway
Authentication Service
Purchase Database
Product Database
Native
mobile
app
1
3
2
4
5
The authentication logic can be
replaced with a 3rd party
authentication BaaS (like Auth0).
The client is allowed direct
access to a subset of our
database. The database is fully
3rd party hosted.
Server application
logic now moves
to the client
application, making
it often a native
mobile app or a
single-page web
application.
Some functionality might be kept in the
„server“. It might be compute intensive
or requires access to a significant amount
of data like a search function.
Such functionality is provided as FaaS
functions that often respond to HTTP
requests.
Some functionality might be kept
in the „server“ for security
reasons or for interfacing
further 3rd party BaaS.
6
An API Gateway is basically a web server that
receives HTTP requests and routes them to
subsequent FaaS functions or other backend services.
So, service complexity is reduced at the costof a more complex service-of-servicearchitecture. Complexity is not reducable, itcan be shifted, capsuled, managed, ... – butnever reduced! If you reduce complexity, youdefine a different – simpler – problem!

15
Benefits and Drawbacks

Benefits
• Reduced cost due to intensive timesharing of
infrastructure and implicit sharing of the operational
staff
• Reduced development costs due to intensive use of
BaaS (services that already exist and not have to be
implemented again and again)
• Easier operational management because the scaling is
automatic
• Reduced packaging and deployment efforts
• Better time to market
• Opportunities for experiments
Maybe even „greener computing“? OK, no one proofed that,
so far … (but there are some reasonable arguments for it)
16

Repetition of client logic
• Serverless makes it easier to reuse
logic on the service side.
• But this involves very often similar
and repetitive implementations on
the client side.
Inherent Drawbacks
Vendor control
• Vendor lock-in
• Loss of server optimizations
17
Security concerns
• Multitenancy vulnerabilities
• Increased surface
• Losing the protective barrier of a
server-side application
No in-server state for FaaS
• No in-memory or in-process cacheing
• External cacheing solutions like Redis
or Memcached are compensating
options but a order of magnitude
slower.

Implementation Drawbacks
18
Do not Denial-of-Service
yourself
Typically your number of
concurrent executions of
functions is limited.
This limit is applied to your
account which you may use for
production and testing.
So testing may surprisingly affect
your production deployments.
Consider execution
durations and latencies
Remember, the executation
duration of functions is limited
and long lived tasks are not
suited for FaaS.
Startup latencies affect respond
times of FaaS functions.
Especially JVM-implemented
functions tend to show suddenly
high latencies, especially if
events occur infrequently or as
sudden spikes.
Do not underestimate
integration testing
Unit testing is fairly simple. But
integration testing of Serverless
can be complicated.
That is because the units of
integration (functions) are a lot
smaller and therefore Serverless
architectures normally rely on
integration testing a lot more
than other architectural styles.

19
(In-)appropriate
Use Cases

It is always about the use case
Stupid!
• As we have seen, Serverless architectures
come with benefits and drawbacks.
• So, some use cases are more suited for
Serverless approaches than others.
• The question is, which use cases look
promising and which use cases do not?
20
Mike Roberts
„There is a lot to like about Serverless
architectures […], but they come with
significant trade-offs. Some of these are
inherent […] and can‘t be fixed. Others […] we
could expect to be resolved.“

Remember the implementation drawbacks
• Multitenancy performance A high load user or
even your testing can cause other users to slow down.
• Repetition of logic across client platforms This
might increase development efforts for the client side.
• Stateless functions Therefore FaaS seems
suboptimal to realize stateful (database-like) services.
• No in-process state and cacheing This might
cause higher latencies.
• Limitations of execution durations This might
prevent long running tasks like streaming or analysis.
• Startup latency This might result in long respond
times especially for very infrequent requested
services.
21

And the ugly … ?happens whenever youtry to apply Serverlessto bad use cases.
The good, the bad, and the ugly
By use case
Good use cases
• User group with an uniformly
distributed request volume.
• Applications with a limited set of
client devices.
• User prefers native apps.
• Stateless services.
• Varying respond times can be
tolerated.
• Short running requests (no batch-
like jobs)
• No extreme low-latency or even
hard real-time requirements
22
Bad use cases
• User group with spikes in request
volumes.
• Necessity to support an unlimited
set of client devices.
• User prefers web access via
browser.
• Stateful services.
• Respond times are low-latency and
must be assured in a defined range.
• Long running batch-like jobs.
• Low-latency or hard real-time
requirements known from low-
latency trading or real-time control
systems.

And the ugly example?
Well, try to implement
a FaaS video streaming
function. The result will
be ugly!
The good, the bad, and the ugly
By example
Good examples
• Single image processing
• Videosharing
• Social network sharing features
• Single entity categorization using
a trained neural network
• Event-based processing of social
media streams
• (short running) database querying
• Messaging
23
Bad examples
• Batch-like image processing
• Videoprocessing/-streaming
• Large scale social network analysis
• Neural network training
• Batch-like entity categorization
• Continuous observing of social
media streams
• Database-like storage service
• Persistant message bus

24
Open Issues

Serverless is (still) immature
Sad, but this leaves room for improvements
• As we have seen, Serverless
architectures have some open issues.
• That is not perfect and should be
considered.
• However, the following points are
likely to be tackled in the future.
25
Mike Roberts
„The remaining drawbacks […] are down
purely to the current state of the art. With
inclination […] and a heroic community these
can be all wiped out.“

Open issues
Improvements for tooling, in-process-state, platform features
26
Tooling
There is the need for more
mature and more
integrated deployment,
application bundling,
configuration, monitoring
& logging, and debugging
tools.
It is likely that these tools
will be better integrated
with future FaaS runtime
environments.
State
To avoid in-process-state is
astonishing hard to accept
for a lot of developpers.
This may even a show-
stopper for several use
cases.
Better integrations with
out-of-process data
solutions like Redis or
memcached would be
helpful.
Platforms
Current FaaS platforms
come with limitations
regarding execution
duration of functions,
startup latency, and non-
separation of execution
limits.
It is likely that these
limitations will be
attentuated in the future.

Open issues
Finding services and proven solution patterns
27
Service Discovery
There are no well defined or
standardized solutions for service
discovery of FaaS functions. This is
even getting worse due to the fine
granular nature of FaaS and a lack
of application and versioning
definition.
Serverless Patterns
Serverless is about to avoid
‚always-on‘ components. But
‚always-on‘ will ever be necessary.
So, Serverless is typically applied
as part of an hybrid architecture.
How to do that is an open issue,
as well as proven patterns for
common use cases like media
processing.

Open issues
Operation still needs monitoring, testing, and debugging
28
Monitoring and debugging
Serverless architectures rely on
the monitoring and debugging
side with whatever the vendor
provides. This support is often
very basic. Open and standardized
APIs would be helpful to integrate
more sophisticated 3rd party
services.
Integration testing
Serverless involves to work with
smaller units (functions) that are
easier to unit test. However, this
involves more complex
integration. So, pragmatic and
efficient integration testing is
especially essential for
microservice and Serverless
approaches.

Serverless
Services, Platforms, and Frameworks
30
Public FaaS Cloud
Services
Most public cloud service
providers offer Serverless
compute services, also known
as function as a service (FaaS).
Currently there exist no FaaS
standard and Vendor Lock-In is
likely.
Standalone Serverless
Platforms
Due to missing standards
public FaaS cloud services are
prone to create vendor lock-in.
Open Serverless platforms
might be an alternative. But
these platforms need
infrastructures for operation.
Platform Agnostic
Serverless Frameworks
Platform agnostic frameworks
provide a provider and
platform agnostic way to
define and deploy Serverless
code on various Serverless
platforms or FaaS cloud
services.
• AWS Lambda
https://aws.amazon.com/lambda
• Google Cloud Functions
https://cloud.google.com/functions
• Azure Functions
https://azure.microsoft.com/services
/functions
• OpenWhisk
https://openwhisk.apache.org
• Nuclio
https://nuclio.io
• Fn project
https://fnproject.io
• OpenFaaS
https://www.openfaas.com
• Serverless Framework
https://serverless.com
• Squeezer Framework
https://squeezer.io/framework
• SpringCloud Functions
https://cloud.spring.io/spring-
cloud-function
This list is not complete! You will find curatedand continuously updated lists here:
• http://bit.ly/2rBWzXr
• http://bit.ly/2FcKSbF

Further Reading
• Serverless Architectures by Mike Roberts, 2016
https://www.martinfowler.com/articles/serverless.html; This blog
post was one of the most influencing references for this presentation.
• What Is Serverless? by John Chapin, Mike Roberts, O‘Reilly, 2017;
You can get this ebook for free from O‘Reilly. The authors take you
through the Serverless landscape: design considerations, tooling, and
approaches to operational management.
• Why the Future Is Serverless? by Ken Fromm, Badri Janakiraman,
2012 http://readwrite.com/2012/10/15/why-the-future-of-software-
and-apps-is-serverless; Might be one of the first articles using the
term Serverless in its current meaning.
• The State of The Serverless Ecosystem by Tal KimHi, 2017,
https://medium.com/@talkimhi/the-state-of-the-serverless-ecosystem-
c8a8b5ca56ae; This post tries to gather every company, product, tool, or
framework that has something to do with the Serverless paradigm. To
reach this goal seems impossible, but it is simply an awesome work.
• Serverless Computing by Wikipedia,
https://en.wikipedia.org/wiki/Serverless_computing, A Wikipedia
link is a must. But remember: Wikipedia is a good start for a search, it
should never be the end.
31

Acknowledgement
Picture Reference
• Cloud: Pixabay (CC0 Public Domain)
• Defintion: Pixabay (CC0 Public Domain)
• Serverrack: Pixabay (CC0 Public Domain)
• Smileys: Pixabay (CC0 Public Domain)
• Peanuts: Pixabay (CC0 Public Domain)
• Question marks: Pixabay (CC0 Public Domain)
• Lego bricks: Pixabay (CC0 Public Domain)
• All icons: The Noun Project (CC-BY-2.0 License)
32
This contribution resulted as a side-effect from research that is
funded by German Federal Ministry of Education and Research
(Project Cloud TRANSIT, 13FH021PX4).
Presentation URL

About
33
Nane Kratzke
CoSA: http://cosa.fh-luebeck.de/en/contact/people/n-kratzke
Blog: http://www.nkode.io
Twitter: @NaneKratzke
GooglePlus: +NaneKratzke
LinkedIn: https://de.linkedin.com/in/nanekratzke
GitHub: https://github.com/nkratzke
ResearchGate: https://www.researchgate.net/profile/Nane_Kratzke
SlideShare: http://de.slideshare.net/i21aneka

Serverless Architectures - Where have all the servers gone?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Serverless Architectures - Where have all the servers gone?

Similar to Serverless Architectures - Where have all the servers gone? (20)

More from Nane Kratzke

More from Nane Kratzke (20)

Recently uploaded

Recently uploaded (20)

Serverless Architectures - Where have all the servers gone?