What is serverless? What is different? What is benefits? What is drawbacks? Dive into architecture design for serverless.

1. Serverless
Young Yang

2. About Me
• Director, EHS
• Principle Engineer, Yahoo
• Sr. Engineering Manager, Yahoo
• Solution Architecture, Yahoo
• Yahoo Frontpage
• Yahoo EC Shopping – VIP Box
• Yahoo iOS movies
• Yahoo Finance
• Yahoo News
• IT Head, Puma/Starlike
• SAP IS-Retail
I don’t know
what I don’t
know until I
know it. Ask
and learn.

3. Agenda
• What is serverless?
• What’s different?
• What is the benefits?
• What is the drawback?

4. What is Serverless?

5. Backend as a services (BaaS)
Serverless was first used to describe
applications that significantly or fully depend
on 3rd party applications / services (‘in the
cloud’) to manage server-side logic and state.

6. Functions as a services (Faas)
Serverless can also mean applications where
some amount of server-side logic is still written by
the application developer but unlike traditional
architectures is run in stateless compute
containers that are event-triggered, ephemeral
(may only last for one invocation), and fully
managed by a 3rd party.

7. History of Serverless
• 2012 - used to describe BaaS and Continuous Integration services run
by third parties
• 2014 - AWS launched Lambda
• 2015 - AWS launched API Gateway
• 2015 - AWS re:Invent The Serverless Company Using AWS Lambda
• 2016 - Serverless Conference

8. What’s different?

9. Let us discuss it from 3 different angels:
Application/services, Infrastructure, and Architecture
Application/
Services
Infrastructure
Architecture

10. Application / Services
Serverless is lightweight event-based
microservices.
- Google Functions

11. Application / Services
Google Cloud Functions is a lightweight, event-
based, asynchronous compute solution that
allows you to create small, single-purpose
functions that respond to cloud events without
the need to manage a server or a runtime
environment.

12. Infrastructure

13. Infrastructure
• Fully-managed by vendor
• Without managing server system: OS, CPU, memory, network
• Without managing server applications: apache, node.js, configurations
• Functions
• AWS Lambda: Javascript, Python, JVM language, C#
• Deploy
• BYOC: bring your own code. ZIP or code in console
• Scaling
• Request based automatically
• Trigger by events
• Defined by vendors: eg AWS S3, CloudWatch (schedule), Message Queue
• Http/s requests: eg. API Gateway, Webtask

14. FaaS vs PaaS

15. Architecture
• Stateless Function
• API gateway
• Event driven architecture

16. Stateless function

17. Shopping Cart Services

18. In first version, all three operations are scaled based on
the overall load across them, and state is transferred
in-memory, creating complication around caching
and/or routing sessions to instances in order to
maintain state.

19. Shopping Cart Services in FaaS

20. In the second version, each function scales individually
and as-needed. All state is in DynamoDB so there is no
cluster coordination of any sort, at the cost of a small
increase in latency.

21. Stateless functions
• Processes are stateless and share-nothing
• Any data that needs to persist must be stored in a stateful backing
service, typically a database.
• Don’t use “sticky sessions”
• https://12factor.net/processes

22. What is the benefits?

23. Benefits
• Reduce operational costs
• Infrastructure cost: without pay as it is idle*
• People cost: focus on function development
• Reduce development cost
• BaaS: eg. Firebase, Auth0
• Spend development time on business-specific code
• Maximize iterations
• Minimize dependences: IT Ops, DBAs
• Easier Operational management
• Scaling benefits of FaaS
• Reduced packaging complexity

24. Cost Benefits
If your
traffic is uniform and
would consistently
make good utilization
of a running server,
you may not see this
cost benefit and may
actually spend more
using FaaS.

25. What is the drawback?

26. Drawbacks
• Vendor Lock-in
• Vendor control
• Startup latency: worst case 3 second for JVM
• Execution duration: 300 second for AWS
• DoS yourself: AWS 1,000 concurrent / second
• Versioning and deployment
• If you have 20 functions for your application, how you handle deployment?
• Any easy way to roll back all 20 functions atomically?
• Testing/Monitoring / Debugging
• Repetition library or codes

28. Take Away

31. Where serverless make sense
• Fast is more important than elegant.
• Change in the application's functionality and usage is frequent.
• Change occurs at different rates within the application, so functional
isolation and simple integration are more important than module
cohesiveness.
• Functionality is easily separated into simple, isolatable components.
• Each single-function has one action.

32. Part II -
Deep into Architectures

33. API Gateway

37. Benefits
• Encapsulates the internal
structure of the application.
• Reduces the number of round
trips between the client and
application.
• Simplifies the client code
Drawbacks
• Yet another highly available
component.
• Performance and Scalability
• Wait inline to update the
gateway

38. Inter Process Communication

39. Is it Microservice? How to decouple?

41. Synchronous, Request / Responses

42. Asynchronous, Message-Based Communication Benefits
• Decouples the
client from the
service
• Message buffering
• Flexible client-
service interactions
• Explicit
inter-process
communication
Drawbacks
• Operational Cost
• Complexity of
implementing
request/response-b
ased interaction

43. Types of inter process commination
One-to-One One-to-Many
Synchronous Request / Response
Asynchronous Notification Publish / Subscribe
Request / async response Publish / async response

45. Define APIs
• Interface definition language (IDL)
• RAML (RESTful API Modeling Language)
• Swagger
• Two ways of defining APIs
• Top down (API-first approach)
• Bottom up (spec from your codes)
• Versions of a services
• Robustness principles

46. API Versioning

47. API Versioning
• General versioning rule. Suggest put this info in the return data.
• Major.Minor.Patch (example: 1.2.331)
• Four common ways to version a REST API
• URI Path
• Facebook: https://abc.com/api/v1/products
• URI parameters:
• NetFlix: https://abc.com/api/products?version=1
• Header Custom Field:
• Azure: x-ms-version: 2011-08-18
• Header Accept Field:
• GitHub: application/vnd.github.v3+json (vnd means vendor)

48. API Versioning Design Principle
• Should I handle API Versions in Routing Level?
• API Users focus first, URI or Domain name?
• Each API versioning in the same speed?
• How to handle default versioning?
• How to handle specific versioning?

49. Robustness Principle
Be conservative in what you do, be liberal in
what you accept from others.
Jon Postel, TCP father

50. API Maturity Model

51. Level 0 –
HTTP POST requests to its sole URL endpoint.

52. Level 1 –
HTTP POST requests to related resource

53. Level 2 – HTTP Verbs

54. Leve 3 - HATEOAS (Hypertext As The Engine Of
Application State)

55. Leve 3 - HATEOAS (Hypertext As The Engine Of
Application State)

56. API Error Handling

57. • Network timeout
• Limiting the number of outstanding requests
• Circuit break pattern
• Provide fallbacks

59. Before dive into event driven
architecture(EDA)

60. Something You Show Know
• Principles of microservices
• CAP Theorem
• ACID Model
• BASE Model
• Challenges of data consistent in distribution systems

61. Principles of microservices

63. Not allow to
change other
microservices’
internal database
directly

64. CAP Theorem

65. CAP Theorem (定理) - pickup 2 Consistency
Every read receives the most
recent write or an error
Availability
Every request receives a (non-
error) response – without
guarantee that it contains the
most recent write
Partition tolerance
The system continues to operate
despite an arbitrary number of
messages being dropped (or
delayed) by the network between
nodes

66. CAP Theorem in distributed network

67. ACID Model (CA)
• Atomic
• All operations in a transaction succeed or every operation is rolled
back.
• Consistent
• On the completion of a transaction, the database is structurally sound.
• Isolated
• Transactions do not contend with one another. Contentious access to
data is moderated by the database so that transactions appear to run
sequentially.
• Durable
• The results of applying a transaction are permanent, even in the
presence of failures.

68. BASE Model (AP)
• Basically Available:
• The database appears to work most of the time.
• Soft state
• Stores don’t have to be write-consistent, nor do different
replicas have to be mutually consistent all the time.
• Eventually consistent
• Stores exhibit consistency at some later point (e.g., lazily
at read time).

69. Challenges
• CAP theorem requires to choose between availability and ACID-style
consistency.
• 2 phases commit should be avoided
• Need a distributed transaction manager to support 2PC
• All distributed systems such Database and queue must support 2PC
• Most modern technologies, such as NoSQL database, does not have 2PC

70. Event Driven Architecture

71. Event Driven Architecture Example - Step 1

72. Event Driven Architecture Example - Step 2

73. Event Driven Architecture Example - Step 3

74. Event Driven Architecture Example
use events to maintain
materialized views that
pre-join data owned by
multiple microservices.
Customer Order View
could maintain by
MongoDB.

75. Event Driven Architecture
• Benefits
• Scale: it enables the implementation of transactions that span
multiple services and provide eventual consistency.
• Materialized view: It enables an application to maintain a data
store that contains the result of a query. It may be a join result or a
summary using an aggregate function.
• Drawbacks
• Handle atomically updating and publishing
• Deal with inconsistent data: change in-flight, materialized view is
not updated yet.
• Handle reverse state
• Handle time-ordered sequence of changes if needed

76. Achieving Atomicity

77. Atomically updating the database and
publishing an event
• Order Service must insert a row into the ORDER table and publish an
Order Created event.
• It is essential that these two operations are done atomically.
• If the service crashes after updating the database but before
publishing the event, the system becomes inconsistent.

78. Publishing Events Using Local Transactions
Benefit
• guarantees an event is
published for each
update without
relying on 2PC
Drawbacks
• Potential error-prone
on query job.
• Limited
implementation if not
sql-style database
Atomically

79. Mining database transaction or commit logs
Example:
AWS DynamoDB contains
time-ordered sequence of
changes (create, update, and
delete operations) made to
the items in a DynamoDB table
in the last 24 hours
Benefits
• guarantees that an
event is published
for each update
without using 2PC.
• simplify the
application by
separating event
publishing from the
application’s
business logic.
Drawbacks
• Transaction log
format is different
to each database
Atomically

81. Event Sourcing
Benefits
• Complete Rebuild
• Temporal Query
• Event Replay
Drawbacks
• Complexity on
implementation of
event store:
database plus
message broker
• Applications must
handle eventually
consistent data.

82. Deep into Event Sourcing

88. Command Query Responsibility
Segregation (CQRS)

89. CRUD vs CQRS
CRUD
Create, Read, Update, and Delete
CQRS
Command Query Responsibility Segregation

92. Event sourcing design principle
• Modeling events forces behavioral focus
• Modeling events forces temporal focus
• Event brainstorming.
• Events are immutable
• Optimization using snapshots
• CQRS

93. Take away

96. Data Centric vs Event Centric
The source of truth is the data store.
First priority: Preserve data
The source of truth is the log of events
First priority: React to events

97. One More Thing…

98. AWS Step Functions
https://aws.amazon.com/step-functions/details/

99. λλ
λ
DBMS
λ
λ
λ
λ
λ
λ λ
λ
λ
Queue
Modern
Serverless
app

100. Modern
Serverless
app

101. “I want to sequence functions”
“I want to select functions based on data”
“I want to retry functions”
“I want try/catch/finally”
“I have code that runs for hours”
“I want to run functions in parallel”
Functions into apps

102. Integration is the problem,
not the solution

103. AWS Serverless WorkShop Material
Download link: https://goo.gl/Ugdjsp

104. References
• https://martinfowler.com/articles/serverless.html
• https://www.youtube.com/watch?v=U8ODkSCJpJU
• https://cloud.google.com/functions/
• https://intl.aliyun.com/forum/read-499
• https://webtask.io
• https://blog.fugue.co/2016-01-31-aws-lambda-and-the-evolution-of-the-cloud.html
• https://12factor.net/processes
• https://www.slideshare.net/ServerlessConf/joe-emison-10x-product-development
• https://www.nginx.com/blog/building-microservices-using-an-api-gateway/
• https://www.nginx.com/blog/building-microservices-inter-process-communication/
• https://www.nginx.com/blog/event-driven-data-management-microservices/
• http://raml.org/
• http://swagger.io/
• https://en.wikipedia.org/wiki/Robustness_principle
• https://martinfowler.com/articles/richardsonMaturityModel.html
• https://www.xmatters.com/integrations/blog-four-rest-api-versioning-strategies/
• http://www.lexicalscope.com/blog/2012/03/12/how-are-rest-apis-versioned/
• https://developer.github.com/v3/media/#request-specific-version
• https://www.slideshare.net/danieljacobson/top-10-lessons-learned-from-the-netflix-api-oscon-2014

105. • http://open.taobao.com/docs/api.htm
• https://martinfowler.com/bliki/CircuitBreaker.html
• http://samnewman.io/talks/principles-of-microservices/
• https://en.wikipedia.org/wiki/CAP_theorem
• http://robertgreiner.com/2014/08/cap-theorem-revisited/
• http://queue.acm.org/detail.cfm?id=1394128
• http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.Tutorial.html
• https://github.com/cer/event-sourcing-examples/wiki/WhyEventSourcing
• https://www.slideshare.net/chris.e.richardson/building-and-deploying-microservices-with-event-sourcing-cqrs-and-
docker-microxchg-munich-microservices-meetup-2015
• https://msdn.microsoft.com/en-us/library/dn568103.aspx
• https://ordina-jworks.github.io/domain-driven%20design/2016/02/02/A-Decade-Of-DDD-CQRS-And-Event-
Sourcing.html
• https://www.youtube.com/watch?v=LDW0QWie21s
• https://www.slideshare.net/AmazonWebServices/announcing-aws-step-functions-december-2016-monthly-webinar-
series
• https://aws.amazon.com/step-functions/details/
• http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.html
• www.datawire.io/microservices-vs-soa
References

106. Appendix

112. Other message protocols / message format
• Thrift
• ProtoBuff - Protocol Buffers
• Apache Avro