Serverless enables us to develop applications that scale quickly and reliably based on incoming requests. These requests are often in the form of events that go well beyond API Gateway requests and scheduled rules in CloudWatch. In this Dev Chat, we’ll explore the different types of events available to your serverless applications, where they come from, and how to utilize them to build powerful applications that can automate business processes and provide more value to your customers.

1. S U M M I T
NEW YORK

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building event-driven applications
with serverless and AWS
Jeremy Daly
Chief Technology Officer, AlertMe.news
@jeremy_daly
D E V 1 1

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
How Serverless Works
Event-Driven Applications
Multicasting Events
Serverless Webhook Patterns

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
About Me
• CTO at AlertMe.news
• Consult with companies building in the cloud
• 20+ year veteran of technology startups
• Started working with AWS in 2009 and started
using Lambda in 2015
• Blogger (jeremydaly.com), open-source
contributor, speaker
• Publish the Off-by-none serverless newsletter
• Host of the Serverless Chats podcast

5. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Basic Serverless Workflow
LAMBDA
EVENT SOURCES
…
CLOUD RESOURCES
Event Trigger Unique
business logicEVENT
Output

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple Web Service
Client
API Gateway Lambda DynamoDB
“Synchronous”
Request

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple S3 Trigger
Client
Lambda DynamoDBS3 Bucket
“Asynchronous”
Request

9. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What are Events?
• Events are FACTS
• Events are IMMUTABLE
• Events are OBSERVABLE
• Events are TEMPORAL
Timestamp Event
2019-07-11 16:35:20 OrderCreated
2019-07-11 16:35:43 PaymentProcessed
2019-07-11 16:36:01 InventoryUpdated
2019-07-11 16:36:07 ConfirmationSent
… …

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Event-Driven versus Event-Sourcing
Event-Driven
• Domain events, i.e. something happened within a bounded context
• High-value semantics (user registered, order placed, payment processed)
• Relevant within a bounded context and across bounded contexts
Event-Sourcing
• A stored sequence of events that describes the current state
• Events are specific to a bounded context, i.e. domain logic is required to
reconstruct state (provide email, verify email)
• These events aren’t particularly interesting outside of the bounded context

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Communication Types
Synchronous Communication
Services can be invoked by other services and must wait for a reply. This is
considered a blocking request, because the invoking service cannot finish
executing until a response is received.
Asynchronous Communication
This is a non-blocking request. A service can invoke (or trigger) another service
directly or it can use another type of communication channel to queue information.
The service typically only needs to wait for confirmation (ack) that the request was
received.

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Orchestration versus Choreography 💃🕺
Orchestration
• Use a controller to coordinate the behavior of multiple components or services
• Uas a way to maintain state between multiple steps in a transaction
Choreography
• Messages are sent to an event bus or queue and are picked up by other services
• We are not waiting for downstream services to complete before continuing
“Each service is smart enough to understand its role in the whole dance.”
~ Sam Newman
AWS Step Functions

14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Using SNS to Route Events
Client
SNSTopic
“Asynchronous”
Request
ack
Serverless Microservice
HTTP
SMS
Lambda
SQS
Email
Key Points:
• SNS has a “well-defined API”
• Decouples downstream processes
• Allows multiple subscribers with message filters

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event Forking with SNS
“Asynchronous”
Request
ack
SQS
Queue LambdaSNS
Topic
DynamoDBClient API
Gateway
Lambda
Event processing pipeline
SQS
Queue Lambda Kinesis Firehose
Event Search & Analytics pipeline
Elasticsearch
S3 Dead Letter Bucket
SQS
Queue Lambda Kinesis Firehose
Event Storage & Backup pipeline
S3 Backup
Bucket
amount > $100
Key Points:
• SNS to SQS is “guaranteed” (100,010 retries)
• Filters allow you to control pipeline data
• Common patterns that can be reused
Event Fork Pipelines
Available in the Serverless Application Repository
Event Fork Pipelines
Available in the Serverless Application Repository

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distribute & Throttle
Stripe API
ack
SQS
Queue Lambda
(concurrency 25)
SNS
Topic
Client API
Gateway
Lambda
Order Service
total > $0
RDS
SQS
Queue Lambda
(concurrency 10)
SMS Alerting Service
Twilio API
SQS
Queue Lambda
(concurrency 5)
Billing Service
status == ”order_complete”
Event Service
Key Points:
• Filter events to selectively trigger services
• Manage throttling/quotas per service
• Reuses the same SNS topic to consolidate events

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudWatch Events
https://aws.amazon.com/blogs/aws/
building-serverless-pipelines-with-
amazon-cloudwatch-events/

19. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
EventBridge has 95+ Event Sources and 17 Target Types!
• Batch job queue
• CloudWatch log group
• CodeBuild project
• CodePipeline
• EC2 (CreateSnapshot,
RebootInstances, StopInstances,
TerminateInstances APIs)
• ECS task
• Event bus in another AWS account
• Firehose delivery stream
• Inspector assessment template
• Kinesis stream
• Lambda function
• SNS topic
• SQS queue
• SSM Automation
• SSM OpsItem
• SSM Run Command
• Step Functions state machine
NO SMS, HTTP, MOBILE or EMAIL

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
SNS versus CloudWatch Events versus Kinesis
• SNS subscriptions can FILTER events
• Advanced pattern matching through filter policies
• String and Numeric Value Matching (Whitelisting, Blacklisting, and Prefix
Matching and Range Matching) – ONLY ON MESSAGE ATTRIBUTES
• CloudWatch Event rules can MATCH events
• Matching is exact (character-by-character) using event patterns, without case-
folding or any other string normalization – ON ALL EVENT FIELDS
• Kinesis Streams can do NEITHER
• Event filtering must be done by the consumer

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Multicasting with EventBridge
Asynchronous
“PutEvents” Request
ack
w/ event id
Key Points:
• Create up to 100 event buses per account
• Allows multiple subscribers with RULES and EVENT PATTERNS
• Forward events to other accounts
Amazon
EventBridge
Lambda
SQS
Client
Step Function
Event Bus
+13 others

24. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Creating Rules with Targets

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Create custom event buses

26. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple Serverless Webhook
Client API Gateway Lambda DynamoDB
Highly Scalable Highly Scalable Highly Scalable

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple Serverless Webhook
Client API Gateway Lambda
Highly Scalable Highly Scalable
^
not so
RDS
NotThat Scalable 😳
RDBMS and FaaS don’t play nicely together:
• Concurrency model doesn’t allow connection pooling
• Limited number of DB connections available
• Recycled containers create zombies

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scalable Serverless Webhook
RDSClient API Gateway
SQS Queue
SQS (DLQ)
Lambda Lambda
(throttled)
ack
“Asynchronous”
Request
Synchronous
Request

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
High Throughput Serverless Webhook
RDSClient
SQS Queue
SQS (DLQ)
Lambda Lambda
(throttled)
ack
“Asynchronous”
Request
Synchronous
Request
ALB
Lambda
TargetType
Use Application Load Balancer
as your HTTP endpoint

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Serverless Webhook
RDSClient API Gateway
SQS Queue
SQS (DLQ)
Lambda
(throttled)
ack
“Asynchronous”
Request
Synchronous
Request
Use service integrations
with VTL to bypass Lambda

32. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Integrating with Partners (no more webhooks)
• Provide AWS Account ID to an approved Partner
• Partner submits request to publish events to AWS Account
• Select Partner Event Sources, approve access, and create event bus
• Partner publishes events (e.g. standard webhook events) using the
PutPartnerEvents API
• Account creates rules and event patterns to subscribe to events and route
them to targets
• Partner is responsible for retries to EventBridge

34. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EventBridge Partners

35. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EventBridge -Add Partners in 3 Steps

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EventBridge “Webhook”
RDSSaaS
Provider
SQS Queue
SQS (DLQ)
Lambda
(throttled)
SaaS Provider sends events
directly to an Partner specific
Event Bus in your AWS account
Amazon
EventBridge
Use rules and event patterns
to subscribe consumers

37. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Takeaways
• Events are immutable facts that can be multicast
• Events minimize service coupling and increase autonomy
• Think asynchronously and embrace eventual consistency
• Event-driven applications should broadcast events with high-value
semantics that are relevant across bounded contexts
• Use the right tool for the job, and don’t reinvent the wheel (SNS,
CloudWatch Events, Partner Event Buses, SQS, Event Fork Pipelines)
• Use Step Functions for distributed transactions

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Jeremy Daly
@jeremy_daly
jeremydaly.com

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.