AWS Lambda and the Serverless Cloud

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Andreas Chatzakis, Solutions Architect
AWS Pop-Up Loft London
October 2015
AWS Lambda and the
Serverless Cloud

2. AWS Lambda
A compute service where you
don’t have to think about:
• Servers
• Being over/under capacity
• Deployments
• Scaling and fault tolerance
• OS or language updates
• Metrics and logging
…but where you can easily
• Bring your own code…
even native libraries
• Run code in parallel
• Create backends, event
handlers, and data
processing systems
• Never pay for idle!

3. AWS Lambda – Benefits
EVENT-DRIVEN SCALESERVERLESS SUBSECOND BILLING

4. AWS Lambda – Capabilities
BRING YOUR OWN CODE SIMPLE RESOURCE MODEL
FLEXIBLE INVOCATION PATHS GRANULAR PERMISSIONS CONTROL

5. AWS Lambda – How It Works
DEPLOYMENT
AUTHORING
MONITORING & LOGGING
STATELESS

6. The AWS Lambda execution environment
Linux kernel version : 3.14.44-32.39.amzn1.x86_64
Node.js: v0.10.36 (ImageMagick, AWS JavaScript SDK
2.2.3)
Java: Java 8
Python: Python 2.7 (AWS SDK for Python - Boto 3)

7. AWS Services Integrated with AWS Lambda
Amazon
S3
Amazon
DynamoDB
Amazon
Kinesis
AWS
CloudTrail
Amazon
CloudWatch
Logs
AWS
CloudFormation
Amazon
SNS
Amazon
SWF
Amazon
SES
Amazon
API Gateway
Amazon
Cognito
November 13, 2014

8. AWS Lambda – Partner Blueprints

9. AWS Lambda – Push Model
• Respond to a specific event
• Services
• Amazon S3
• Amazon SNS
• Amazon Cognito
• Amazon Echo

10. AWS Lambda – Pull Model
• Polls the event source
• Services
• Amazon Kinesis
• Amazon DynamoDB
Streams

11. AWS Lambda – Direct Invocation Model
• Respond to invocation
• Services
• Custom code
• Amazon API Gateway

12. Languages and Features
re:Invent 2014
Preview Launch
April 2015
GA
Summer 2015 re:Invent 2015
• Node.js
• Event handlers
• CORS
• Mobile
Backends
• Sync calls
• Resource
policies
• Java
• Amazon S3
uploads
• Blueprints
• 1.5 GB
• Tokyo region
• Alexa Skills
• …and…

13. In case you missed it…

14. Amazon Simple Email Service Inbound Rules
// Spam check
if (sesNotification.receipt.spamVerdict.status === 'FAIL‘ ||
sesNotification.receipt.virusVerdict.status === 'FAIL')
console.log('Dropping spam');

15. Anonymous Email

16. Anonymous Email — Requirements
• Protect customer information
• Utilize email standards
• Secure and resilient to abuse
• Fault tolerant
• Provide auditing and history

17. Amazon DynamoDB or Amazon RDS
Anonymous Email — With AWS
Receiving email
servers
Spam & virus
scanning
Routing and
application logic
Database
Sending email
servers
Seller Buyer
Amazon SESAmazon EC2
Amazon SES
Amazon Lambda

19. Amazon CloudWatch Logs Processing
Scan, audit, or index log entries in near real time
AWS LambdaAmazon CloudWatch
Logs
Amazon
DynamoDB
Amazon S3
Amazon
Redshift

20. New Feature: Python 2.7

21. Python 2.7 Support in AWS Lambda
• Available today in
• All SDKs (including mobile SDKs)
• AWS CLI
• Lambda console (including interactive editing)
• Includes
• Version 1.1.3 of boto3 (AWS Python SDK)
• Documentation and walkthroughs
• Console blueprints

22. New Feature: Longer-running Functions

23. Longer-running AWS Lambda Functions
• Run functions for up to 5 minutes
• Available today in the following SDKs
• Python (boto3)
• Java
• Ruby
• Node.js
• PHP
• AWS Mobile SDK for Android
• AWS Mobile SDK for iOS

24. Resource Sizing
• AWS Lambda offers 23 “power levels”
• Higher levels offer more memory and more CPU power
• 128 MB, lowest CPU power
• 1.5 GB, highest CPU power
• Higher power levels == lower latency for CPU-bound
and bursty tasks
• Compute price scales with the power level
• Duration ranging from 100ms to 5 minutes
• Free Tier: 1M free requests and 400,000 GB-s / month

25. New Feature: Scheduled Functions

26. Scheduled AWS Lambda Functions
• Available today in the Lambda console
• Schedule functions at a specific time or recurring
• Accepts standard cron syntax
• 5 minute granularity
• You can get sub-second granularity using a Lambda function
• Easily poll Amazon SQS or other data sources!
• Coming later in 2015: CLI, SDK support

27. New Feature: Versioning

28. Versioning: Development
Developing in AWS Lambda stays simple:
• Upload code
• Make changes any time
• Last update wins
exports.handler =
function(event,context)
{context.succeed(“bye”);}
exports.handler =
function(event,context)
{context.succeed(“hi”);}

29. Versioning: Publishing
Publish new versions from development at any time:
• “Copies” dev version to a numbered version
• Published versions are read-only (including configuration)
• Simple, integer counter per function
exports.handler =
function(event,context)
{context.succeed(“bye”);}
exports.handler =
function(event,context)
{context.succeed(“hi”);}
1
2
Versions

30. Versioning: Calling Lambda Functions
Development version:
FunctionName (or)
FunctionName:$LATEST
Specific version:
FunctionName:1
FunctionName:2
Named version:
FunctionName:production
FunctionName:v1_2_3_4

31. Versioning: Aliases
Create named aliases to any version:
• Allows function owner to map ARNs to code
• Can be updated without changing clients
exports.handler =
function(event,context)
{context.succeed(“bye”);}
exports.handler =
function(event,context)
{context.succeed(“hi”);} prod
dev
Aliases

32. Amazon API Gateway:
Version your APIs
/prod/my_url_endpoint 
MyFunction:prod_rel
Versioning APIs and Code
MyFunction:prod_rel 
Function:3 
{your code}
AWS Lambda:
Version your code

33. New Feature: IoT Backends

34. AWS IoT + AWS Lambda

35. Sneak Peek: VPC Access

36. AWS Lambda VPC Access
• Select the functions to run in your VPC
• Select subnets and security groups to use
• Your Lambda function can access the private resources
you choose:
• Amazon Elasticache
• Amazon RDS
• Private EC2 endpoints
• Any other resources in your VPC
• Launching later this year in all AWS Lambda regions

37. What can you do with AWS
Lambda?

38. Five Sample Use Cases for AWS Lambda
Serverless
Microservices
Adding a
Feature to
Amazon S3
Extending
Platforms
Scalable
Mobile
Backends
Real-Time
Streaming
Analysis

39. SOURCE of
data
Fleet of servers poll for
changes
Listening to source Pushes to queue Pull off queue
Fleet of servers act as
workers to process the data
Auto-scale
worker nodes
to adjust with
load
S3 objects
Amazon Kinesis records
DynamoDB tables
SNS messages
IoT devices
…
Cross-AZ
replication for
high
availability
Load
balancer to
distribute
traffic
Data Processing Architecture with Servers
Application code

40. SOURCE of
data
Attach a Lambda function
And that’s it!
Lambda does
Listening/polling
Queuing
Auto scaling
Redundancy
Load balancing
Data Processing Architecture Without Servers
Application code

41. Real-Time Data Processing
Architecture Patterns

42. Sample Real-Time File Processing Workflow
Using Amazon S3 & AWS Lambda
Amazon S3 AWS Lambda
Amazon S3
New file
uploaded
Amazon
DynamoDB

43. Sample – Video Clip Transcode
// Transcode video files uploaded to an S3 bucket
var async = require('async');
var fs = require('fs');
var exec = require('child_process').exec;
var aws = require('aws-sdk');
var s3 = new aws.S3();
// Lambda entry point
exports.handler = function(event, context) {
var inputVideo = '/tmp/myVideo.avi';
var outputVideo = '/tmp/myVideo.mp4';
var inputBucket = event.Records[0].s3.bucket.name;
var inputKey = event.Records[0].s3.object.key;
var outputBucket = inputBucket + '-out';
var outputKey = inputKey.substr(0,inputKey.length-4) + '.mp4';
// Runs the array of functions in series, each passing their results to the next
async.waterfall([
Set up variables
Serialize steps

44. Sample – Video Clip Transcode
// Download the object from S3 into a buffer
function download(next) {
s3.getObject({Bucket: inputBucket, Key: inputKey}, next); },
// Write the buffer to a file
function write(response, next) {
fs.writeFile(inputVideo, response.Body, next); },
// Spawn a process to transcode
function transcode(next) {
exec("./ffmpeg -i " + inputVideo + " -vcodec mpeg4 -b:v 1200k “
+ outputVideo, next); },
// Read the file to a buffer
function read(next) {
fs.readFile(outputVideo, next); },
// Upload the buffer to an object in S3
function upload(objectBuffer, next) {
s3.putObject({Bucket: outputBucket, Key: outputKey, Body: objectBuffer},
context.done); },
], context.fail);
};
Get file from S3
Write to disk
ffmpeg transcode
Read from disk
Upload to S3

45. Sample Real-Time Stream Processing Workflow
Using Amazon Kinesis & AWS Lambda
Amazon Kinesis AWS Lambda
Amazon Redshift
Amazon SNS
New records
available

46. Sample Real-Time DB Triggers Workflow Using
Amazon DynamoDB & AWS Lambda
New data
available
Amazon DynamoDB AWS Lambda
Amazon
DynamoDB
Amazon
Redshift

48. Sample Real-Time Message Handling Workflow
Using Amazon SNS & AWS Lambda
New message
published
Amazon SNS AWS Lambda
Amazon SNS
Amazon Kinesis

49. Sample CRUD Backend Workflow Using
Amazon API Gateway & AWS Lambda
New API
called
AWS Lambda
Amazon S3
Amazon
DynamoDB

50. New Capabilities for AWS Lambda Functions
Public HTTP endpoints
Throttling controls
API Key management
Results caching
SDK Generation and Swagger support
Apache Velocity templates
API mocking

51. Managed
A new, fully managed model
InternetMobile apps
AWS Lambda
functions
AWS
API Gateway
cache
Endpoints on
Amazon EC2
Any other publicly
accessible endpoint
Amazon
CloudWatch
Amazon
CloudFront
API
Gateway
Other AWS
services
AWS Lambda
functions

52. Method and integration

55. More Triggers for AWS Lambda
AWS
CloudFormation
Custom Resources
Amazon SES
Actions
Amazon Cognito
Sync Triggers
… and the list will
continue to grow!
AWS IoT ActionsAWS Lambda
Scheduled Events
Amazon Echo
Skills
Amazon SWF
Tasks

56. Using Lambda to Audit CloudTrail Activity
AWS
Lambda
Amazon S3Amazon CloudTrail
Amazon SNS
AWS IAM

57. Using Lambda for Automated Infrastructure
Management
AWS
Lambda
Amazon
SNS
Amazon
CloudWatch Alarm
ec2
runInstance
ecs
startTask
beanstalk
updateApp
kinesis
splitShard
Any API call

58. https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-and-aws-lambda/

59. Using Lambda to Forward AWS Events to
External Endpoints
http://danilop.net/aws/2015/07/26/sns2ifttt/ | https://github.com/danilop/SNS2IFTTT
AWS
Lambda
Amazon
SNS
IFTTT via
the Maker
channel
Amazon
CloudWatch Events
Auto Scaling

60. Using Lambda to Deploy Lambda Functions
https://aws.amazon.com/blogs/compute/dynamic-github-actions-with-aws-lambda/
AWS
Lambda
Amazon
SNS
GitHub Repo
lambda
createFn ()

61. A Few Other Tips and Tricks for Extending
AWS Lambda Functionality
 Use /tmp space as cache
 Run arbitrary binaries
 Use Kappa, Grunt and Jenkins plugins for deploying
 Hook up to Slack
 Build event-driven Amazon ECS
 Try out JAWS: The Server-less Application Framework

62. We’ve been busy.
Now it’s your turn.

63. Go to the AWS Lambda console,
create a function, and run it.
(The first million invokes are on us!)

64. Congrats, you’re a Lambda
function expert! Add an event
source or an HTTP endpoint.

65. Thank you!
Follow AWS Lambda!
aws.amazon.com/blogs/compute
aws.amazon.com/lambda
AWS Lambda Forum