This talk dives into Trustpilot's journey to serverless compute. The journey starts at re:Invent 2016 and follows how the company fast-tracked its adoption within its engineering organization using a "serverless first" engineering principle. A representative from Trustpilot shares lessons learned and insights gained from running over 200 AWS Lambda functions with 12M invocations/day in production. Also covered are fun stories of what helped the company adopt serverless, how to make those stories actionable, a review of architectural patterns, and a discussion of why they choose serverless over traditional compute every day. This session is part of re:Invent Developer Community Day, a series led by AWS enthusiasts who share firsthand technical insights on trending topics.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless: It All Started in Vegas
Martin Buberl
VP Engineering | AWS Community Hero
Trustpilot
D V C 3 0 6

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Martin Buberl
• VP Engineering at Trustpilot
• AWS Community Hero
• AWS User Group Leader
Copenhagen, Denmark
@martinbuberl
martinbuberl.com | trustpilot.com

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• Back then ... and now
• What are all those AWS Lambda functions doing?
• Some ideas to take away and start building
• Fast track to serverless – how we did it
• Some ideas to take away to adopt serverless
• Looking back – any advice?

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Las Vegas, December 2016

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tatonka challenge

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless “FaaS” started to make sense
λ
λ λ
λ λ
λ λ
λ λ
λ λ
λ
λ
λ
λ
λ λ
λ
λ
λ λ
λλ λ
λ
λ
λ
λ
λ
λ
λ

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless “FaaS” wasn’t a total stranger back then
• Couldn’t use Lambda before February 2016
Requirement to run them in VPCs
• Already had about 40 Lambda functions in production
Mostly tooling, data pipelines, and some experiments

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
8K Charity Fun Run

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud compute with AWS
0
100
200
300
400
500
600
700
January-00 January-00 January-00 January-00
Virtual servers, containers, and serverless compute
Series1

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud compute with AWS
0
100
200
300
400
500
600
700
January-00 January-00 January-00 January-00
Virtual servers, containers, and serverless compute
Series1 Series2 Series3

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud compute with AWS
0
50
100
150
200
250
300
350
January-00 January-00 January-00 January-00
Virtual servers, containers, and serverless compute
Series1 Series2 Series3

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cloud compute with AWS
December 2016
180 virtual servers
80 containers
40 Lambda functions
November 2018
95 virtual servers
283 containers
252 Lambda functions
-53%
+354%
+630%

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
+630%

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trustpilot’s high-level architecture
• Cloud native
• Event-driven microservices
• REST APIs

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trustpilot’s high-level architecture
Web clients & applications
Data
API management & orchestration
Pub/Sub messaging system
Compute
Third-party applications
Amazon API Gateway
REST APIs
trustpilot.com
Amazon Simple Notification Service (Amazon SNS)
Amazon Simple Queue Service (Amazon SQS)
Cloud compute
Amazon Relational Database
Service (Amazon RDS) NoSQL Amazon Simple Storage Service (Amazon S3)
Amazon CloudFront CDN
b2b.trustpilot.com
Amazon CloudFront CDN
widgets.trustpilot.com
Amazon CloudFront CDN

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trustpilot’s high-level architecture
Web clients & applications
Data
API management & orchestration
Pub/Sub messaging system
Compute
Third-party applications
Amazon API Gateway
REST APIs
trustpilot.com
Amazon Simple Notification Service (Amazon SNS)
Amazon Simple Queue Service (Amazon SQS)
Cloud compute
NoSQL Amazon Simple Storage Service (Amazon S3)
Amazon CloudFront CDN
b2b.trustpilot.com
Amazon CloudFront CDN
widgets.trustpilot.com
Amazon CloudFront CDN
Amazon Relational Database
Service (Amazon RDS)

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trustpilot’s high-level architecture
Web clients & applications
Data
API management & orchestration
Pub/Sub messaging system
Compute
Third-party applications
Amazon API Gateway
REST APIs
trustpilot.com
Amazon Simple Notification Service (Amazon SNS)
Amazon Simple Queue Service (Amazon SQS)
Cloud compute
NoSQL Amazon Simple Storage Service (Amazon S3)
Amazon CloudFront CDN
b2b.trustpilot.com
Amazon CloudFront CDN
widgets.trustpilot.com
Amazon CloudFront CDN
Amazon Relational Database
Service (Amazon RDS)

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Publish/Subscribe messaging pattern
Publisher
Subscriber
Publisher
Subscriber
Subscriber
Topic

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Traditional” auto scaling on queues
Queue
Auto Scaling group
Auto Scaling group
Containers
Instances
Queue
Message
Message
Amazon SQSAmazon SNS
Topic
Publisher
Publisher

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instant scaling
Serverless “instant” scaling on topics
Message
Message
Amazon SNS
Topic
Publisher
Publisher
Lambda
function
Lambda
function

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instant scaling
Serverless “instant” scaling on queues
Queue
Queue
Message
Message
Amazon SQSAmazon SNS
Topic
Publisher
Publisher
Lambda
function
Lambda
function

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What are all those AWS Lambda functions doing?
Most of our Lambda functions in production ...
• Integrate with API Gateway
• Subscribe to topics from our pub/sub messaging system
Perfect for decoupled, event-driven microservices
Our next step is to ...
• Bring back queues for “auto-healing”

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Git webhook to Slack pub/sub
AWS Cloud
Webhook
Publish Subscribe
Incoming webhook(s)
developer.github.com/webhooks
HTTP POST
api.slack.com/incoming-webhooks

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda for git security audit
• Ever accidentally committed a secret to source control?
For example, connection strings, tokens, API keys and secrets, private certificate keys
• Have a Lambda check for secrets and post to #secrets

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda for git security audit – Version 2
• Send a message directly to the pull request in GitHub

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda for git privacy audit
• Ever accidentally committed PII to source control?
For example, email addresses, usernames, social security numbers
• Have a Lambda check for PII and post to #privacy
• GDPR anybody?

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda for naming discussions
• There are only two hard problems in computer science:
Cache invalidation, naming things, and off-by-one errors
• Have a Lambda post new repositories to #naming-things

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda for Trello integration
• Have a Lambda automatically attach a PR to a Trello card

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Git webhook to SNS pub/sub
Subscribing Lambda functions for ...
• git security audit
• git privacy audit
• naming discussions
• Trello integration

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Engineering principle

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Engineering principle
github.com/trustpilot/principles

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Golden circle
WHY
“Start With Why”
by Simon Sinek

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Engineering principle
github.com/trustpilot/principles

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why should I care?
Because we ...
• Have a vision and want to be part of the movement
• Acknowledge it’s not always right but start there
• Want to modernize our cloud compute stack
• See operational benefits
• Anticipate cost savings

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Engineering principle
Virtual servers are considered legacy and should be avoided.
github.com/trustpilot/principles

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Virtual servers are considered legacy
If that’s the case ...
• Amazon Elastic Compute Cloud (Amazon EC2) is legacy
Which means also ...
• .NET framework is legacy
As it’s not supported with Docker containers

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
+

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hackathons
• Hackathons are great for serverless
Not just FaaS, all fully managed cloud services
• Serverless lets us ship prototypes that scale
Which we probably shouldn’t do

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Labs

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Trustmap
Node.js
Application
24h Cache

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event “anomaly” detection
S3 event trigger
NotificationEvent logs
Analyze event

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GTFO initiative

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GTFO EC2
• Inventory of Amazon EC2s per team
• Weekly check-ins with tech leads
• Dedicated Slack channel
#gtfo-ec2

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
#gtfo-ec2

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
#gtfo-ec2

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The cake alert

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS IoT to Slack
Incoming webhook
tech.trustpilot.com/the-cake-alert-28652f13607d

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How did we fast track serverless?
• Serverless first
Engineering principle
• Hackathons and labs
• GTFO Amazon EC2

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Are AWS Lambda functions really cheaper?

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
!==

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda functions
• Gutfeel is – for us – they are 10x cheaper

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda functions
• Gutfeel is – for us – they are 10x cheaper
• Less ops heavy
• Faster development for services at scale

64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Lambda functions
• Gutfeel is – for us – they are 10x cheaper
• Less ops heavy
• Faster development for services at scale
• Decrease of observability and traceability
We’re investing in this area

65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

66. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Martin Buberl
@martinbuberl

67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.