Amazon CloudFront Best Practices and Anti-patterns

1. BEST PRACTICES & ANTI-
PATTERNS
CLOUDFRONT

2. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
ABOUT ME
• Solutions Architect
• Early AWS adopter (2007)
• Built Cotton On Group’s AWS Infrastructure (2012)
2

3. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
AGENDA
• CloudFront in a nutshell
• CloudFront Best Practices/Anti-Patterns
3

4. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
01
Content Delivery Network
Serves Static and Dynamic
Content
4
CLOUDFRONT

5. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
01
Global, Redundant, Scalable
Low Latency, High Bandwidth
Applications
5
CLOUDFRONT

6. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
01
Cost Effective
Transparent, PAYG, Price Classes
6
CLOUDFRONT

7. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
01
Wish list
Edge Side Includes and Reporting
7
CLOUDFRONT

8. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
MINIMUM TTL
REQUESTS
PROTOCOL
VIEWER
COOKIES &
QUERY STRING
FORWARDING
PATH
PATTERNS
CNAME
ALIASES
CACHE
BEHAVIORS
STREAMING
DISTRIBUTION
DELIVERY
METHOD
ORIGIN
SERVER
DOMAIN
NAME
OBJECTS
OBJECT PATH/
CACHE KEY
ORIGIN
PROTOCOL
DOWNLOAD
CLOUDFRONT (CF) CONCEPTS
8
[1-10, ORDERED]
[M:M]
[1:1]
[1-10]

9. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS 9
CloudFront Distribution
Origin Servers Cache Behavior
S3 Bucket with images
S3 Bucket with photos
EC2 Instance
default (*)
photos(photos/*)
thumbnail(thumbnail/*)
Cached objects with cache key
Edge Locations Viewer
Client Browser
Cache key = Object path relative to origin + forwarded query string/cookies
From nearest edge location GET
http://my.cloudfront.net/photos/profile.png
Returns object with matching cache key photos/profile.png
Cached Etag, Date, LastModified for each objects
http://<CloudFront domain name>/<object name in origin>
01CLOUDFRONT (CF) CONCEPTS

10. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS 9
CloudFront Distribution
Origin Servers Cache Behavior
S3 Bucket with images
S3 Bucket with photos
EC2 Instance
default (*)
photos(photos/*)
thumbnail(thumbnail/*)
Cached objects with cache key
Edge Locations Viewer
Client Browser
Cache key = Object path relative to origin + forwarded query string/cookies
From nearest edge location GET
http://my.cloudfront.net/photos/profile.png
Returns object with matching cache key photos/profile.png
Cached Etag, Date, LastModified for each objects
http://<CloudFront domain name>/<object name in origin>
Custom Origin
Dynamic Content
S3 Origin
Static Content
01CLOUDFRONT (CF) CONCEPTS

11. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
HOW IT WORKS
10
1. Client request file
2. If file in cache CF return fileNearest Edge
Location
a. If file not in cache,
CloudFront request to origin
b. Origin returns latest version of file,
status 200, cached in CloudFront
1. Client request file
2. If file in cache CF return fileNearest Edge
Location
a. If file in cache but expired,
CloudFront request to origin
b. Cached file is latest, status 304,
CloudFront keeps file in cache,
No change in Date, Expire
Refreshed Age
Condition GET
Routing based on low latency/price class 01

12. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. 10 Origin Servers per CF Distribution
2. 10 Cache Behavior per CF Distribution
3. 10 CNAME per CF Distribution
4. 10 Cookies forwarded per Cache Behavior
11
01MAKE A NOTE

13. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Cache invalidation
ANTI-PATTERNS
12

14. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS 13
02BEST PRACTICES

15. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
13
02BEST PRACTICES

16. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
13
02BEST PRACTICES

17. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
3. Domain Sharding
13
02BEST PRACTICES

18. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
3. Domain Sharding
4. Expiration
13
02BEST PRACTICES

19. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
3. Domain Sharding
4. Expiration
5. Bucket Organization
13
02BEST PRACTICES

20. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
3. Domain Sharding
4. Expiration
5. Bucket Organization
6. Logging
13
02BEST PRACTICES

21. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
1. Versioning
2. Compression
3. Domain Sharding
4. Expiration
5. Bucket Organization
6. Logging
7. Performance Testing
13
02BEST PRACTICES

22. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Expensive
Unmanageable if you have object
dependencies
CACHE INVALIDATION
14

23. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Programmatic
3 invalidation requests at any
given time with each include
maximum of 1000 files
CACHE INVALIDATION
15

24. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Eventual Consistency
Invalidation takes time to
propagate across all edge
locations
CACHE INVALIDATION
16

25. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02Deal breaker: Browser cache
Versioning is best way to avoid
the invalidation related issues
CACHE INVALIDATION
17

26. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Versioning using query strings
File name plus query string with version: /static/profile.png?versionID=123
VERSIONING
18

27. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Requires Versioning enabled
Origin (Custom/S3)
VERSIONING
19

28. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
UPLOAD TO S3
STATIC/IMAGES/PROFILE.PNG
PROFILE.PNG RETURN
X-AMZ-VERSION-ID = 644C69E1
CF KEY
STATIC/IMAGES/PROFILE.PNG?
VERSIONID=644C69E1
VERSIONED ENABLED S3 CONTENT
20
02

29. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
MINIFY+COMBINE
MAIN.CSS
MAIN.JS
CSS AND JS
PHOTO.CSS, APP.CSS, PROFILE.CSS
PHOTO.JS, APP.JS, PROFILE.JS
VERSIONIFY
BUILD NUMBER OR HASH OF
GIT/HG HEAD
CF KEY
MAIN.CSS?V=468DF6B
MAIN.JS?V=468DF6B
VERSIONIFY APPLICATION CONTENT
21
02

30. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Versioning using content based
hash key
File name as unique key based on file content: /static/712vds57tr18929812312enb.png
VERSIONING
22

31. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
MD5 HASHUSING FILE CONTENT
B723EH0F0DF
PROFILE.PNG UPLOAD TO S3
USE HASH AS FILE NAME
STATIC/IMAGES/B723EH0F0DF.PNG
CF KEY
STATIC/IMAGES/B723EH0F0DF.PNG
FOR S3 CONTENT
23
Avoid content duplication

32. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
MINIFY+COMBINE
STATIC/MAIN.CSS
STATIC/MAIN.JS
CSS AND JS
PHOTO.CSS, APP.CSS, PROFILE.CSS
PHOTO.JS, APP.JS, PROFILE.JS
MD5 HASHOF FILE CONTENT AS FILE NAMES
STATIC/8972BW7DYF2H.CSS
STATIC/67BFWU9HHUW.JS
CF KEY
STATIC/8972BW7DYF2H.CSS
STATIC/67BFWU9HHUW.JS
VERSIONIFY APPLICATION CONTENT
24
02

33. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Versioning using prefix/suffix
File name with version prefix/suffix: /static/profile_123.png
File name with version prefix/suffix: /static/profile_ 712vds57tr18929812312enb.png
VERSIONING
25

34. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Compressed content is served
faster and uses less bandwidth
COMPRESSION
26

35. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Can serve both compressed and
uncompressed version of files
COMPRESSION
27

36. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
For compression CF relies on the
origin servers (Custom vs S3)
COMPRESSION
28

37. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
55
Viewer request
To receive compressed content
browser request must include
Accept-Encoding: gzip
COMPRESSION
29

38. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Upload both gzip and non-gzip
versions of the file in the same
S3 bucket
COMPRESSION
30

39. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
When uploading gzip file to S3
set Content-Encoding to gzip
COMPRESSION
31

40. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
HTML, CSS, JS etc
On-the-fly compression by
custom origin server (Nginx,
Apache)
COMPRESSION
32

41. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02Use one/both of these
•On Origin server set Cache-
Control max-age header
•In Cache Behavior set Minimum
TTL
EXPIRATION
33

42. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Default expiration 24hrs
Set Cache-Control max-age or
Minimum TTL very far future
max {Cache-Control max-age directive, Cache Behavior Minimum TTL}
EXPIRATION
34

43. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Depending on request frequency,
CloudFront might remove the
object before its expiration
EXPIRATION
35

44. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Avoid Expire header
After expiry browser cachability
issues (stale response)
Can not cache object if {Age <=0 OR Expire Date - Current Date <=0}
EXPIRATION
36

45. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Browser limit
number of simultaneous
synchronous connections to one
server (n)
DOMAIN SHARDING
37
Client Browser Server
time->
GET
200 OK
GET
200 OK
X n

46. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Browser treat CNAM aliases as different servers
Bypass parallel download limit
using 2 or more CNAM aliases
Route 53, Wild Card CNAME, Custom SSL
DOMAIN SHARDING
38

47. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Sharding across 2 CNAME aliases
will double the parallel download
DOMAIN SHARDING
39
Client Browser CNAME2 ->Server
time->
GET
200 OK
GET
200 OK
X n
Client Browser CNAME1 ->Server
time->
GET
200 OK
GET
200 OK
X n

48. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
On application side
Enable CF sharding logic
(template compilation etc)
DOMAIN SHARDING
40

49. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
41
<span style="width:inherit;">
<a href="http://mysite.com/about"><img src="http://cf.mycloudfront.net/about.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/blogs"><img src="http://cf.mycloudfront.net/blogs.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/contact"><img src="http://cf.mycloudfront.net/cont.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/about"><img src="http://cf1.mycloudfront.net/about.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/blogs"><img src="http://cf2.mycloudfront.net/blogs.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/contact"><img src="http://cf1.mycloudfront.net/cont.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/about"><img src="{{ CDN_DOMAINS }}/about.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/blogs"><img src="{{ CDN_DOMAINS }}/blogs.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/contact"><img src="{{ CDN_DOMAINS }}/cont.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/about"><img src="http://mysite.com/about.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/blogs"><img src="http://mysite.com/blogs.jpg" alt=""/></a>
</span>
<span style="width:inherit;">
<a href="http://mysite.com/contact"><img src="http://mysite.com/cont.jpg" alt=""/></a>
</span>
CDN with domain shardingCDN without domain sharding
HTML Template
Without CDN
Compiled OR
DOM Manipulation
Compiled OR
DOM Manipulation

50. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Performance Considerations
•When to shard?
•DNS lookup, Connection time, Evenly distribute
•Mobile browsers, Network congestion, Battery Life
•SPDY (HTTP MULTIPLEXING)
DOMAIN SHARDING
42

51. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Limited Cache Behavior
Plan your bucket organisation
(object keys)
BUCKET ORGANISATION
43

52. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Watch out for the object paths
/video/uploads/, /audio/uploads/, /images/uploads/ BAD
/uploads/video/, /uploads/audio/, /uploads/image/ GOOD
Use wild card patterns (/uploads/*)
BUCKET ORGANISATION
44

53. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02Reporting non-existence
•Object/Content Popularity Report
•Cache Hit Ratio
•Edge Location Report
•Error Report
•Mapping Geo-IP and Geo-location
LOGGING
45

54. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Enable logging
Store access log file to S3. One
bucket per distribution.
LOGGING
46

55. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Tons of log files
As traffic levels climb, the
number of log files will increase
Delayed (24H), missing logs, not in order
LOGGING
47

56. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Combine log files and analyze
Using EMR (Custom scripts,
CloudFront LogAnalyzer,
EmrEtlRunner)
LOGGING
48

57. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Measuring Latency
Delay in DNS resolution and
content delivery
PERFORMANCE TESTING
49

58. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Backbone Testing (Ideal Word)
Measuring latency from
CDN Server → Backbone
PERFORMANCE TESTING
50

59. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Last Mile Testing (Real World)
Measuring latency from
CDN Server → Backbone → Device
3rd party service (Gomez), Application instrument
PERFORMANCE TESTING
51

60. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02
Quantifying Scalability
Tradition Load Testing not useful
One Client → Same DNS Request → Same Set of IPs
PERFORMANCE TESTING
52

61. 06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS
02Load Testing (Edge Location, EIP)
1.Multiple Client from different geolocations
2.Independent DNS request, different set of IPs
3.Distribute load across set of IPs
PERFORMANCE TESTING
53

62. Q & A
THANK YOU
http://bit.ly/abhishektiwari
http://abhishek-tiwari.com