More Related Content Similar to How to send gzipped requests with boto3 (20) More from Luciano Mammino (20) How to send gzipped requests with boto31. How to send gzipped requests
with boto3
Luciano Mammino ( )
@loige
AWS UserGroup Dublin
2021-07-01
loige.link/gzip-boto3
1
5. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
3
6. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
nodejsdp.link
Co-Author of Node.js Design Patterns 👉
3
7. Let me introduce myself first...
I'm Luciano ( 🍕🍝) 👋
Senior Architect @ fourTheorem (Dublin )
nodejsdp.link
Co-Author of Node.js Design Patterns 👉
Connect with me:
(blog)
(twitter)
(twitch)
(github)
loige.co
@loige
loige
lmammino 3
8. We are business focused technologists that
deliver.
| |
Accelerated Serverless AI as a Service Platform Modernisation
We are hiring: do you want to ?
work with us
loige 4
15. for event in payload['logEvents']:
# ...
cw_client.put_metric_data(...)
loige 10
16. for event in payload['logEvents']:
# ...
cw_client.put_metric_data(...)
loige
Sending 1 metric per log line... 🙈
10
17. [
{
'MetricName': 'SomeMetric1',
'Dimensions': [
{
'Name': 'Dimension1Name',
'Value': 'Dimension1Value'
},
# Up to other 9 dimensions here ...
],
'Unit': 'Count',
'Values': [217, 220, 221], # Up to 150 items here ...
'Timestamp': 1624290910000
},
# Up to other 19 metric items here ...
]
loige 11
18. [
{
'MetricName': 'SomeMetric1',
'Dimensions': [
{
'Name': 'Dimension1Name',
'Value': 'Dimension1Value'
},
# Up to other 9 dimensions here ...
],
'Unit': 'Count',
'Values': [217, 220, 221], # Up to 150 items here ...
'Timestamp': 1624290910000
},
# Up to other 19 metric items here ...
]
loige
⚠Payload size
limit: 40 KB
11
22. import boto3
import gzip
endpoint_url = "http://localhost:8000/"
cw_client = boto3.client('cloudwatch', endpoint_url=endpoint_url, use_ssl=False)
cw_client.put_metric_data(
MetricData = [
{
'MetricName': 'TEST_BOTO',
'Dimensions': [
{
'Name': 'APP_VERSION',
'Value': '1.0'
},
],
'Unit': 'None',
'Value': 17
},
],
Namespace='BotoTest'
)
loige 14
23. import boto3
import gzip
endpoint_url = "http://localhost:8000/"
cw_client = boto3.client('cloudwatch', endpoint_url=endpoint_url, use_ssl=False)
cw_client.put_metric_data(
MetricData = [
{
'MetricName': 'TEST_BOTO',
'Dimensions': [
{
'Name': 'APP_VERSION',
'Value': '1.0'
},
],
'Unit': 'None',
'Value': 17
},
],
Namespace='BotoTest'
)
loige 14
24. import boto3
import gzip
endpoint_url = "http://localhost:8000/"
cw_client = boto3.client('cloudwatch', endpoint_url=endpoint_url, use_ssl=False)
cw_client.put_metric_data(
MetricData = [
{
'MetricName': 'TEST_BOTO',
'Dimensions': [
{
'Name': 'APP_VERSION',
'Value': '1.0'
},
],
'Unit': 'None',
'Value': 17
},
],
Namespace='BotoTest'
)
loige 14
29. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
loige 19
30. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
loige 19
31. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
5
6
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
loige 19
32. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
5
6
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# get the event system for the lambda_client
event_system = lambda_client.meta.events
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
9
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
loige 19
33. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
5
6
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# get the event system for the lambda_client
event_system = lambda_client.meta.events
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
9
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
12
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
loige 19
34. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
5
6
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# get the event system for the lambda_client
event_system = lambda_client.meta.events
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
9
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
12
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
15
16
loige 19
35. import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
1
2
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
5
6
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# get the event system for the lambda_client
event_system = lambda_client.meta.events
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
9
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
12
13
14
# invoke a lambda function
15
lambda_client.invoke(FunctionName='my-function')
16
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
import boto3
1
2
lambda_client = boto3.client('lambda')
3
4
# our event handler
5
def add_xtrace_header(request, **kwargs):
6
request.headers.add_header('x-trace-id', 'trace-trace')
7
8
# get the event system for the lambda_client
9
event_system = lambda_client.meta.events
10
11
# attach an event handler to the client for
12
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
13
14
15
16
import boto3
lambda_client = boto3.client('lambda')
# our event handler
def add_xtrace_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
# get the event system for the lambda_client
event_system = lambda_client.meta.events
# attach an event handler to the client for
event_system.register('before-sign.lambda.Invoke', add_xtrace_header)
# invoke a lambda function
lambda_client.invoke(FunctionName='my-function')
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
loige 19
40. loige
You can be a 🌟too!
* (every event)
after-call.*.* (all responses)
after-call.lambda.* (all responses for lambda)
21
41. loige
OK, now we know enough to write
our own event handler for gzip
compression! 🤓
22
42. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
loige 23
43. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
44. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
45. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
46. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
47. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
request.headers.add_header('Content-Encoding', 'gzip')
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
48. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
request.headers.add_header('Content-Encoding', 'gzip')
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
loige 23
49. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
request.headers.add_header('Content-Encoding', 'gzip')
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
13
14
15
16
cw_client.put_metric_data(...)
17
loige 23
50. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
request.headers.add_header('Content-Encoding', 'gzip')
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
13
14
15
16
cw_client.put_metric_data(...)
17 cw_client.put_metric_data(...)
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
17
loige 23
51. import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
import gzip
import boto3
1
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
import boto3
1
import gzip
2
3
4
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
7
8
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
request.headers.add_header('Content-Encoding', 'gzip')
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
10
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
cw_client.put_metric_data(...)
17
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
13
14
15
16
cw_client.put_metric_data(...)
17 cw_client.put_metric_data(...)
import boto3
1
import gzip
2
3
cw_client = boto3.client('cloudwatch')
4
event_system = cw_client.meta.events
5
6
# Gzip handler
7
def gzip_request_body(request, **kwargs):
8
request.headers.add_header('Content-Encoding', 'gzip')
9
gzipped_body = gzip.compress(request.body)
10
request.data = gzipped_body
11
12
# registers the custom handler
13
event_system.register('before-sign.cloudwatch.PutMetricData',
14
gzip_request_body)
15
16
17
import boto3
import gzip
cw_client = boto3.client('cloudwatch')
event_system = cw_client.meta.events
# Gzip handler
def gzip_request_body(request, **kwargs):
request.headers.add_header('Content-Encoding', 'gzip')
gzipped_body = gzip.compress(request.body)
request.data = gzipped_body
# registers the custom handler
event_system.register('before-sign.cloudwatch.PutMetricData',
gzip_request_body)
cw_client.put_metric_data(...)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
loige 23
53. loige
Other potential use cases
Log data before/after it is sent to AWS
Add additional headers to AWS requests (tracing)
Additional input validation (your custom rules)
Payload enrichment (e.g. make sure certain tags exist)
25
54. loige
What if you don't use Python?
Customizing the AWS SDK for Go V2 Client Requests
Introducing Middleware Stack in Modular AWS SDK for
JavaScript
Handlers and Middleware in the AWS SDK for PHP Version 3
26
55. loige
What did we learn?
Always have alarms for your lambdas!
(have you checked already? 😉)
You can use Gzip compression to send big payloads to AWS
Boto3 is extensible through its event system!
SLIC Watch
27
56. loige
Do you think boto3 should make it easier to
send gzipped payloads?
👍UPVOTE this issue: github.com/boto/botocore/issues/2425
28
58. Cover picturea by on
Thanks doggie by Stefanie Shank
Moritz Mentges Unsplash
loige
nodejsdp.link
loige.link/gzip-boto3
30