Deep Dive on AWS IoT Core

AWS IoT Day
Hands-on Workshop: Deep Dive on AWS IoT Core
Gavin Adams, IoT Specialist Solutions Architect
Anton Shmagin, Partner Solutions Architect – IoT
May 3rd, 2018
© 2018 Amazon Web Services, Inc. and its affiliates. All rights served. May not be copied, modified,
or distributed in whole or in part without the express consent of Amazon Web Services, Inc.

Housekeeping
• Administrative access to:
– your laptop?
– AWS account (needed for both sessions)
– Credits will cover all exercises, as long as you clean up the
resources
• Our Commitment…

Session Agenda – AWS IoT Core
• 10:00a - 11:00a
– AWS Loft Introduction and Logistics
– Overview and Shadows
• 11:00a – 12:30p
– Labs 1 & 2 (Getting Started and Shadows)
• 12:30p – 1:30p
– Security and Rules Engine
• 1:30p – 3:00p
– Labs 3 & 4 (Security and Rules Engine)

Session Agenda – AWS Greengrass
• 3:00p – 6:30p
– AWS Loft Introduction and Logistics
– Overview of AWS Greengrass
– Lab Preparation Tips
– Greengrass Core Bootcamp

IoT solutions are complex & multidimensional
Connecting,
communicating,
securing
Devices &
sensors
Infrastructure
providers,
building blocks
Connectivity &
infrastructure
Incisive,
actionable,
predictive
Analytics
& insights
Engage,
empower,
delight
Applications
& services
Business
transformation,
cultural change
Change
management

Devices
Sense & Act
Cloud
Storage & Compute
Intelligence
Insights & Logic → Action
Three pillars of IoT

IoT with AWS
Enterprise
Applications
Enterprise
Users
Corp Apps
Amazon
QuickSight
Amazon
EMR
Amazon
Redshift
Amazon
S3
Machine
Learning
AWS
Lambda
All
AWS
IoT Partners
Edge
ARM, Broadcom, Digi,
Expressif, Intel, MediaTek,
Microchip, NXP, ST, TI,
Qualcomm, …
Gateway
Adlink Technology,
Advantech, MachineShop,
Samsung, Technicolor, …
ISV (Platform)
Ayala, Bright Wolf,
BSquare, C3IoT, Mnubo,
Salesforce, Splunk,
Thinglogix, …
Connectivity
Amdocs, Asavie, AT&T,
Eseye, Soracom, TATA
Communications, Telus,
Verizon, …
Consulting / SI
Accenture, Aricent,
Clearscale, CTP, Luxoft,
Mobiquity, Solstice,
Storm Reply, Sturdy
Networks, TCS, Trek10, …
Cloud
Device
Shadow
Rules
Engine
AWS IoT
Core
Certificate
Authority
AWS IoT
Device
Management
AWS
IoT
Users
Over-The-Air
(OTA)
Updates
Analytics
Data Store
Data
Pipelines
Templated
Reports
Batch Fleet
Provisioning
Real-Time
Fleet Index &
Search
AWS IoT
Device
Defender
Ad-hoc & In-
depth Analysis
Risk
Mitigation
Monitor
Device
Behavior
Alerts
Message
Broker
Audit Device
Configurations
Amazon
Kinesis
AWS IoT
Analytics
AWS IoT 1-Click
MQTT
MQTT
Endpoints Gateway/PLC
Device
Shadow
Lambda
Functions
Local Comms Long-range Comms
Amazon
FreeRTOS
Certificate
Authority
Local
Resources
IoT SDK
OPC-UA
MQTT
Edge
Users
Cert
WiFi
MQTT
Edge
OTA
OTA
Amazon
FreeRTOS
Integrated
Client
Snowball
Edge
AWS Greengrass
MQT
T
AWS Greengrass
Message
Broker
ʥ
A
Protocol
Adapter

IoT with AWS
Enterprise
Applications
Amazon
QuickSight
Amazon
EMR
Amazon
Redshift
Amazon
S3
Machine
Learning
AWS
Lambda
All
AWS
Cloud
Device
Shadow
Rules
Engine
AWS IoT
Core
Certificate
Authority
AWS
IoT
Users
Message
Broker
Endpoints
IoT SDK
OPC-UA
MQTT
Edge

AWS IoT Core
All in one service
• Message Broker
• Rules Engine
• Certificate Authority
• Shadow
• Unbundles pricing by charging for these
components independently
Managed service
• No installation
• Automatic scaling
• No pre-provisioning
• Redundant across AZ
• Pay as you go
Device
shadow
Rules
Engine
AWS IoT
Core
Certificate
Authority
Message
Broker

Overview
• AWS IoT Core capabilities and related services, including:
Authentication and Authorization
Devices & Device Shadows
Message Broker
Rules Engine
Other AWS Services
Applications & API
Corp Apps

1
Authentication & Authorization
(brief)

Authentication and Authorization
Security is Job Zero
• Mandatory authentication
• Device policies
• IAM fine-grained access controls
• Auditing and logging
Authentication
• TLS 1.2 with X.509 certificates
• HTTP/SigV4
• IAM Service Roles
Authorization
• Device+Certificate+AWS IoT Policy
• Cognito User+AWS IoT Policy
• IAM Policy/Roles

AWS IoT Authentication
• X.509 certificates for devices
– TLS 1.2, SHA-256 RSA (or ECC), supported cipher suite
• IAM users, groups, and roles
– TLS 1.0+, SHA-256 RSA certificate validation, supported cipher
suite
• Amazon Cognito identities
• Federated identities

AWS IoT Authorization
• AWS IoT Data Plane
– Client certificate or Cognito identity associated with an AWS IoT
Policy
– SigV4 with credentials associated with an IAM policy
• API Calls
– SigV4 with credentials associated with an IAM policy
– Service roles allowing AWS IoT to access other AWS services

Authentication/Authorization Examples
AWS IoT
Device
Credentials
Establish TLS 1.2 Connection, request server certificate
Sign connection with server certificate, request client certificate
Validate server certificate, sign response with client certificate
Connection authenticated, AWS IoT policy
associated to client certificate applied
Username: alice
Password: redQueen!
Establish HTTPS Connection, request server certificate
Sign connection with server certificate, wait for message (REST API)
Validate server certificate, sign response with credentials (Cognito or IAM/STS)
Connection authenticated, IAM policy associated
with access key/secret key used, or AWS IoT
policy for Cognito identities
Credentials
Note: MQTT and HTTP can use cert or SigV4 on auth mechanism

Device Gateway
Based on MQTT 3.1.1.1
• Native MQTT, MQTT+Websockets, HTTP
• QoS 0 & 1
• Single clientId connection
Integration
• Services use native format
• Policy defines access
• Last Will & Testament
• Reserved topics ($aws/#)
• Lifecycle events
Message Format
• (Nested) JSON
• Binary

Topics
• Ephemeral
• Publish/Subscribe
– Devices Publish to individual topics
– Devices Subscribe to one or topics and hierarchies
– Published messages and subscribed responses are metered for billing
• Wildcards
– Single level (+)
• myhome/groundfloor/+/temperature
• Returns temperature messages for all groundfloor things
• Only between topic levels
– Multi-level (#)
• myhome/groundfloor/#
• Returns all messages for all groundfloor things and subtopics

Topic Variables (Fan-in Example)
home/ac/AAA/temperature
home/ac/BBB/temperature
home/ac/CCC/temperature
home/ac/DDD/temperature
Device: AAA
Device: BBB
Device: CCC
Device: DDD
PUB: home/ac/clientId/temperature SUB: home/ac/+/temperature

Messages and Pricing
• $1 per million messages, 5,120 byte size
• Device connectivity $0.08/million minutes, PING
messages are not billed at >= 30 seconds
• Rules Engine $0.15/million invocations, 5K
message size
• Device Shadow/Registry Updates $1.25/million
updates, 1K size
• Message can be binary, but the Rules Engine can
only act on JSON payload

Rules Engine
Tasks
• SQL-like syntax to write rules
• Augment or filter data
• Save data to other services
• Send data to Amazon Machine Learning
• Make predictions based on ML model
Services Supported
• Amazon DynamoDB
• Amazon S3
• Amazon SNS
• Amazon SQS
• Amazon Kinesis
• Amazon Elasticsearch
• AWS Lambda
• and more...

Rules Engine
• SQL-like query language
– SELECT * FROM 'topic/structure' WHERE temperature > 35
• Actions
– Send message to other services
– Score results against machine learning
– Republish message or modifications to other topics

Developers
Application Development
• AWS IoT SDK
• AWS SDK’s
• Authentication & Authorization
• Cross account access
• Lifecycle events
• Monitoring
• Troubleshooting
Corporate
Applications

Endpoints
Cloud
Device
shadow
Rules
Engine
AWS IoT
Core
Certificate
Authority
IoT SDK
AWS
Amazon
QuickSight
Amazon
EMR
Amazon
Redshift
Amazon
S3
Machine
Learning
MQTT
IoT Users
AWS
Lambda
All
AWS
IoT with AWS
Things
Message
Broker

Example – Sending a Command
Device
Applications

Example – Sending a Command
Device
Shadow
Applications

1. Device publishes current state
2. Persist to JSON data store
3. App requests device’s current state
4. App requests change the state
5. Device shadow syncs
updated state
6. Device publishes current state 7. Device shadow confirms state change
AWS IoT Core Device Shadow Flow

AWS IoT Core Device Shadow
{
"state" : {
"desired" : {
"lights": { "color": "RED" },
"engine" : "ON"
},
"reported" : {
"lights" : { "color": "GREEN" },
"engine" : "ON"
},
"delta" : {
"lights" : { "color": "RED" }
}
},
"version" : 10,
"timestamp" : 28034023492,
"clientToken": "UniqueClientToken"
}
Device
Report its current state to one or multiple shadows
Retrieve its desired state from shadow
Mobile app
Set the desired state of a device
Get the last reported state of the device
Delete the shadow
Shadow
Shadow reports delta, desired and reported
states along with metadata and version

AWS IoT Core Shadow Delta
Sensor Reported Desired Delta
LED1 RED YELLOW
LED1 =
Yellow
TEMP = 60F
ACCEL X=1,Y=5,Z=4 X=1,Y=5,Z=4
TEMP 83F 60F

Building Blocks of the AWS IoT Core Device Shadow
Device Shadow Topics
Device Shadow State
Device Shadow Metadata

AWS IoT Core Device Shadow Topics (MQTT)
UPDATE: $aws/things/{thingName}/shadow/update
GET: $aws/things/{thingName}/shadow/get
DELETE: $aws/things/{thingName}/shadow/delete
DELTA: $aws/things/{thingName}/shadow/update/delta
DOCUMENTS: $aws/things/{thingName}/shadow/update/documents

UPDATE Shadow Topics (MQTT)
PUBLISH : $aws/things/{thingName}/shadow/update
{
"state": {
"desired" : {
”speed" : 65,
"engine" : "ON"
}
}
}

GET Shadow Topics (MQTT)
PUBLISH : $aws/things/{thingName}/shadow/get
SUBSCRIBE : $aws/things/{thingName}/shadow/get/accepted
{
"state": {
"reported": {
"lights": {
"color": "GREEN"
} },
"metadata": {
"reported": {
"lights": {
"color": { "timestamp": 789012 }
}
}
}, "version": 10, "timestamp": 123456789
}

DELETE Shadow Topics (MQTT)
PUBLISH: $aws/things/{thingName}/shadow/delete

DELTA Shadow Topics (MQTT)
AWS IoT Publish: $aws/things/{thingName}/shadow/update/delta
{
"state": {
"desired": { "color": "RED” },
"reported": { "color": "GREEN” },
"delta": { "color": "RED"}
…
}
}

8
Device Shadow
Considerations

Device Shadow Considerations
• Max Device Shadow size is 8KB
• AWS Shadow Data Types:
– String
– Number
– Boolean
– Null
– JSON object
– Array

Devices & Shadows
Devices are Constrained
• Limited resources (CPU, RAM, etc.)
• Fixed hardware capabilities
• Intermittent connectivity
Markets
• Consumer
• Embedded
• Industrial/Utility
• Agriculture
Shadows
• States: Reported, Desired, Delta, Timestamp
• Available all the time

Typical Device Characteristics
• One or more sensors
• Telemetry and/or actuation
• Firmware with connectivity
• Communicates with
defined message format
• Can operate without
connection to IoT services 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
1 0 1 0 1 0 1 0 1 0 1 0 1 1 0
1 0 1 0 1 0 0 1 1 0 1 0 1 0 1
0 1 1 0 1 0 1 0 1 0 1 1 0 1 0
1 0 0 1 0 0 1 0

Device Shadows
• Publishes reported
state
• Listens for updates
(acts on desired state)
• Tracks reported and
desired states by
timestamp and versions
• Accessible via API or
topics
• Reads reported
state
• Publishes new
values (becomes
desired state)
Topics: $aws/things/myDevice/shadow/...

Device Shadows
reported: what current
color?
desired:(device not connected)
reported:
1
2
4 3
5

Labs
• Lab guides at: http://loft.baah.io
• Virtual Things
• Node-RED (https://nodered.org)
• Created via CloudFormation
– Do not delete stack until end of first workshop, it’s used for other
modules

Node-RED Environment
Created during first lab
virtual private
cloud
Amazon
EC2
Complete Lab
Your Laptop
Root
certificate
IoT
certificate
IoT Private
Key
AWS IoT

Lab Errata
Console Changes – Icons the same, names have changed

Labs (Continued)
• Workshop Labs
– 1-Getting Started
– 2-Shadows
If you need help, please ask any of the AWS
staff supporting the workshop

One-stop-shop for Information
http://loft.baah.io

AWS IoT Core Workshop – Part 2
• Security!
• Rules Engine
• Labs 3 & 4

10
AWS Security Overview
(before there was AWS IoT there was AWS)

AWS Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure
Regions
Availability Zones
Edge Locations
Network
Security
Identity &
Access
Control
Customer applications & content
You get to define
your controls IN
the Cloud
AWS takes care
of the security
OF the Cloud
You
AWS and You Share Responsibility for Security
Inventory &
Config
Data
Encryption

AWS Identity and Access Management (IAM)
• Enables you to control who can do what in your AWS account
• Users, groups, roles, and permissions
• Control
– Centralized
– Fine-grained - APIs, resources, and AWS Management Console
• Security
– Secure (deny) by default
– Multiple users, individual security credentials and permissions

• {
• "Statement":[{
• "Effect":"effect",
• "Principal":"principal",
• "Action":"action",
• "Resource":"arn",
• "Condition":{
• "condition":{
• "key":"value" }
• }
• }
• ]
• }
JSON-formatted documents
Contain a statement (permissions)
that specifies:
• Which actions a principal can
perform
• Which resources can be accessed
Principal
Action
Resource
Condition
You can have multiple statements and
each statement is comprised of PARC.
IAM Policy specification basics

Principal – Examples
•
• An entity that is allowed or denied access to a resource
• Indicated by an Amazon Resource Name (ARN)
• With IAM policies, the principal element is implicit (i.e., the user, group, or role attached)

"Principal":"AWS":"*.*"

"Principal":{"AWS":"arn:aws:iam::123456789012:root" }
"Principal":{"AWS":"123456789012"}

"Principal":"AWS":"arn:aws:iam::123456789012:user/username"

"Principal":{"AWS":"arn:aws:iam::123456789012:role/rolename"}
Principal
Action
Resource
Condition

Action – Examples
•
• Describes the type of access that should be allowed or denied
• You can find actions in the docs or use the policy editor to get a drop-down list
• Statements must include either an Action or NotAction element

"Action":"iam:ChangePassword"
<!– Amazon S3 action -->
"Action":"s3:GetObject"

"Action":["sqs:SendMessage","sqs:ReceiveMessage"]
Principal
Action
Resource
Condition

Endpoints
Cloud
Rules
Engine
Certificate Authority
IoT SDK
MQTT
IoT Users
Things
Message Broker

Cloud
Rules
Engine
AWS
Amazon RedshiftAmazon
IoT Users
IoT with AWS
Message Broker

Device shadow
Rules
Engine
AWS IoT
Core
Certificate Authority
Long-range Comms
Amazon
QuickSight
Amazon
EMR
Amazon RedshiftAmazon
S3
Machine Learning
AWS Lambda
All
AWS

AWS IoT: Securely Connect Devices

TLS mutual authentication
• Create CSR
• Create X.509 certificate from CSR
• Activate the certificate
• Create policy
• Attach policy to certificate

Certificates and keys
• Private key (authenticate the device)
• Certificate (register the device with IoT)
• Root certificate authority (authenticate IoT)

AWS IoT Permissions
• Control what a thing is allowed to do
• Connect, publish, subscribe, receive
• Attach policy to certificates

AWS IoT Policies
{
"Effect": "Allow",
"Action": "iot:Publish",
"Resource": [
"arn:*:topic/private-topic/${iot:ClientId}",
"arn:*:topic/open-topic-space/*"
]
},
{
"Effect": "Allow",
"Action": "iot:Subscribe",
"Resource": "arn:*:topicfilter/private-topic/${iot:ClientId}/*"
}

Best Practice for Securing Devices
• Each device should use a unique private key and
certificate
• An IoT Policy should follow least privilege for
permissions

Intermediate certificate authority
locally provisioned
async registration

Just-in-time registration
AWS
Lambda

Creating the trust relationship with AWS IoT
P P
P
Role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": “iot.amazonaws.com”
},
"Action": "sts:AssumeRole"
}
]
}

Securing user access
• WebSocket support SigV4 authentication
• Use AssumeRole with IAM
• Use IoT policies with Cognito
• Amazon Cognito identity pools
– Anonymous access to iot:Subscribe
– Authenticated Cognito for fine grained permissions and IoT
Policies
– Use your own application-level authentication patterns

Cognito User and Federated Identities
Cognito User
Identities
(Your User Pool)
User
Sign-in1
Returns Access
and ID Tokens
2
Cognito Federated
Identities
(Identity Pool)
Get AWS scoped
credentials
3
Access
to AWS Services
4
AWS IoT IoT Policy

Example – Sending Event to AWS Services
Lambda
function
Amazon Kinesis
Firehose
payload:
{
"temp": 33,
"wind": 1.02
}
SELECT *, timestamp() as timestamp
FROM 'pws/#'
WHERE temp > 30
topic: pws/station123

Example – Sending Event to AWS Services
Lambda
function
Amazon Kinesis
Firehose
payload:
{
"temp": 33,
"wind": 1.02
}
SELECT *, timestamp() as timestamp
FROM 'pws/#'
WHERE temp > 30
topic: pws/station123
"context": {...},
"event":
{
"temp": 33,
"wind": 1.02,
"timestamp": 1000209900
}
"deliveryStreamName":
{...},
"msgNNN":
{
"temp": 33,
"wind": 1.02,
"timestamp": 1000209900
}

pws/
station1/
temp
wind
station2/
temp
wind
temp_status
$aws/
things/
station1/...
station2/...
Topics Rules Actions
{
"sql": "SELECT * FROM 'pws/station1/#'",
"actions": [
{
"s3": {
"bucketName": "myBucket",
"key": "myS3Key"
}
}]
}
{
"sql": "SELECT * FROM 'pws/#'",
"actions": [
{
"elasticsearch": {
"endpoint": "http://my-endpoint",
"index": "my-index",
"id": "${newuuid()}"
}
}]
}
{
"sql": "SELECT * FROM 'pws/+/temp'",
"actions": [
{
"republish": {
"topic": "pws/temp_status"
}
}]
}
Amazon ES
Amazon
S3
AWS IoT

Breakdown of a Rule (JSON)
{
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"key": "myS3Key"
},
"republish": {
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role",
"topic": "historical/pws"
}]
}

Breakdown of a Rule (JSON)
{
"actions": [
{
"s3": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"key": "myS3Key"
},
"republish": {
"roleArn": "arn:aws:iam::123456789012:role/my-iot-role",
"topic": "historical/pws"
}]
}
SQL
ACTIONS
ROLES
ACTION SPECIFIC PARAMETERS

Format of an AWS IoT SQL Statement
• SELECT – What values to include for Action
• FROM – What topic structure to act upon
• WHERE – Logic to determine if the statement
should execute
SELECT *, timestamp() as timestamp FROM 'pws/#' WHERE temp > 30

SQL Example 1
SELECT *, newuuid() as uniqueId FROM 'a/b'
• Process messages in the a/b topic,
• Select entire message and create new attribute and
value,
• Action: Write object to S3, where key is ${uniqueId}

SQL Example 2
SELECT * FROM 'factory/+/pump_status'
WHERE machinelearning_predict( 'vibration-model',
'arn:aws:iam::123456789012:role/my-iot-aml-role',
*).predictedLabel=1
• Action: Republish to topic: factory/maint_required

SQL Example 3
Incoming Payload:
{
"sensor":
{
"temp": 78.2,
"humid": 42.5
},
"bat_stat": "ok"
}
SELECT (sensor.temp – 32) * 5/9 as celsius, sensor.humid as humid,
upper(bat_stat) as battery, timestamp() as timestamp
FROM 'a/b'
• Action: Send to Elasticsearch indexed on timestamp key/value

Support Actions for Messages
• cloudwatchAlarm to change a CloudWatch alarm
• cloudwatchMetric to capture a CloudWatch metric
• dynamoDB to write data to a DynamoDB database
• dynamoDBv2 to write data to a DynamoDB database
• elasticsearch to write data to a Amazon Elasticsearch Service domain
• firehose to write data to an Amazon Kinesis Firehose stream
• kinesis to write data to a Kinesis stream
• lambda to invoke a Lambda function
• s3 to write data to a Amazon S3 bucket
• sns to write data as a push notification
• sqs to write data to an SQS queue
• republish to republish the message on another MQTT topic
• salesforce to write a message to a Salesforce IoT Cloud Input Stream
• New: Call Lambda function in SQL SELECT or WHERE clauses to enrich data

Understanding Action Components
• Creating an Action - Permissions
– iam:PassRole on your account to pass a role to the rules engine
– IAM Role with permissions required on target service
– For Lambda, addition of permissions on the resource-based
policy (iam:PassRole not required)
• Service Unique Parameters
– E.g., S3 bucket and key; Kinesis stream and partition key
• Service Payload Access/Timing

Action Examples
• Persist Data to S3:
– bucket: S3 bucket to which to write data - mybucket
– cannedacl: Canned ACL for created objects –
bucket-owner-full-control
– key: path to the object where data is written –
${timestamp()-foo}
• Stream Data to Kinesis:
– stream: Kinesis stream to which to write data – my_stream
– partitionKey: Used to determine which shard to write data -
${newuuid()}
• Republish
– watch out for recursive calls, infinite loops

Action Examples (continued)
• Process via Lambda
– Grant lambda:Invoke to source ARN of topic rule
– Lambda event object contains SELECT results
• Republish
– topic: AWS IoT topic to republish the message - foo/bar
– Watch out for recursive calls, infinite loops
– Republish of messages is metered for billing purposes

Rules Engine
Considerations
18

Rules Engine Considerations
• Consider Ordering and Overlap of Rules
• Enable CloudWatch Logs for Debugging
– Authorization
– WHERE clause matching
• Understand Deployment Considerations
– Updates effective immediately

Rules Engine Summary
• SELECTs messages FROM topic(s) and delivers
them to other AWS services
• Can transform or create new values
• Multiple actions can be associated with a rule
• Rule creation or modification takes effect
immediately, unless disabled
• IAM roles (or Lambda granted permissions)
required to access or invoke other services

Labs
• Workshop Labs
– 3-Security
– 4-Rules Engine
If you need help, please ask any of the AWS
staff supporting the workshop

Thank you for your time today!
Any questions?

Deep Dive on AWS IoT Core

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Deep Dive on AWS IoT Core

Similar to Deep Dive on AWS IoT Core (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Deep Dive on AWS IoT Core