The introduction of low-cost, high-performance embedded processors coupled with improvements in Neural Network model optimization lay the foundation for AI and Computer Vision at the edge. Moving intelligence from the cloud to the edge offers many advantages including the reduction of network traffic, predicable ML inference times, and data security to name a few. Challenges exist as many development teams do not have data scientist or AI development engineers. What is needed are practical AI solutions including ML development tools, optimized inference engines and reference platforms that will abstract out the development complexities to stream line prototyping and development. In this joint webinar with Au-Zone Technologies we will discuss: - Development challenges and solutions which can be use to enable AI/ML at the edge to implement object detection, classification and tracking for medical and industrial use-cases - Visualization techniques for activity monitoring and object detection

1. Integrated Computer Solutions Inc.
Leveraging Artificial Intelligence
Processing on Edge Devices
Andrew Caples, Au-Zone Technologies
Justin Noel, ICS

2. Integrated Computer Solutions Inc.
AI and ML
Artificial Intelligence
Machine Learning
Deep Learning
1950’s 1980’s 2010’s1970’s1960’s 1990’s

3. Integrated Computer Solutions Inc.
Au-Zone Overview
3
Au-Zone specializes in the enablement of Computer Vision with AI / ML Intelligence for IoT and
Edge devices. Au-Zone’s product portfolio includes industry leading AI / ML software solutions
including Machine Learning development tools and inference engines, vision development kits,
vision and image processing software, and design services.
● ● ● ●
● ● ● ●
● ● ●

4. Integrated Computer Solutions Inc.
Why Machine Learning Now?
1. Data Availability
- ImageNet http://www.image-net.org/
- Millions of labeled images
- Microsoft COCO (Common Objects in Context)
- COCO is a large scale object detection, segmentation and
captioning dataset.
2. Compute Power
- Powerful parallel computing processes in the cloud for model
training and edge for inferencing
3. Advances in Machine Learning
Iconic
image
Non Iconic
image
4

5. Integrated Computer Solutions Inc.
Machine Learning Proliferation
Automotive
Medical
Industrial
5

6. Integrated Computer Solutions Inc.
Functional Safety?
As defined by IEC:
“Freedom from unacceptable risk of physical injury or of damage to the health of people, either
directly, or indirectly as a result of damage to property or the environment.”
• From IEC61508:
The part of the overall safety that depends on a system or equipment operating correctly
in response to its inputs.
• From ISO26262
Absence of unacceptable risk due to hazards caused by mal-functional behavior of
electrical and/or electronic systems
Source:
IEC Website: https://www.iec.ch/functionalsafety/explained/
Safety involves protecting the world from the device
6

7. Integrated Computer Solutions Inc.
Safety Relationships
IEC 61508
Base Functional Safety Specification
(Industrial)
IEC 62304
Adaption of 61508
for Medical Devices
EN50128
Adaption of 61508
for railway
ISO 26262
Adaption of 61508
for Automotive
Electronics
ISO 25119
Adaption of ISO
26262 for Tractors
7

8. Integrated Computer Solutions Inc.
Safety Mappings
Safety
Standard
IEC 61508
Industrial
IEC 62304
Medical
ISO 26262
Automotive
SIL 4
SIL 3 Class C ASIL D
SIL 2 Class B ASIL B/C
SIL 1 Class A ASIL A
SafetyLevelHigher
Lower
8

9. Integrated Computer Solutions Inc.
Safety Lifecycle for Software Development
Hazard and Risk Analysis
• Perform a Hazard Analysis
• Potential cause of
harm
• Access the risks
associated with each
Hazard
• Probability
• Severity
Hazard Analysis / Risk
assessment
Specification
of Safety goals
Specification of functional
safety requirements
Specification of technical
safety requirements
Specification of software
safety requirements
Architectural Design
Unit Design and
implementation
Specification of software
safety requirements
Specification of software
safety requirements
Unit Testing
9

10. Integrated Computer Solutions Inc.
Acceptable Risk?
The goal is to reduce risk to acceptable levels
Hazard Matrix
ProbabilityofHazard
Severity of Hazard
Acceptable?
Acceptable?
10

11. Integrated Computer Solutions Inc.
Safety Lifecycle for Software Development
Software Design Test
• Classify the Hazards using
Industry Specific
Methodology
• Industrial: SIL 1 – 3
• Medical: Class A - C
• Define Safety Requirement
to Mitigate the risks
Hazard Analysis / Risk
assessment
Specification
of Safety goals
Specification of functional
safety requirements
Specification of technical
safety requirements
Specification of software
safety requirements
Architectural Design
Unit Design and
implementation
Specification of software
safety requirements
Specification of software
safety requirements
Unit Testing
11

12. Integrated Computer Solutions Inc.
Machine Learning Obstacles in Functional Safety
A) Specification
• Lack of a Specification is an obstacle to safety assurance as
the assumption is given the left side of the V model, the safety
requirements of the component are completely satisfied
B) Interpretability
• ML models are difficult to interpret
• Interpretability is an obstacle to safety as it prevents use of
white box verification methods such as inspection or other
activities such as static analysis
Other Considerations
C) Error Rate
- Models do not operate perfectly and exhibit some error rate
D) Training
- the training set may not represent all possible inputs
- Other issues such as overfit
Need to show traceability
from the Hazard to
Software and Testing as
evidence of Risk mitigation.
12

13. Integrated Computer Solutions Inc.
Implementation of the Safety Requirement
A) Can the Safety Requirement be completely specified?
• A traditional programming approach to implementing the
requirement is likely and should be taken.
• Detect all objects within 10 feet of CT Scanner
• Implementation can use a Time of Flight sensor to detect
with programming control
B) Split the Safety Requirement into multiple components
• Detect all objects within 10 feet of CT Scanner
• Detect all humans within 10 feet of CT Scanner
• Creates a Programmable requirement and Machine
Learning requirement
• Conservative requirement (object detection) can
strengthen the Safety Case since detecting all objects
includes humans.
Safety
Requirement
Machine
Learning
Requirement
Programmable
Requirement
13

14. Integrated Computer Solutions Inc.
Machine Learning Specification
• Data Set
• The ML specification can help to better define quality data sets for training
• Model
• The ML specification can be used to select the appropriate ML model for the problem or
use-case
• Performance
• The ML specification can be used to ensure the model trained will conform to the
specification
• Verification
• After the model is trained, it can be verified against the ML specification
14

15. Integrated Computer Solutions Inc.
Machine Learning Challenges for Developers
Customer
Use Cases
1000’s
Process
or
options
100’s
Training
Frameworks
& Model
Convertors
5 - 10
Datatypes,
Datasets
& Public Models
1000’s
Time Series
Data
Image
Data
Video
Data
✔ Detection
✔ Classification
✔ Recognition
✔ Events
✔ Actions
✔ Gestures
✔ Vibration
✔ Acoustics
✔ Temp etc.
15

16. Integrated Computer Solutions Inc.
Edge Device Challenges
When adapting public models or deploying custom models to edge devices and computer vision
IoT edge devices, development teams are often faced by several challenges:
1. Model Performance
2. Model Memory Requirements
3. Model Portability
4. Ease of use
16

17. Integrated Computer Solutions Inc.
DeepView Workflow
Model
Conversion
17

18. Integrated Computer Solutions Inc.
DeepViewRT Baseline Engine Size
• TensorFlow Lite 2.0 beta (2019-08) – publicly available
• All measurements are resident set size (RSS).
* Estimated size by dividing weights by 2 (FP16) and by 4 (int8)
DeepViewRT engine
> 1/10 size of alternatives
18

19. Integrated Computer Solutions Inc.
DeepViewRT Footprint Example
• TensorFlow Lite 2.0 beta (2019-08) – publicly available
• All measurements are resident set size (RSS)
• Other == heap, stack, global memory, library dependencies etc. * Projected sizes based on calculations of FP32 buffers to FP16 and INT8
1/2
3/5
19

20. Integrated Computer Solutions Inc.
DeepViewML Toolkit Highlights
Bring Your
Own Data
Bring Your
Own Model
Tuning and
Optimization
Transfer Learning Pretrained Model ImportGraphical UX
20

21. Integrated Computer Solutions Inc.
DeepViewML Workflow: Bring Your Own Data
Trained / Optimized
Model
Optimize for Accuracy /
Performance
Classify or DetectUser Data
Runtime Profiling
21

22. Integrated Computer Solutions Inc.
DeepViewML Workflow: Bring Your Own Model
ConvertPublic or Custom Model Runtime Profiling
22

23. Integrated Computer Solutions Inc.
IoT Vision Sensor Family
Family of Vision Sensors enabling OEM’s to integrate visual intelligence into their products
Applications include:
• Industry
• Healthcare
• Agriculture
• Smart Home/City
• Consumer
• Logistics
• Retail
Wireless (5G & WiFi) Wired
23

24. Integrated Computer Solutions Inc.
IoT Vision Sensor Family
• Purpose built for Intelligent IoT Computer Vision Sensor with Au-Zone Perception Engine
• Reference Software available
• Reference Hardware available
24

25. Integrated Computer Solutions Inc.
ICS Experience In Machine Vision
● Campus wide surveillance systems
● Manufacturing QA Systems
● Automated Agriculture
25

26. Integrated Computer Solutions Inc.
Lots of CPU and/or Cloud Processing
26

27. Integrated Computer Solutions Inc.
Au-Zone’s Solution is Local and Efficient!
● DeepViewRT ML Engine
● Runs on Embedded HW - NXP imx6 / imx6
● Runs on Microcontrollers - NXP RT1050
● Runs along side your UI and/or controls system
● Using regular Qt Embedded or Qt for MCUs
● Au-Zone even offer a QML Toolkit for their DeepViewRT ML Engine
27

28. Integrated Computer Solutions Inc.
DeepView ML QML Components and Examples
● Single Shot Detection (SSD) Camera
● Detect multiple objects in a single image
● Image Classification
● Load public models, transfer learn custom models or execute custom,
proprietary model
● PoseNet and Gesture
● Detect and overlay an outline of a person or persons’ joints and limbs onto a video
feed using a PoseNet model
28

29. Integrated Computer Solutions Inc.
Why Is Local Better?
● Privacy
● No frames transferred outside of the controlled system.
● Bandwidth / Connectivity
● No high speed internet connection required.
● Avoid Fees and Infrastructure Costs
● Cloud services love to bill coming and going.
29

30. Integrated Computer Solutions Inc.
Standalone IoT Vision Sensor
30

31. Integrated Computer Solutions Inc.
Why is Standalone Better?
● Leverage Au-Zone’s optimization of ML on dedicated microprocessor.
● “Do one thing and do it well.”
● Save your precious CPU cycles for your own real time processing.
● Bandwidth
● Only needs local bandwidth to transmit image metadata.
● Does the condition I’m looking for exist? What % certainty?
● Scalability
● Add more cameras with more views or conditions without impacting your
systems performance.
31

32. Integrated Computer Solutions Inc.
Use Case Examples
Medical / Industrial Safety
● Where are the people?
● In the X-Ray Room?
● Within the danger zone of robotic arms?
● Is the correct protective equipment being worn?
32

33. Integrated Computer Solutions Inc.
Use Case Examples
Agriculture
● Grade and sort crops as they are being picked.
● What constitutes a Grade A tomato?
● Train a model and load it using DeepViewRT ML Engine.
Automated Farming
● Safety - What is in the path of the vehicle?
● Inspection - Fly drones over the field and inspect the crops.
● Record the location of anything interesting.
33

34. Integrated Computer Solutions Inc.
Use Case Examples
Machine / Process Calibration
● Medical
● Is the patent positioned property before a CT/PET/MRI Scan?
● Has the table been “zeroed in” for the patient and type of scan?
● Industrial
● Are there calibration or wear checks that need to performed by a human?
● ML can provide a critical safety net or even perform better than humans.
34

35. Integrated Computer Solutions Inc.
Thank you!
Questions?
35