Deep dive into self driving car computer

1. April 4-7, 2016 | Silicon Valley
Shri Sundaram, April 4, 2016
DRIVE PX 2
SELF DRIVING CAR COMPUTER

2. 2
SELF DRIVING AND AI SUPERCOMPUTING

3. 3
SELF-DRIVING CAR PLATFORM
NVIDIA DRIVE PX 2NVIDIA DIGITS
NVIDIA DRIVENET
Localization
Planning
Visualization
Perception
DRIVEWORKS

4. 4
NVIDIA DRIVE PX 2
12 CPU cores | Pascal GPU | 8 TFLOPS | 24 DL TOPS | 16nm FF | 250W | Liquid Cooled
World’s First AI Supercomputer for Self-Driving Cars

5. 5
INTERFACES
Sensor Fusion Interfaces
GMSL Camera, CAN, GbE, BroadR-Reach, FlexRay,
LIN, GPIO
Displays and Cockpit Computer Interfaces
HDMI, FPDLink III and GMSL
Development and Debug Interfaces
HDMI, GbE, 10GbE, USB3, USB 2 (UART/debug),
JTAG
70 Gigabits per second of IO
Auto Grade connectors Debug/Lab interfaces

6. 6
COMPUTATION ENGINES
•Scalable + Redundancy Capable
Dual CPU-GPU Cluster
Connected over
Gigabit Ethernet
•Independent and specialized
compute
Each GPU has
dedicated memory
•Greatly accelerate DNN
performance
Specialized
Instructions for
Discrete GPU
24 DL TOPS, 8 TFLOPS,
High Performance CPU Complex
DRIVE PX 2

7. 7
SOFTWARE
NVIDIA Vibrante Linux & Comprehensive BSP
Rich Middleware
SDK, Samples and more
A full stack of rich software components

8. 8
DRIVE PX 2
Designed For
Sensor Fusion, Point Cloud etc.
SENSOR PROCESSING
Detection/Perception (DNN Inference),
Localization, Path Planning, Visualization
COMPUTATION
Automobile Network
COMMUNICATION
Algorithm development, Hardware in
Loop etc.
DATA COLLECTION

9. 9
READYING FOR
PRODUCTION
RAPID
PROTOTYPING
DEVELOPING
APPLICATIONS
DRIVE PX 2
Built for
Develop applications using
DRIVE PX 2
Deploy a pre-trained DNN
on DRIVE PX 2
Migrate applications from
PC to DRIVE PX 2
Typically in a lab set up or
a few cars
DRIVE PX 2 as reference
for custom ECU
Safety concept to derive
certain Safety Integrity
Level
Validate algorithms in car
with live cameras/sensors
Partition existing system
(say, PC); include DRIVE
PX 2 for sensor fusion
10s of cars

10. 10
READYING FOR
PRODUCTION
RAPID
PROTOTYPING
DEVELOPING
APPLICATIONS
DRIVE PX 2
Built for
Develop applications using
DRIVE PX 2
Deploy a pre-trained DNN
on DRIVE PX 2
Migrate applications from
PC to DRIVE PX 2
Typically in a lab set up or
a few cars
DRIVE PX 2 as reference
for custom ECU
Safety concept to derive
certain Safety Integrity
Level
Validate algorithms in car
with live cameras/sensors
Partition existing system
(say, PC); include DRIVE
PX 2 for sensor fusion
10s of cars

11. 11
READYING FOR
PRODUCTION
RAPID
PROTOTYPING
DEVELOPING
APPLICATIONS
DRIVE PX 2
Built for
Develop applications using
DRIVE PX 2
Deploy a pre-trained DNN
on DRIVE PX 2
Migrate applications from
PC to DRIVE PX 2
Typically in a lab set up or
a few cars
DRIVE PX 2 as reference
for custom ECU
Safety concept to derive
certain Safety Integrity
Level
Validate algorithms in car
with live cameras/sensors
Partition existing system
(say, PC); include DRIVE
PX 2 for sensor fusion
10s of cars

12. 12
READYING FOR
PRODUCTION
RAPID
PROTOTYPING
DEVELOPING
APPLICATIONS
DRIVE PX 2
Built for
Develop applications using
DRIVE PX 2
Deploy a pre-trained DNN
on DRIVE PX 2
Migrate applications from
PC to DRIVE PX 2
Typically in a lab set up or
a few cars
DRIVE PX 2 as reference
for custom ECU
Safety concept to derive
certain Safety Integrity
Level
Validate algorithms in car
with live cameras/sensors
Partition existing system
(say, PC); include DRIVE
PX 2 for sensor fusion
10s of cars

13. 13
SPRINGBOARDS
TO PRODUCTION
TOOLS &
ECOSYSTEM
CORE DRIVE PX 2
CAPABILITIES
DRIVE PX 2
Benefits
Computation &
Memory
Interfaces & IO
Bandwidth
Rich SW & HW
Capabilities Functional
Safety
Automotive
HW
Automotive SW
(RTOS, AUTOSAR &
Code Compliance)
Product
Partnerships
(HW & SW)

14. 14
COMPUTATION & MEMORY
CPU Complex: 2x Denver2 plus 4x Cortex-A57
Fully coherent HMP system; ARM V8 64-bit
Pascal GPU – 5th Gen GPU Architecture
Custom acceleration for deep learning
Separate memory for Tegra (CPU + iGPU) and dGPU
Tegra (CPU + iGPU) to 8GB LPDDR4 (UMA): 50+GB/s
Discrete GPU to 4GB GDDR5: 80+ GB/s
24 DL TOPS from GPU; High Perf. CPU
CORE DRIVE PX 2 CAPABILITIES

15. 15
RICH SW/HW CAPABILITIES
Prepared to handle many use-cases
Train on DevBox/DIGITS; Inference on DRIVE PX 2
Develop on PC (CUDA); Run on DRIVE PX 2
Develop on another NVIDIA embedded platform; Run on PX 2
…
By making the experience across homogeneous
Unified tool chain (across DevBox, PX2 and other embedded
platforms)
Multimedia APIs aligned across all platforms (Linux)
Full Open GL (not just ES)
Ubuntu Unity desktop
CUDA/CuDNN/DL Frameworks
…
Binary Compatibility
CORE DRIVE PX 2 CAPABILITIES

16. 16
RICH SW/HW CAPABILITIES
Data Logging
Data Logger
DRIVE PX 2
Camera
Bypass
Vehicle Information
Other Sensors
CAN/FlexRay
Gigabit Ethernet
Other Sensors & GPS
(LiDAR/Radar)
2x 10 Gigabit Ethernet
Uncompressed Camera stream
12x Camera
GMSL
2x HDMI (4K/60) as Data pipe
Optional: for uncompressed video
10 Gigabit Ethernet
Optional: for uncompressed video
ECU debug data
12x Camera
GMSL
TOOLS & ECOSYSTEM

17. 17
RICH SW/HW CAPABILITIES
User can define 20 different CAN node IDs
between Infineon Aurix and NVIDIA Tegra.
No special tool-chain required.
Messages defined through configuration files on
Linux (running on Tegra A).
E.g. one of the 20.
CAN Channel F (on Aurix):
Standard Can Id (0x244) - forwarded to Tegra B
CAN Bus configuration – EasyCAN*
* Provided by NVIDIA Partner Elektrobit
TOOLS & ECOSYSTEM

18. 18
RICH SW/HW CAPABILITIES
Graphics Debugger, System Profiler
Multi Process, Multi Node, Multi GPU
CUDA tools
Exquisite Developer Tools
TOOLS & ECOSYSTEM

19. 19
RICH SW/HW CAPABILITIES
DRIVEWORKS
Next session (S6834 - DriveWorks) @ 3 PM
Tools and building blocks to create your
autonomous driving applications.
Object detection, map localization, path planning
and visualization.
TOOLS & ECOSYSTEM

20. 20
PRODUCT PARTNERSHIPS
Camera, LiDAR and more
USB/GigE Cameras
LiDAR
Image Sensors
Hardware Reference board partners
SDKs and frameworks from Partners
TOOLS & ECOSYSTEM

21. 21
FUNCTIONAL SAFETY
Being developed as SEooC for autonomous driving
Developing work products
Functional and Technical Safety Concept – Scheme to achieve certain Safety Integrity Level for a
particular use-case.
System Design Specification – Functional details of HW / SW components & interfaces that are
compliant to various standards (i.e. ISO26262)
Hardware-Software Interface Specification – Register level details of HW/SW components
Safety Manual – Safety architecture (e.g. RTOS/AUTOSAR) and list of all safety mechanisms and
instructions to use them (e.g. DRAM ECC)
FMEDA - Documentation of random failure metrics
SEooC & Work Products for DRIVE PX 2
SPRINGBOARDS TO PRODUCTION

22. 22
FUNCTIONAL SAFETY
Certified Hypervisor and foundation partitions
Safety OS (CPU Complex, SCE, Safety MCU)
Safety supervision framework
Safety diagnostics libraries
Safety AUTOSAR Stack (SCE, Safety MCU)
Applications, Middleware and Libraries
SW architecture
L1SS
L2SS
L3SS
IPC
SPI
nSAFE
WD
WD
STLib
STLib
Tegra
Tegra/Pascal HW
SPRINGBOARDS TO PRODUCTION

23. 23
AUTOMOTIVE SW
AUTOSAR
DRIVE PX comes integrated with AUTOSAR 4.x-
compliant software suite from Elektrobit
Running on NVIDIA Tegra and Infineon AURIX™ 32-bit
TriCore™ microcontroller
Integrated Linux and AUTOSAR apps
Functionality to monitor & redundancy
management.
IPC in safe/reliable execution environment.
Also available reference stacks from other major
AUTOSAR solution providers.
SPRINGBOARDS TO PRODUCTION

24. 24
DRIVE PX 2
DRIVE PX 2: AI Supercomputer for Self Driving Cars
Built for application development, rapid embedded
prototyping and to help migrate to series
production.
Delivers powerful IO and processing capabilities,
rapidly expanding product ecosystem and several
means to shorten the path to production.
Closing remarks

25. April 4-7, 2016 | Silicon Valley
THANK YOU
JOIN THE NVIDIA DEVELOPER PROGRAM AT developer.nvidia.com/join

26. 26
REFERENCES
DRIVE PX 2 Launch at CES
https://youtu.be/C_8MZZ2TZUk (View Part 1 through 9)
Contact
ssundaram@nvidia.com