In this intermediate-level session, Jose De Oliveira, who heads up Unity's Autonomous Vehicles Strategy, will reveal our roadmap to leverage the leading 3D real-time rendering engine for creating a simulation and visualization framework for automotive projects. In particular, he will focus on how Unity is ideal for supporting the development of autonomous vehicle technologies and processes. Jose De Oliveira - Unity Technologies

1. “Autonomous Vehicles is a very easy technology to
develop. Self-driving cars are, in fact, a given.”
-- Nobody Ever

2. How can we increase the brainpower and
optimize the efforts applied to one of the
hardest problems in tech?

3. Jose De Oliveira
Lead Engineer for Autonomous Vehicles
Unity

4. Synthetic Environments for
Autonomous Vehicles
Development

5. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap?
• SimViz Announcement

6. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap
• SimViz Announcement

7. Product Design
Sales & Marketing
Service & Training
HMI
Autonomous Vehicles
Automotive Areas

8. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap
• SimViz Announcement

9. • ML is core for AV and is
data-centric
• Collecting data is expensive
• Annotating data is both
expensive and error prone
• Real-world data can only
prepare for what has already
happened
The autonomous vehicles data problem

11. Simulation can get autonomous
systems ready for unforeseen
scenarios and edge conditions in a
fast and cost effective way.
Why Simulate?

12. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap
• SimViz Announcement

13. Sim.Viz (/simviZ/), noun
1. Concatenation of the words “simulation” and "visualization" (plus
some alliteration for flair).
SimViz: !
• Unity’s initiatives, features and products to support
simulation and visualization applications in the fields
of autonomous vehicles (AV) and beyond.!
!

14. External Data!
(HD Maps,V2X)!
Autonomous System
(Real or Simulated)!
Simulation
Environment!
Control!
Planning!
Perception!
Coordination!
Vehicle
Dynamics!
Environment!
Sensor
Model!
Parameters
(weather, time)!
Dynamic
Elements!
Static
Elements!
Scenarios!
Simulation systems

15. External Data!
(HD Maps,V2X)!
Autonomous System
(Real or Simulated)!
Simulation
Environment!
Control!
Planning!
Perception!
Coordination!
Vehicle
Dynamics!
Environment!
Sensor
Model!
Parameters
(weather, time)!
Dynamic
Elements!
Static
Elements!
Scenarios!
Simulation systems
SimViz!

16. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap
• SimViz Announcement

17. How can Unity help AV development
• Incentivize development community: environments and
tools for simulations
• Build AV and ML support into the platform
• Promote an ecosystem of companies that extend the
platform and create simulation solutions
Licensed image from Adobe Stock

18. Asset Store Auto Packages
EasyRoads3D HD Pack (Andasoft)!
Enviro - Sky and Weather (Hendrik Haupt)!
https://assetstore.unity.com/lists/real-world-simulation-54066!

19. Robust ecosystem

20. Sensors!
Content! Integration!
SimViz Solution Templates Wording
Sensor SDK!
Reference models!
Leverage 2018+ features!
Communication layer!
Performance gains!
Linux improvements!
ML Agents!
Support to open formats!
Pipelines for data ingestion!
Dynamic elements models!
Starter content!

21. Unity ML Agents Workflow
Set Up Game
for Training!
Train
Agents!
Embed
Agents!

22. Q4! Q1! Q2! Q3! Q4!
SimViz Reference
Package!
• Sensor template!
• Content template!
ROS
Integration!
Platform Integration!
• DxM (Data Exchange
Module)!
• OpenDrive Support!
• Sensor Ready Materials!
Scalable Deployment!
• Headless runtime!
• AI based pedestrian
model!
• Sensor Validation
Standards!
• ML agent integration!
SimViz Solution Roadmap
2018! 2019!

23. Agenda
• Context
• Why simulate
• Unity SimViz
• SimViz Roadmap
• SimViz Announcement

24. SimViz Solution Template: Environments
Artificial Scenes included: !
o City streets
o Open Highway
o Residential streets

25. SimViz Solution Template: Sensors
Sensor Models included:!
o RGB-D Camera
o LiDAR
o Radar
o IMU
o GPS
SynCity Lite

26. Integration of Scenes and Sensor Templates