Autonomous Vehicle Development with Unity

1. “Autonomous Vehicles is a very easy technology to develop. Self-driving cars are, in fact, a given.” -- Nobody Ever

2. How can we increase the brainpower and optimize the efforts applied to one of the hardest problems in tech?

3. Jose De Oliveira Lead Engineer for Autonomous Vehicles Unity

4. Synthetic Environments for Autonomous Vehicles Development

5. Agenda •  Context •  Why simulate •  Unity SimViz •  SimViz Roadmap? •  SimViz Announcement

6. Agenda •  Context •  Why simulate •  Unity SimViz •  SimViz Roadmap •  SimViz Announcement

7. Product Design Sales & Marketing Service & Training HMI Autonomous Vehicles Automotive Areas

9. •  ML is core for AV and is data-centric •  Collecting data is expensive •  Annotating data is both expensive and error prone •  Real-world data can only prepare for what has already happened The autonomous vehicles data problem

10. Simulation can get autonomous systems ready for unforeseen scenarios and edge conditions in a fast and cost effective way. Why Simulate?

12. Sim.Viz (/simviZ/), noun 1.  Concatenation of the words “simulation” and "visualization" (plus some alliteration for flair). SimViz: ! •  Unity’s initiatives, features and products to support simulation and visualization applications in the ﬁelds of autonomous vehicles (AV) and beyond.! !

13. External Data! (HD Maps,V2X)! Autonomous System (Real or Simulated)! Simulation Environment! Control! Planning! Perception! Coordination! Vehicle Dynamics! Environment! Sensor Model! Parameters (weather, time)! Dynamic Elements! Static Elements! Scenarios! Simulation systems

14. External Data! (HD Maps,V2X)! Autonomous System (Real or Simulated)! Simulation Environment! Control! Planning! Perception! Coordination! Vehicle Dynamics! Environment! Sensor Model! Parameters (weather, time)! Dynamic Elements! Static Elements! Scenarios! Simulation systems SimViz!

16. How can Unity help AV development •  Incentivize development community: environments and tools for simulations •  Build AV and ML support into the platform •  Promote an ecosystem of companies that extend the platform and create simulation solutions Licensed image from Adobe Stock

17. Asset Store Auto Packages EasyRoads3D HD Pack (Andasoft)! Enviro - Sky and Weather (Hendrik Haupt)! https://assetstore.unity.com/lists/real-world-simulation-54066!

18. Robust ecosystem

19. Sensors! Content! Integration! SimViz Solution Templates Wording Sensor SDK! Reference models! Leverage 2018+ features! Communication layer! Performance gains! Linux improvements! ML Agents! Support to open formats! Pipelines for data ingestion! Dynamic elements models! Starter content!

20. Unity ML Agents Workflow Set Up Game for Training! Train Agents! Embed Agents!

21. Q4! Q1! Q2! Q3! Q4! SimViz Reference Package! •  Sensor template! •  Content template! ROS Integration! Platform Integration! •  DxM (Data Exchange Module)! •  OpenDrive Support! •  Sensor Ready Materials! Scalable Deployment! •  Headless runtime! •  AI based pedestrian model! •  Sensor Validation Standards! •  ML agent integration! SimViz Solution Roadmap 2018! 2019!

23. SimViz Solution Template: Environments Artificial Scenes included: ! o  City streets o  Open Highway o  Residential streets

24. SimViz Solution Template: Sensors Sensor Models included:! o  RGB-D Camera o  LiDAR o  Radar o  IMU o  GPS SynCity Lite

25. Integration of Scenes and Sensor Templates

Autonomous Vehicle Development with Unity

Unity Technologies

Autonomous Vehicle Development with Unity