ECS: Streaming and Serialization - Unite LA
•Download as PPTX, PDF•
4 likes•3,099 views
In this in-depth talk we'll cover entity serialization to and from disk, how we built seamless play mode streaming and efficient editor workflow, and what a separate ECS world can do for you. Fabrice Lété (Unity Technologies), Simon Mogensen (Unity Technologies)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to ECS: Streaming and Serialization - Unite LA
Similar to ECS: Streaming and Serialization - Unite LA (20)
More from Unity Technologies
More from Unity Technologies (20)
Recently uploaded
Recently uploaded (20)
ECS: Streaming and Serialization - Unite LA
- 1. 1
- 2. GenerativeArt–MadewithUnity ECSStreaming and Serialization Fabrice Lété Software Engineer, Unity fabricel@unity3d.com Simon Mogensen Software Engineer, Unity simonm@unity3d.com 2
- 3. This is not about streaming resources. 3
- 5. GenerativeArt–MadewithUnity 5 - 6M Entities - 4.5M Transforms - 1.2M LOD Groups - 100K Sound Emitters - 10K Dynamic Lights
- 6. GenerativeArt–MadewithUnity 6 - 6M Entities - 4.5M Transforms - 1.2M LOD Groups - 100K Sound Emitters - 10K Dynamic Lights - 2.2 GB of raw data - 100 Subscenes
- 7. GenerativeArt–MadewithUnity 7 - 6M Entities - 4.5M Transforms - 1.2M LOD Groups - 100K Sound Emitters - 10K Dynamic Lights - Streaming < 2ms - 2.2 GB of raw data - 100 Subscenes
- 8. 8 1. Convert GameObjects to Entities via ComponentDataWrapper or custom code. 2. No Conversion not everything can be ECS yet.GameObjects GameObjects Entities / Components EditorWorkflow Overview
- 9. 9 GameObjects GameObjects Entities / Components 1. Asynchronous Loading to a Staging ECS World 2. Merge via MoveEntities SubsceneMain Scene Entities / Components EditorDisk Save Load Merge Workflow Overview
- 10. 10 GameObjects GameObjects Entities / Components SubsceneMain Scene Entities / Components Entities / ComponentsGameObjects Save Load Merge Merge EditorDiskGame
- 11. 11 Entity Manager Array of entities
- 12. 12 Entity Manager Array of entities Chunk Components Chunk Components Chunk Components Chunk Components
- 13. ArchetypeArchetypeArchetype 13 Entity Manager Array of entities Chunk Components Chunk Components Chunk Components Chunk Components
- 14. ArchetypeArchetypeArchetype 14 Entity Manager Array of entities Chunk Components Chunk Components Chunk Components Chunk Components Type Manager Array of types
- 15. ArchetypeArchetypeArchetype 15 Entity Manager Array of entities Chunk Components Chunk Components Chunk Components Chunk Components Shared Component Manager Array of C# objects Type Manager Array of types
- 16. World ArchetypeArchetypeArchetype 16 Entity Manager Array of entities Chunk Components Chunk Components Chunk Components Chunk Components Shared Component Manager Array of C# objects Type Manager Array of types
- 17. 17 File Formats SomeBuilding.scene GameObject Scene Editor Only Editor and Game
- 18. 18 File Formats SomeBuilding.scene GameObject Scene SomeBuilding_header.asset Meta Data Editor Only Editor and Game
- 19. 19 File Formats Shared Components SomeBuilding_shared.prefabSomeBuilding.scene GameObject Scene SomeBuilding_header.asset Meta Data Editor Only Editor and Game
- 20. Classic Unity File Format 20 File Formats Shared Components SomeBuilding_shared.prefabSomeBuilding.scene GameObject Scene SomeBuilding_header.asset Meta Data Editor Only Editor and Game
- 21. Classic Unity File Format ECS Specific File Format 21 File Formats Shared Components SomeBuilding_shared.prefabSomeBuilding.scene GameObject Scene SomeBuilding.entities Types / Archetypes / Chunks SomeBuilding_header.asset Meta Data Editor Only Editor and Game
- 22. 22 Staging Worlds Main World Staging WorldStaging World Staging World
- 23. 23 Staging Worlds Main World Staging WorldStaging World Staging World Available Available Available
- 24. 24 Staging Worlds Main World Staging WorldStaging World Staging World Load Request Available Busy Available
- 25. 25 Staging Worlds Main World Staging WorldStaging World Staging World Load Shared Components Load Entities Load Request Async Async
- 26. 26 Staging Worlds Main World Staging WorldStaging World Staging World Load Shared Components Load Entities Load Request Async Async ECS Data Deserialize
- 27. 27 Staging Worlds Main World Staging WorldStaging World Staging World Load Shared Components Load Entities Load Request Async Async ECS Data Move Deserialize - Move leaves the staging world empty.
- 28. 28 External References Main World Subscene BSubscene A Subscene C Move MoveMove
- 29. 29 External References Main World Subscene BSubscene A Subscene C Move MoveMove Ref Ref Ref
- 30. 30 Subscene Identification Subscene A Chunk Components Chunk Components Chunk Components Shared Component Subscene ID Subscene B Chunk Components Chunk Components Chunk Components Shared Component Subscene ID
- 31. 31
- 32. 32
- 33. Moving Entities into the main world 33 ● Allocate Entity ids ● Remap Entity ids in chunks
- 34. 34 Allocating entity ids 1 1 1 1 A 2 1 C 1 1 C 4 1 A 1 1 C 0 1 A 3 1 C 3 1 C 2 1 B 2 Index A 0 0 B 1 1 B 0 2 3 4 5 6 7 8 9 10 11 Version Chunk Index in Chunk
- 35. Allocating entity ids 35 Version Chunk Index in Chunk 1 A 2 1 - 4 1 C 4 1 A 1 1 - 9 1 A 3 1 C 3 1 - 1 1 B 2 1 - 10 1 - 11 1 - -1 Index 0 1 2 3 4 5 6 7 8 9 10 11 First free index
- 37. 37 Version Chunk Index in Chunk 1 A 2 1 - ? 1 C 4 1 A 1 1 - ? 1 A 3 1 C 3 1 - ? 1 B 2 1 - ? 1 - ? 1 - ? Index 0 1 2 3 4 5 6 7 8 9 10 11 Head of free list Free index array 11 10 9 4 1 7 ? ? ? ? ? ? Allocating entity ids
- 39. Remapping entity ids 39 Source version Destination index Destination version 3 1 7 2 3 6 6 5 2 1 6 2 2 9 5 7 10 3 8 13 3 2 14 2 Source index 0 1 2 3 4 5 6 7
- 40. 40 Main world Staging world Remap array Remapping entity ids
- 41. Subscene B 41 PublicEntityRef SceneGUID Subscene ID c7a37858 Main World EntityUID 36bf20ba 4ccc438a 50c1ccb0 a26722ee db774f0d EntityID 2 8 23 45 34 1 3 1 3 4
- 42. Subscene B Subscene A 42 PublicEntityRef EntityUID ExternalEntityRef 4ccc438a 50c1ccb0 db774f0d TargetSceneGUID ExternalEntityRefInfo FirstDummyIndex SceneGUID Subscene ID c7a378588 c7a37858 Main WorldStaging World EntityUID 36bf20ba 4ccc438a 50c1ccb0 a26722ee db774f0d EntityID 2 8 23 45 34 1 3 1 3 4
- 43. Subscene B 43 PublicEntityRef SceneGUID Subscene ID c7a37858 Main WorldStaging World EntityUID 36bf20ba 4ccc438a 50c1ccb0 a26722ee db774f0d EntityID 2 8 23 45 34 1 3 1 3 4 Subscene A EntityUID ExternalEntityRef 4ccc438a 50c1ccb0 db774f0d TargetSceneGUID ExternalEntityRefInfo FirstDummyIndex c7a378588
- 44. 44 Source version Destination index Destination version 3 1 7 2 3 6 6 5 2 1 6 2 2 9 5 7 10 3 8 13 3 2 14 2 Source index 0 1 2 3 4 5 6 7 1 4 1 1 7 3 1 12 6 8 9 10 Dummy indices for external references Cross scene references
- 45. 45 Main world Staging world Remap array Remapping entity ids
- 46. 46 Unloading subscenes ● Find all chunks with the correct SubsceneID shared component ● Delete chunks
- 47. 47 But ...
- 49. Performance 49
- 50. Performance 51436 entities integrated into the main world in 1.20 ms 50
- 51. Performance Unloading / deleting 51436 entities 0.96 ms 51
- 52. Latency 52 ● 100 to 500 ms ● Depends on number of concurrent loads ● After data is loaded: 1-2 frames
- 53. Future work 53 ● Chunk allocator / stream data directly into final chunks ● System state components ● Canceling load requests ● Compression