.Net Garbage Collector is an important factor in improving development efficiency by eliminating the need to manually manage memory. But sometimes, it's overhead may cause very long pauses in application execution. In this talk, I will show why this happens and how we can deal with it.

1. STAYING FRIENDLY WITH
THE GC
Oren Eini
oren@ravendb.net

2. CONTEXT
• RavenDB
• Document database
• Written in .NET
• Managed code
• High performance requirements

3. THE GARBAGE COLLECTOR
• It cleans up after you
• “No more memory leaks”
• Avoid manual memory management
• Allow you to focus on the real problems
you need to solve…
• Can be faster than manual memory management

4. GARBAGE COLLECTION
• Can be expensive
• Will pause the entire process
• Can cause pathologic behavior
• Will only find out about such issues very late in the game
(production)

5. HOW IT WORKS?
• Generations
• Bump allocator
• Mark & Sweep
• Allocations are cheap
• Memory is retained long term or
quickly discarded
• Freeing memory is cheap, scan small
sections, free in bulk

6. THE HEAP
• Allocated on Gen 0
• If survived GC, moved to Gen 1
• Collections happens on generations
• Gen 0 – often
• Gen 1 – sometimes
• Gen 2 - rare
Large
Object
Heap
Gen 0
• Per core
Gen 1Gen 2

7. BAD CASES?
• Many objects
• Cost per object, not per size
• 100 x byte[10] > 10 x byte[100]
• Retained for a while, then discarded
• Pinned memory (I/O, interop)
• Memory fragmentation
• LOH fragmentation
JSON
documents
DataSet
Xml
Document
Strings
> 80Kb

8. CONSEQUENCES
• High performance
• High concurrency
• Complex objects
• Retentions for caching, indexing, etc
• Direct I/O requires pinning
• We saw 90%+ CPU time, all in GC

9. SUPPORT CALL ROULETTE
• Got pretty bad
• Hard to retro-fix
• Without manual memory management,
how do you control it? GC
Memory
Frament-
ation

10. DO YOU NEED THIS?
• Requirements:
• High performance (tens of thousands of request / second)
• Throughput (dealing with 100,000s – 1,000,000s documents / second)
• Size 1 (dealing with documents that are multi KB range)
• Size 2 (dealing with databases that are tens to hundreds of GB typically, TBs as well)
• Latency matters (KPI measurement)

11. SOLUTION
• Don’t allocate
• No allocations, no GC
• Buffering, pooling, re-use objects
• ArrayPool<T>, MemoryPool<T>
• Span<T>, Memory<T>
• Stackalloc
• struct vs. class
• Understand the GC
• Work with it
• Long lived allocations & object reuse
• Very short term allocations
• Nothing in between

12. USING THE STACK
Unmanaged
Heap
Managed
Heap
Stack
• Free allocation & free
• Limited
• Stack Overflow Exception
• Process die
• struct, enum, ValueType
• stackalloc
• Unsafe – sometimes
• Beware of boxing

13. ANALYZE THIS?

14. BETTER…

15. NO ALLOCATIONS

16. BENEFITS
• Other:
• Code is simpler for the compiler
• More chances for optimizations
• Auto vectorization, instruction level parallelism
• Might allow inlining
• Can be used in hot spots

17. UNMANAGED HEAP
• You control allocations / free
• Can happen when you want it to
• Adjust to your own behavior
• Still benefit from pooling
• Avoid excessive alloc / free (system
calls!)
• Marshal.AllocHGlobal
• Marhsal.FreeHGlobal

18. REVERSE STRING…
Heap allocation
Heap allocation
Heap allocation

19. DOING IT MANUALLY
Heap allocation
Heap allocation

20. UNSAFE CODE…
Heap allocation
Allow to access
the memory directly

21. MODERN C#

22. USING UNMANAGED MEMORY
• Span<T> / Memory<T>
• T[] - array
• T* - unmanaged memory
• But how to manage?
• Arena Allocator
• Context
• GC is:
• _allocated = 0;
Segment – 64
KB
Segment –
128 KB
Segment –
256KB
Allocated

23. RESULTS…
Unmanaged Managed

24. BEWARE
• Memory leak
• Use after free
• Buffer overlap
• Double free
• Buffer overrun
• With great power…

25. THE CONTEXT
• A unit of work in the application
• Processing of a single request
• Handling a single document
• Short lived
• Frequent
• Scope of actions
• Hang memory management
• Reset arena
• Pool objects
• Pool contexts
• Explicit in API, explicit in design

26. TWO FACED GC

27. QUESTIONS?