Slides of the Golang Lyon meetup talk done on 15/11/2017 about memory allocation in Go (stack management, escape analysis, ...).

1. Advanced memory
allocation in Go

2. Joris Bonnefoy
DevOps @ OVH
@devatoria

3. Before we get started...

4. Keep it simple, readable
● Avoid premature optimizations
○ Sometimes, readability and logicalness are better than performances
● Use go tools in order to point out real problems
○ pprof
○ gcflags

5. Introduction to
memory management

6. Virtual and physical memory
● When you launch a new process, the kernel
creates its address space
● In this address space, the process can
allocate memory
● For the process, its address space looks like
a big contiguous memory space
● For two identical processes, (logical)
addresses can be the same
○ Depending on the compiler, the architecture, ...

7. Virtual and physical memory
● Each process has its own “memory
sandbox” called its virtual address
space
● Virtual addresses are mapped to
physical addresses by the CPU (and
the MMU component) using page
tables
● Per-process virtual space size is
4GB on 32-bits system and 256TB
on 64-bits one

8. Virtual and physical memory
● Virtual address space is split into 2 spaces
○ Kernel space
○ User mode space (or process address space)
● Split depends on operating system
configuration

9. How is user space managed? (C example)

10. Stack vs. Heap

11. The stack
● On the top of the process address space
○ Grows down
● Last In First Out design
○ Cheap allocation cost
● Only one register is needed to track content
○ Stack Pointer (SP register)
● Calling a function pushes a new stack frame
onto the stack
● Stack frame is destroyed on function return
● Each thread in a process has its own stack
○ They share the same address space
● Stack size is limited, but can be expanded
○ Until a certain limit, usually 8MB

12. The heap
● Grows up
● Expensive allocation cost
○ No specific data structure
● Complex management
● Used to allocate variables that must
outlive the function doing the allocation
● Fragmentation issues
○ Contiguous free blocks can be merged

13. Memory allocation
in Go

14. Like threads, goroutines have
their own stack.

15. <= Go 1.2 - Segmented stacks
● Discontiguous stacks
● Grows incrementally
● Each stack starts with a segment (8kB in Go 1.2)
● When stack is full
a. another segment is created and linked to the stack
b. stack segment is removed when not used anymore (stack is shrinked)
● Stacks are doubly-linked list

16. <= Go 1.2 - Segmented stacks

17. <= Go 1.2 - Hot split issue

18. >= Go 1.3 - Copying stacks
● Contiguous stacks
● Each stack starts with a size of 2kB (since Go 1.4)
● When stack is full
a. a new stack is created, with the double size of the previous one
b. content of the old one is copied to the new one
c. pointers are re-adjusted
d. old one is destroyed
● Stack is never shrinked

19. >= Go 1.3 - Copying stacks

20. Benchmarks

21. Efficient memory
allocation (and compiler
optimizations)

22. Reminders
Heap
(de)allocation
is expensive
Stack
(de)allocation
is cheap

23. Reminders
Go manages memory automatically

24. Reminders
Go prefers allocation on the stack

25. Go functions
inlining

26. Go functions inlining
● The code of the inlined function is inserted at the place of each call to this function
● No more assembly CALL instruction
● No need to create function stack frame
● Binary size is increased because of the possible repetition of assembly instructions
● Can be disabled using //go:noinline comment just before the function declaration

27. Go escape
analysis system

28. Escape analysis
● Decides whether a variable should be allocated on the heap or on the stack
● Creates a graph of function calls in order to track variables scope
● Uses tracking data to pass checks on those variables
○ Those checks are not explicitly detailed in Go specs
● If checks pass, the variable is allocated on the stack (it doesn’t escape)
● If at least one check fails, the variable is allocated on the heap (it escapes)
● Escape analysis results can be checked at compile time using
○ go build -gcflags '-m' ./main.go

29. One basic rule (not always right…)
If a variable has its address taken,
that variable is a candidate for allocation on the heap

30. Closure calls
Closure calls are not analyzed

31. Assignments to slices and maps
A map can be allocated on the stack,
but any keys or values inserted into the map will escape

32. Flow through channels
A variable passing through a channel
will always escape

33. Interfaces
Interfaces can lead to escape
when a function of the given interface is called
(because the compiler doesn’t know
what the function is doing with its arguments)

34. And a lot of other cases...
● Go escape analysis is very simple and not so smart
● Some issues are opened to improve it

35. Conclusion

36. Keep it simple, readable
● Avoid premature optimizations
○ Sometimes, readability and logicalness are better than performances
● Use go tools in order to point out real problems
○ pprof
○ gcflags

37. Remember the basics
● Pointers are only useful if you directly manipulate variable value
○ Most of the time, a copy of the value is sufficient
● Closures are not always sexy
○ Do not overuse them just to overuse them
● Manipulate arrays when possible
○ Slices are cool, but arrays too... :)

38. Thank you
for listening!

39. References
● https://segment.com/blog/allocation-efficiency-in-high-performance-go-services/
● https://en.wikipedia.org/wiki/Stack-based_memory_allocation
● http://gribblelab.org/CBootCamp/7_Memory_Stack_vs_Heap.html
● https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast
● https://blog.cloudflare.com/how-stacks-are-handled-in-go/
● http://www.tldp.org/LDP/tlk/mm/memory.html
● http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory/
● http://agis.io/2014/03/25/contiguous-stacks-in-go.html
● https://medium.com/@felipedutratine/does-golang-inline-functions-b41ee2d743fa
● https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpTe5q32QDmz8l0BouG0Cw/preview