The document discusses Linux kernel memory management. It covers how Linux uses virtual memory to extend physical memory and allows processes to be relocated. The memory manager unit and translation lookaside buffer help manage virtual memory addresses. Managing memory is one of the main tasks of the Linux kernel, as it enables advanced features like larger address spaces and on-demand paging.
2. + Memory Management
Linux Kernel Memory Management
( https://www.kernel.org/doc/Documentation/vm/ )
Linux uses Virtual Memory to manage the
address space of the entire OS
VM extends the size of the physical memory
All OS processes 'live' in the VM and can
occasionally be relocated due to the kernel
demand
3. + Memory Management
Linux Kernel Memory Management
Memory Manager Unit (MMU) is an
implementation of a hardware object
( in the CPU ) which manages the VM
Translation Lookaside Buffer (TLB) is a table
maintained by the CPU which holds recently
used main-to-virtual 'translations’
( diigo.com/085lsy )
5. + Memory Management
Linux Kernel Memory Management
Managing virtual memory is the 'heaviest' task of
the Linux Kernel
It makes most of the OS advanced features
possible
Either dig deep, or run auto-pilot
( Tuned and Ktune diigo.com/085ltv )
6. + Memory Management
Linux Kernel Memory Management
Linux Kernel uses VM to provide a much larger
memory than the amount of physical memory
When a memory page needs to be written and
there isnt a free page on physical memory Linux
uses swapping to ‘make room’ for that page
Linux Kernel uses LRU (Least Recently Used)
algorithm to calculate which page is to be
‘swapped out’
7. + Memory Management
Random Access Memory
RAM is being used by the Kernel for
– Storing the Kernel Code and Data
– Handle Kernel's buffers and descriptors
requests
– Handle Processes generic memory areas
and file mapping requests
– *Cache buffered devices I/O
8. + Memory Management
Random Access Memory
Dividing the limited amount of RAM between all
of these tasks is crucial if one is to achieve
proper performance from the system
Memory fragmentation can also be a set back in
terms of performance and needs to be dealt with
by the Kernel
9. + Memory Management
Kernel Memory Allocator
KMA is a kernel subsystem which handles
memory allocation requests from all portions of
the OS
Linux Kernel uses SLAB allocator with the
'buddy' algorithm
– Allocate pages according to request type
– Reuses kernel functions (e.g: exec)
metadata pages
10. + Memory Management
Process Virtual Address Space
The address space of each process contains all
of the memory addresses which are accessible
to that process, including:
execute code
un/initialized program data
shared libraries
heap
11. + Memory Management
Process Virtual Address Space
The Linux Kernel uses advanced strategies in
order to improve performance on these tasks
– Demand Paging: Allows a program to
start execution before all of it's pages are
actually loaded
– Copy-on-Write: Fork child processes with
a read-only reference to the parent
process address space
12. + Memory Management
Cache (Page Cache)
Most of the available RAM on the system is used
as a buffered I/O cache.
This is done in order to improve performance
when working with 'slow' devices
Write cache ('dirty' buffers) is being periodically
'sync'ed to the original device
13. + Memory Management
Cache (Page Cache)
Linux Kernel caches pages which contains
Data being processed from a file system
Data being processed directly from a block device
Both files and directories data
File structure, inodes, etc’
User mode process data which was swapped out
14. + Memory Management
Cache (Page Cache)
To disable the page cache for a certain
operation, the relevant system call, should be
called with the flag O_DIRECT
Use the command ‘readahead’ to load a file list
into the Page Cache for later processing
15. + Memory Management
Huge Pages
Linux uses 'page' as the smallest portion of RAM
being mapped
When mapping a huge size of sequential pages,
there is a noticeable overhead for managing
these pages
Huge pages address this problem by eliminating
the limitation of small (default: 4k) page size
16. + Memory Management
Kernel Threads
Some of the Kernel tasks related to VM
management are being done using kernel
threads
Keventd (aka events)
Kswapd
Pdflush
Kblockd
Kjournald
17. + Memory Management
Kernel Tune able Parameters
Most of the mechanisms described throughout
this course can be tuned using the 'sysctl' or
'/proc' interfaces
In order to keep track of the tune able
parameters default values, inner-relations and
current effect, make sure to consult the kernel
Documentation tree
(/usr/share/doc/kernel-*/Documentation)
18. + Memory Management
Kernel Tune able Parameters
'drop_caches': invokes a cache 'sync' and
free up cached memory
'vfs_cache_pressure': tune the amount of
resources put into reclaiming inodes and
dentries cache
'dirty_background_ratio': when will pdflush start writing out 'dirty' data
19. + Memory Management
Kernel Tune able Parameters
'dirty_writeback_centisecs': pdflush
interval
'dirty_expire_centisecs': page's maximum
'age' before pdflush writes it out to disk
20. + Memory Management
Kernel Tune able Parameters
'page-cluster': number of pages written to
SWAP during swap in (logarithmic)
'nr_hugepages': number of allocated huge
pages (works with 'hugetlb_shm_group')
21. + Memory Management
Linux Kernel Memory Management
Use the following commands to review memory
information
'top'
'vmstat’
‘sar’
'free’
‘cat /proc/meminfo’