Submit Search
Upload
Memory management in Linux
•
78 likes
•
46,776 views
Raghu Udiyar
Follow
A brief overview of the memory management subsystem in the Linux kernel.
Read less
Read more
Technology
Report
Share
Report
Share
1 of 27
Recommended
Linux Memory Management
Linux Memory Management
Anil Kumar Pugalia
Linux Memory Management
Linux Memory Management
Suvendu Kumar Dash
Linux memory
Linux memory
ericrain911
Linux memory-management-kamal
Linux memory-management-kamal
Kamal Maiti
Physical Memory Management.pdf
Physical Memory Management.pdf
Adrian Huang
Physical Memory Models.pdf
Physical Memory Models.pdf
Adrian Huang
Memory management in Linux kernel
Memory management in Linux kernel
Vadim Nikitin
Linux Memory Management
Linux Memory Management
Ni Zo-Ma
Recommended
Linux Memory Management
Linux Memory Management
Anil Kumar Pugalia
Linux Memory Management
Linux Memory Management
Suvendu Kumar Dash
Linux memory
Linux memory
ericrain911
Linux memory-management-kamal
Linux memory-management-kamal
Kamal Maiti
Physical Memory Management.pdf
Physical Memory Management.pdf
Adrian Huang
Physical Memory Models.pdf
Physical Memory Models.pdf
Adrian Huang
Memory management in Linux kernel
Memory management in Linux kernel
Vadim Nikitin
Linux Memory Management
Linux Memory Management
Ni Zo-Ma
Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)
Pankaj Suryawanshi
Disk and File System Management in Linux
Disk and File System Management in Linux
Henry Osborne
Reverse Mapping (rmap) in Linux Kernel
Reverse Mapping (rmap) in Linux Kernel
Adrian Huang
Linux Kernel Booting Process (2) - For NLKB
Linux Kernel Booting Process (2) - For NLKB
shimosawa
Memory Mapping Implementation (mmap) in Linux Kernel
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
Continguous Memory Allocator in the Linux Kernel
Continguous Memory Allocator in the Linux Kernel
Kernel TLV
Debugging linux kernel tools and techniques
Debugging linux kernel tools and techniques
Satpal Parmar
Slab Allocator in Linux Kernel
Slab Allocator in Linux Kernel
Adrian Huang
New Ways to Find Latency in Linux Using Tracing
New Ways to Find Latency in Linux Using Tracing
ScyllaDB
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
Stefano Stabellini
Linux memory consumption
Linux memory consumption
haish
Performance Tuning EC2 Instances
Performance Tuning EC2 Instances
Brendan Gregg
Linux kernel memory allocators
Linux kernel memory allocators
Hao-Ran Liu
Linux Crash Dump Capture and Analysis
Linux Crash Dump Capture and Analysis
Paul V. Novarese
Operating System-Memory Management
Operating System-Memory Management
Akmal Cikmat
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Anne Nicolas
Windows memory management
Windows memory management
Tech_MX
Linux Performance Profiling and Monitoring
Linux Performance Profiling and Monitoring
Georg Schönberger
Linux MMAP & Ioremap introduction
Linux MMAP & Ioremap introduction
Gene Chang
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
Kernel TLV
Live memory forensics
Live memory forensics
Mehedi Hasan
Vmfs
Vmfs
Erick Treviño
More Related Content
What's hot
Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)
Pankaj Suryawanshi
Disk and File System Management in Linux
Disk and File System Management in Linux
Henry Osborne
Reverse Mapping (rmap) in Linux Kernel
Reverse Mapping (rmap) in Linux Kernel
Adrian Huang
Linux Kernel Booting Process (2) - For NLKB
Linux Kernel Booting Process (2) - For NLKB
shimosawa
Memory Mapping Implementation (mmap) in Linux Kernel
Memory Mapping Implementation (mmap) in Linux Kernel
Adrian Huang
Continguous Memory Allocator in the Linux Kernel
Continguous Memory Allocator in the Linux Kernel
Kernel TLV
Debugging linux kernel tools and techniques
Debugging linux kernel tools and techniques
Satpal Parmar
Slab Allocator in Linux Kernel
Slab Allocator in Linux Kernel
Adrian Huang
New Ways to Find Latency in Linux Using Tracing
New Ways to Find Latency in Linux Using Tracing
ScyllaDB
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
Stefano Stabellini
Linux memory consumption
Linux memory consumption
haish
Performance Tuning EC2 Instances
Performance Tuning EC2 Instances
Brendan Gregg
Linux kernel memory allocators
Linux kernel memory allocators
Hao-Ran Liu
Linux Crash Dump Capture and Analysis
Linux Crash Dump Capture and Analysis
Paul V. Novarese
Operating System-Memory Management
Operating System-Memory Management
Akmal Cikmat
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Anne Nicolas
Windows memory management
Windows memory management
Tech_MX
Linux Performance Profiling and Monitoring
Linux Performance Profiling and Monitoring
Georg Schönberger
Linux MMAP & Ioremap introduction
Linux MMAP & Ioremap introduction
Gene Chang
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
Kernel TLV
What's hot
(20)
Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)
Disk and File System Management in Linux
Disk and File System Management in Linux
Reverse Mapping (rmap) in Linux Kernel
Reverse Mapping (rmap) in Linux Kernel
Linux Kernel Booting Process (2) - For NLKB
Linux Kernel Booting Process (2) - For NLKB
Memory Mapping Implementation (mmap) in Linux Kernel
Memory Mapping Implementation (mmap) in Linux Kernel
Continguous Memory Allocator in the Linux Kernel
Continguous Memory Allocator in the Linux Kernel
Debugging linux kernel tools and techniques
Debugging linux kernel tools and techniques
Slab Allocator in Linux Kernel
Slab Allocator in Linux Kernel
New Ways to Find Latency in Linux Using Tracing
New Ways to Find Latency in Linux Using Tracing
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
Linux memory consumption
Linux memory consumption
Performance Tuning EC2 Instances
Performance Tuning EC2 Instances
Linux kernel memory allocators
Linux kernel memory allocators
Linux Crash Dump Capture and Analysis
Linux Crash Dump Capture and Analysis
Operating System-Memory Management
Operating System-Memory Management
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all started
Windows memory management
Windows memory management
Linux Performance Profiling and Monitoring
Linux Performance Profiling and Monitoring
Linux MMAP & Ioremap introduction
Linux MMAP & Ioremap introduction
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
Similar to Memory management in Linux
Live memory forensics
Live memory forensics
Mehedi Hasan
Vmfs
Vmfs
Erick Treviño
Refining Linux
Refining Linux
Jason Murray
5.6 Basic computer structure microprocessors
5.6 Basic computer structure microprocessors
lpapadop
제2회난공불락 오픈소스 세미나 커널튜닝
제2회난공불락 오픈소스 세미나 커널튜닝
Tommy Lee
Linux Memory
Linux Memory
Vitaly Nahshunov
Understanding And Managing Memory
Understanding And Managing Memory
isma ishak
7989-lect 10.pdf
7989-lect 10.pdf
RiazAhmad521284
virtual memory - Computer operating system
virtual memory - Computer operating system
Electronics - Embedded System
What every data programmer needs to know about disks
What every data programmer needs to know about disks
iammutex
Measuring Firebird Disk I/O
Measuring Firebird Disk I/O
Mind The Firebird
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
Kyle Hailey
Presentation aix performance tuning
Presentation aix performance tuning
xKinAnx
tuningfor_oracle
tuningfor_oracle
styxyx
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Виталий Стародубцев
Operating Systems - A Primer
Operating Systems - A Primer
Saumil Shah
20080528dublinpt3
20080528dublinpt3
Jeff Hammerbacher
Servers and Processes: Behavior and Analysis
Servers and Processes: Behavior and Analysis
dreamwidth
virtual memory
virtual memory
Abeer Naskar
Linux Performance Tunning Memory
Linux Performance Tunning Memory
Shay Cohen
Similar to Memory management in Linux
(20)
Live memory forensics
Live memory forensics
Vmfs
Vmfs
Refining Linux
Refining Linux
5.6 Basic computer structure microprocessors
5.6 Basic computer structure microprocessors
제2회난공불락 오픈소스 세미나 커널튜닝
제2회난공불락 오픈소스 세미나 커널튜닝
Linux Memory
Linux Memory
Understanding And Managing Memory
Understanding And Managing Memory
7989-lect 10.pdf
7989-lect 10.pdf
virtual memory - Computer operating system
virtual memory - Computer operating system
What every data programmer needs to know about disks
What every data programmer needs to know about disks
Measuring Firebird Disk I/O
Measuring Firebird Disk I/O
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
Presentation aix performance tuning
Presentation aix performance tuning
tuningfor_oracle
tuningfor_oracle
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...
Operating Systems - A Primer
Operating Systems - A Primer
20080528dublinpt3
20080528dublinpt3
Servers and Processes: Behavior and Analysis
Servers and Processes: Behavior and Analysis
virtual memory
virtual memory
Linux Performance Tunning Memory
Linux Performance Tunning Memory
Recently uploaded
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Sergiu Bodiu
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
Dilum Bandara
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
Miki Katsuragi
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Lonnie McRorey
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
Pixlogix Infotech
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
Fwdays
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
LoriGlavin3
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
Scott Keck-Warren
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
null - The Open Security Community
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
Lars Bell
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
RankYa
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
Alan Dix
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
Curtis Poe
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
charlottematthew16
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Florian Wilhelm
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
NavinnSomaal
Recently uploaded
(20)
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
Memory management in Linux
1.
Memory management in
Linux Raghu Udiyar Technical Support Engineer Red Hat
2.
3.
Can a disk
be considered memory? Yes, in a way
4.
5.
Latency
6.
Volatile
7.
8.
CPU cache (L1,
L2..) - 3 – 14 cycles
9.
RAM (main memory)
- ~250 cycles
10.
Disk -
~40 million cycles
11.
12.
These are then
taken and divided into page frames of 4Kb
13.
E.g. 4Gb
= ~1 million pages
14.
15.
Every process has
a virtual view of the memory
16.
A process will
think it is the only task running on the system
17.
This virtual memory
is then mapped to physical memory using page tables
18.
Virtual Memory vi
bash 2GB Physical Memory Virtual memory
19.
Virtual Memory vi
bash 2GB Physical Memory Virtual memory Page Table
20.
21.
User and Kernel
address space Kernel User Kernel User 32bit 64bit 3Gb 1GB 256TB 256TB
22.
23.
malloc(1G) = Ok
24.
memset(1G) = Ok
25.
malloc(4GB) =
Ok
26.
memset(4GB) = -ENOMEM
27.
Kernel almost always
grants Virtual memory
28.
Page fault occurs
only when that memory is used
29.
30.
In such cases
the “OOM killer” is invoked
31.
It will try
to free memory by killing a task based on heuristics
32.
OOM killer message
33.
34.
All i/o in
Linux goes through the page cache Write-back (delayed) Clean/Dirty pages Page cache Process Disk
35.
36.
# dd if=/dev/zero
of=512M-file count=512000 bs=1k
37.
# time cat
512M-file &> /dev/null
38.
# echo 1
> /proc/sys/vm/drop_caches
39.
# time cat
512M-file &> /dev/null
40.
41.
Program text -
read only
42.
File backed -
clean or dirty pages
43.
Anonymous pages –
Not file-backed, e.g stack and heap
44.
Only anonymous
memory is paged out to swap area
45.
46.
Considerably slow
- ~40 million cycles
47.
Swapping is a
misnomer, pages paged out/in is more appropriate
48.
Only anonymous pages
are paged out
49.
50.
Tools - vmstat
51.
Tools - ps
52.
53.
(10mins default)
54.
Check “man sar”
for details on all switches
55.
56.
#sar -f sa05
-tq
57.
#sar -f sa05
-tr
58.
#sar -f sa05
-tW
59.
60.
https://access.redhat.com/kb/docs/DOC-1139
61.
62.
Check :
https://access.redhat.com/kb/docs/DOC-52712
63.
64.
https://access.redhat.com/kb/docs/DOC-3368
65.
Swap usage doesn't
necessarily indicate a problem
66.
Instead, check #sar
-tW for problems
67.
68.
In most case,
this is not advised, and arises from misunderstanding of how the page cache works
69.
Controlling page cache
size : https://access.redhat.com/kb/docs/DOC-41749
70.
71.
Some confuse this
with swap in/out
72.
These are just
the number of sectors read in or written to disk.
73.
See :
https://access.redhat.com/kb/docs/DOC-61288
74.
75.
Check :
https://access.redhat.com/kb/docs/DOC-52474
76.
Thank you
Questions?