The document discusses monitoring and analyzing memory usage in Raku processes. It describes using the getrusage(2) system call to retrieve resident set size (RSS) and other memory statistics for a process. It then presents the ProcStats module, which allows periodically sampling getrusage(2) data over time to track RSS and detect changes from an initial baseline. The module outputs differences in memory statistics compared to the first sample, ignoring unchanged values. This provides a concise way to monitor for increases in a process's memory footprint over time.
2. Yes, size() matters.
but it isn’t available in Raku.
Process-level stats.
Mainly “RSS”.
getrusage(2).
Acquiring & analyze data.
Raku tools.
3. RSS?
“Resident Set Size”
Virtual pages in physical memory.
Accessible without a page fault.
Non-resident VM may be swapped.
Requires a page fault to access.
4. Goals of memory managment:
Work within RSS.
Reduce page faults.
Avoid hard faults & swapping.
6. getrusage(2)
struct rusage
{
struct timeval ru_utime; /* user CPU time used */
struct timeval ru_stime; /* system CPU time used */
long ru_maxrss; /* maximum resident set size */
long ru_ixrss; /* integral shared memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims (soft page faults) */
long ru_majflt; /* page faults (hard page faults) */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* IPC messages sent */
long ru_msgrcv; /* IPC messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
POSIX
7. getrusage(2)
struct rusage
{
struct timeval ru_utime; /* user CPU time used */
struct timeval ru_stime; /* system CPU time used */
long ru_maxrss; /* maximum resident set size */
long ru_ixrss; /* integral shared memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims (soft page faults) */
long ru_majflt; /* page faults (hard page faults) */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* IPC messages sent */
long ru_msgrcv; /* IPC messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
Linux
8. getrusage(2)
struct rusage
{
struct timeval ru_utime; /* user CPU time used */
struct timeval ru_stime; /* system CPU time used */
long ru_maxrss; /* maximum resident set size */
long ru_minflt; /* page reclaims (soft page faults) */
long ru_majflt; /* page faults (hard page faults) */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
Only max
RSS.
9. getrusage(2)
struct rusage
{
struct timeval ru_utime; /* user CPU time used */
struct timeval ru_stime; /* system CPU time used */
long ru_maxrss; /* maximum resident set size */
long ru_minflt; /* page reclaims (soft page faults) */
long ru_majflt; /* page faults (hard page faults) */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
Fault
counts.
10. Viewing RSS
Telemetry module.
Takes periodic snapshots.
Allows inserting a label to track events.
Core with nqp.
Not synchronous with tasks.
Raku
11. Viewing RSS
ProcStats
Soon to be on CPAN.
Exports “dump-rusage”.
Differences from first sample.
Only output changes.
Track wallclock time.
Optional label.
12. :final Output all stats compared to first sample
ProcStats
sub dump-rusage
(
Bool() :$final = False,
Bool() :$first = $final,
Bool() :$force = $final,
Stringy() :$label = $final ?? 'Final' !! ''
)
is export( :DEFAULT )
13. :first Values compared to first sample (vs. last).
ProcStats
sub dump-rusage
(
Bool() :$final = False,
Bool() :$first = $final,
Bool() :$force = $final,
Stringy() :$label = $final ?? 'Final' !! ''
)
is export( :DEFAULT )
14. :force Write all stats (vs. only changed).
ProcStats
sub dump-rusage
(
Bool() :$final = False,
Bool() :$first = $final,
Bool() :$force = $final,
Stringy() :$label = $final ?? 'Final' !! ''
)
is export( :DEFAULT )
20. Acquire data
Times are sec + µsec, deal with them separately.
“Z=>” zips fields & values into a hash.
use nqp;
nqp::getrusage( my int @raw );
my ( $user_s, $user_us, $syst_s, $syst_us )
= splice @raw, 0, 4;
my %sample = FIELDS Z=> @raw;
%sample{ IGNORE } :delete;
21. Making time
my $utime
= ( $user_s + $user_us / 1_000_000 ).round( MICRO );
my $stime
= ( $syst_s + $syst_us / 1_000_000 ).round( MICRO );
user & system time begin as two ints.
Round gives reasonable precision in output.
22. Store baseline values.
state %last =
state %first =
(
|%sample,
:$wtime,
:$utime,
:$stime,
);
First is never updated.
Get a working “last” value on the first pass.
23. Store baseline values.
Flatten %sample into pairs.
state %last =
state %first =
(
|%sample,
:$wtime,
:$utime,
:$stime,
);
25. First is last at first.
After first last is last.
my %prior
= $first
?? %first
!! %last
;
26. What to compare?
Force reports full sample.
COMPARE limits keys compare to %prior & output.
my %curr
= ( $force || ! $passes )
?? %sample
!! do
{
my @diffs
= REPORT.grep( { %sample{$_} != %prior{$_} } );
@diffs Z=> %sample{ @diffs }
};
27. Write out one stat heading & value.
Compute column width once during execution.
sub write-stat ( Pair $p )
{
note
sprintf
'%-*s : %s',
once {FIELDS».chars.max},
$p.key,
$p.value
;
}
28. Write progressive value
Numerics compared to starting baseline.
Simplifies tracking code results.
sub write-diff ( Pair $p )
{
my $k = $p.key;
my $v = $p.value - %first{ $k };
write-stat $k => $v;
}
29. First pass writes all stats.
First pass has to report baseline values.
state $write = &write-stat;
30. First pass writes all stats.
First pass has to report baseline values.
After that report differences.
state &write = &write-stat;
...
write $stat;
...
once { &write = &write-diff };
36. Last steps
Up total count.
Store current sample for re-use.
++$passes;
%last = %sample;
once { &write = &write-diff };
37. Baseline usage
Bare for-loop
Shows overhead of rusage output.
#!/usr/bin/env Raku
use v6.d;
use FindBin::libs;
use ProcStats;
dump-rusage for 1 .. 1_000;
dump-rusage( :final );
62. What you see is all you get.
RSS, faults.
Per-process totals.
Not per structure.
Randomized trials simple in Raku.
Monitor results after specific operations.