share the common java memory problem cases solutions,including: 1. java.lang.OutOfMemoryError 2. full gc frequently 3. cms gc error: promotion failed or concurrent mode failure

1. bluedavy
2012-03

2.  Java Runtime Data Area
-Xss
Local Vars Native method Stack
Operator -XX:PermSize –
PC Method Area XX:MaxPermSize
Stack Frame
Method Stack Heap -Xms -Xmx

3.  Oracle JDK Java Heap
-Xmn New Generation
Eden S0 S1 Old Generation
-XX:SurvivorRatio

4.  Oracle JDK startup options for memory manage
 -Xms –Xmx –Xmn –XX:PermSize –XX:MaxPermSize
 -XX:SurvivorRatio
 -XX:+UseParallelGC || -XX:+UseParallelOldGC
▪ -XX:MaxTenuringThreshold -XX:ParallelGCThreads
▪ -XX:-UseAdaptiveSizePolicy
 -XX:+UseConcMarkSweepGC
▪ -XX:CMSInitiatingOccupancyFraction -
XX:+UseCMSInitiatingOccupancyOnly
 -XX:+DisableExplicitGC –
XX:+ExplicitGCInvokesConcurrent

5.  Startup options or tools for memory problem solution
 -XX:+PrintGCDateStamps –XX:+PrintGCDetails -Xloggc
 -XX:+HeapDumpOnOutOfMemoryError
 -XX:+HeapDumpBeforeFullGC
 -XX:+PrintFlagsFinal（JDK 6u21+）
 for CMS GC
▪ -XX:+PrintPromotionFailure -XX:+CMSDumpAtPromotionFailure
 jinfo
 jstat
 jmap
 MAT
 btrace
 google perf-tools

6.  OOM
 java.lang.OutOfMemoryError: {$reason}
 Full GC Frequently
 CMS GC
 promotion failed
 concurrent mode failure

9. -Xms4g –Xmx4g –Xmn2560m

12. Then to find who use Deflater.init

13.  java.lang.OutOfMemoryError: {$reason}
 GC overhead limit exceeded
 Java Heap Space
 Unable to create new native thread
 PermGen Space
 Direct buffer memory
 request {} bytes for {}. Out of swap space?

14.  GC overhead limit exceeded
Java Heap Space
 if code didn’t catch OutOfMemoryError,thread
will exit;
 slow response or no response at all
▪ because gc did frequently
 Out of Swap
 Java process crash

15.  GC overhead limit exceeded
Java Heap Space
 monitor app log & gc log
 heap dump or jmap –histo|-dump when oom
 mat analyze heap dump
 app developer or btrace to find where cause

16.  Out of swap
 crash log
 google-perftools
 then btrace to find where cause
 ps: maybe u can try delete -XX:+DisableExplicitGC
if exist,reason can be found in this link。

17.  use ThreadLocal carefully;
 limit Collection/StringBuilder etc. size；
 limit batch ops size;
 limit the data size return by database;
 avoid infinite loop;

18.  OOM
 java.lang.OutOfMemoryError: {$reason}
 Full GC Frequently
 CMS GC
 promotion failed
 concurrent mode failure

19.  Most Cases
 because memory consumes too much
 a special case
 a special frequently GC Case

20.  slow response or no response at all

21.  According gc log to see if the reason is
consuming too much memory
 if yes then add -XX:+HeapDumpBeforeFullGC or
write a shell file to dump memory when full gc;
 if no then use pstack & source code to see why full
gc executes;

22.  OOM
 java.lang.OutOfMemoryError: {$reason}
 Full GC Frequently
 CMS GC
 promotion failed
 concurrent mode failure

23.  promotion failed Case I
 363439.937: [Full GC 363439.938: [ParNew
(promotion failed): 523840K->523840K(523840K),
0.3404490 secs] 1690606K->1712399K(1834560K),
0.3412880 secs]
 Taobao JDK help to find the reason
▪ ==WARNNING== allocating large array: thread_id[0x00002aaac2d85800],
thread_name[ajp-0.0.0.0-8009-48], array_size[782611408 bytes],
array_length[195652847 elememts]

24.  promotion failed Case II
 1768.432: [GC 1768.432: [ParNew (promotion
failed): 1572827K->1572827K(1572864K),
0.2845480 secs]
 1972.208: [CMS: 10545160K-
>5853155K(14680064K), 8.5355320 secs]
12105682K->5853155K(16252928K), [CMS Perm :
20907K->20873K(34952K)], 212.3113910 secs]
[Times: user=53.38 sys=22.45, real=212.28 secs]
 because cms gc fragmention

25.  four cases
 promotion failed case III
▪ because CMSInitiatingOccupancyFraction too high...
 concurrent mode failure case I
▪ because CMSInitiatingOccupancyFraction too high...
 concurrent mode failure case II
▪ because old gen is too small
 concurrent mode failure case III
▪ because permgen

26.  CMS GC problem will cause serial full gc,so
the app response time will be very slow...

27.  According gc log to see if the reason is
because CMSInitiatingOccupancyFraction is
too high,if yes then adjust the option;
 if no,then
 promotion failed
▪ if because memory ran out,then dump memory;
▪ if because cms gc fragmention,currently we only can do
is execute jmap –histo:live at regular time;
 concurrent mode failure
▪ the same as promotion failed...

28.  OOM
 java.lang.OutOfMemoryError: {$reason}
 Full GC频繁
 CMS GC
 promotion failed
 concurrent mode failure

29.  Young GC slow
 CMS GC Concurrent-Mark causes app stop...
 ...