Java is finally elastic! OpenJDK improvements and new features in Garbage Collection technology resulted in enhancing Java vertical scaling and resource consumption. Now JVM can promptly return unused memory and, as result it can go up and down automatically. In this presentation, we cover the main achievements in vertical scaling direction, as well as share peculiarities and tuning details of different GCs. Find out how to make your Java environments more elastic to follow the load and lower down the total cost of ownership at a large scale.

1. A STATE OF JAVA ELASTICITY
Tuning Java Efficiency

2. About Speaker
● Ruslan Synytsky
● CEO and co-founder of Jelastic PaaS
● Java Champion
● Two-times Duke’s Choice Award Winner
● Former lead of engineering team at
National Data Center (NDC) at National
Space Agency of Ukraine
● @siruslan

3. Agenda
● Java Memory Usage Problems and Use Cases
● Recent Improvements in OpenJDK
● Testing Elasticity with Different Garbage Collectors
● Upcoming Improvements in OpenJDK

4. Unreleased Heap Memory
The Problem Symptoms
Over-Allocation and Underutilization

5. Java Memory Usage Pain Points
Q: Return memory from JVM to OS
Q: Java VM - does the freed memory return to the OS?
Q: More flexible memory management
Q: Force JVM to free memory
Q: Why Java does not release memory?
Q: Does GC release back memory to OS?
Q: Why does this java process not release memory?
...

6. Large Java Memory Requirements
WHAT IS THE MOST CHALLENGING ASPECT OF WORKING WITH JAVA EE?
Jakarta EE Developer Survey 2018
“The most widely acknowledged issue
when employing with Java EE is large
memory requirements (40%)”

7. Reasons to Seek Java Elasticity Getting resources from common
Desktop
Applications,
IDEs
Scheduled
tasks for data
processing
Cloud cost
saving with
pay-per-use
NEED FOR ELASTICITY
Automated scaling of
resources on the fly without
JVM restart and downtimes
Dynamic
stateful
workloads
Right-sizing
problem with
containers

8. Transition from Monolith to Microservices

9. Resource Limit vs Real Usage in VM

10. Resource Limit vs Real Usage in Container

11. Elasticy Use Case in the Cloud

12. Elasticy Use Case in the Cloud
Pool of Resources

13. Elasticy Use Case in the Cloud

14. Elasticy Use Case in the Cloud

15. Elasticy Use Case in the Cloud

16. Elasticy Use Case in the Cloud

17. Real Statistics of Resource Consumption with Containers

18. Using automatic vertical scaling, cloud provides can offer economically
advantageous pricing based on the actual resource consumption
Forbes - Deceptive Cloud Efficiency: Do You Really Pay As You Use?
Pay-As-You-Go Pay-per-Use
Pay-As-You-Go vs Pay-per-Use

19. Understanding JVM Footprint

20. Understanding JVM Footprint
MAXIMUM MEMORY USAGE
[-Xmx] + [-XX:MaxMetaspaceSize] + [-XX:MaxDirectMemorySize] + Num_Of_Threads * [-Xss]
Useful discussion at SoW Java using much more memory than heap size and presentation Memory Footprint of a Java Process

21. -Xmx=[25% of OS Total Memory]
Useful options to adjust defaults in % -XX:MaxRAMPercentage
-XX:MaxMetaspaceSize=[unlimited]
-XX:MaxDirectMemorySize=[-Xmx]
-Xss[1m]
Alternative syntax -XX:ThreadStackSize=[1m]
Specify these options explicitly if you face related OutOfMemoryError
Default Limits (Depend on The Platform!)

22. Understanding of the OutOfMemoryError Exceptions
OutOfMemoryError exception is usually thrown when there is insufficient
space to allocate an object in the Java heap or insufficient native memory to
support the loading of a Java class
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html
● java.lang.OutOfMemoryError: Java heap space
● java.lang.OutOfMemoryError: GC Overhead limit exceeded
● java.lang.OutOfMemoryError: Requested array size exceeds VM limit
● java.lang.OutOfMemoryError: Metaspace
● java.lang.OutOfMemoryError: Compressed class space
● java.lang.OutOfMemoryError: reason stack_trace_with_native_method
● java.lang.OutOfMemoryError: request size bytes for reason. Out of swap space?

23. An Example of OutOfMemoryError Exception
Used Heap > Xmx

24. An Example of OOM Killer Situation (Even Worse)
oom_kill is a job that helps to sacrifice one or
more processes in order to free up memory
for the system

25. The Heap Vertical Scaling Problem Symptoms
Unreleased Heap Memory

26. Calling Full GC Periodically (Workaround)
https://github.com/jelastic-jps/java-memory-agent
As compacting GC cycles are not triggered automatically, we execute
them explicitly by injecting an agent which monitors the memory usage
and calls System.gc() periodically:
-javaagent:jelastic-gc-agent.jar=period=300,debug=true
Consider enabling -XX:+ExplicitGCInvokesConcurrent

27. G1 and Full GC
java -XX:+UseG1GC -Xmx2g -jar app.jar
https://github.com/jelastic/java-vertical-scaling-test

28. Simple Jenkins Example
After Full GC call initiated by Jelastic GC agent the memory consumption goes down to 0.5G. If we
disable the agent then the memory usage at the idle JVM will stay at 3G forever which is a significant
waste of resources. If you put additional load on Jenkins the picture will look even worse.
Container limit - 8G, max heap limit (Xmx) - 6.5G and the memory consumption after the simple initial
sign in action goes to 3G.

29. Timely Reduce Unused Committed Memory (JEP 346)
Make the G1 garbage collector automatically give back Java heap memory
to the operating system when idle
● -XX:G1PeriodicGCInterval=[milliseconds]
● -XX:G1PeriodicGCSystemLoadThreshold=[float]
● -XX:+G1PeriodicGCInvokesConcurrent
JEP 346: Promptly Return Unused Committed Memory from G1
java -Xmx2g -XX:+UseG1GC -XX:G1PeriodicGCInterval=900k
-XX:G1PeriodicGCSystemLoadThreshold=0.6 -jar app.jar
Available from Java 12

30. Immediately Improved Heap Elasticity
Automatically Released Heap

31. Community Recognition

32. G1PeriodicGCSystemLoadThreshold in Docker Container
Using LXCFS to Improve Container Resource Visibility
For getting correct loadavg while running in Docker container use https://github.com/lxc/lxcfs/

33. Running GC
Tests in Jelastic

34. Load Testing Logic
https://github.com/jelastic/java-vertical-scaling-test/blob
/master/src/com/jelastic/verticalscaling/Load.java#L50
java [OPTIONS] -jar app.jar <sleep>

35. Auto Testing Package
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest.yml

36. G1 Collector (-XX:+UseG1GC)
The Garbage-First (G1) is a server-style Garbage Collector for
multiprocessor machines with a large amount of memory. The heap is
partitioned into fixed-sized regions and G1 tracks the live data in those
regions. When Garbage Collection is required, it collects from the regions
with less live data first.
● 2004, Sun Microsystems
JEP 346: Promptly Return Unused Committed Memory from G1

37. G1
-Xmx3g -XX:+UseCompressedOops -XX:+UseG1GC -XX:G1PeriodicGCInterval=1k

38. Shenandoah GC (-XX:+UseShenandoahGC)
Shenandoah GC is a concurrent garbage collector for the JVM. GC tries to
perform most of the activities in parallel without interrupting application
performance. Such parallelism makes “stop-the-world” (STW) pauses
extremely short. Another inherent advantage is an efficient work with small
and large heaps with no impact on STW pauses’ length.
● 2014, Christine H. Flood, Red Hat
https://wiki.openjdk.java.net/display/shenandoah/Main#Main-Heuristics

39. -Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
Shenandoah

40. ZGC (-XX:+UseZGC)
ZGC is low latency scalable garbage collector. Designed for use with
applications that require a large heap and low latency. It uses a bunch of one
generation and performs most (but not all) garbage collection in parallel with
uninterrupted application work. This greatly limits the impact of garbage
collection on your application response time.
● 2018, Per Liden, Oracle
JEP 351: ZGC: Uncommit Unused Memory - available from JDK 13 Release

41. -Xmx3g -XX:+UseZGC -XX:ZUncommitDelay=1 -XX:ZCollectionInterval=30
ZGC @ Oracle OpenJDK

42. C4 (-XX:+UseZST)
The C4 (Continuously Concurrent Compacting Collector) is an
updated generational form of the Azul Pauseless GC Algorithm and is
the default collector of Zing®. C4 differentiates itself from other
generational garbage collectors by supporting simultaneous –
generational concurrency: the different generations are collected using
concurrent (non-stop-the-world) mechanisms that can be
simultaneously and independently active. Unlike other algorithms, it is
not ‘mostly’ concurrent, but fully concurrent, so it never falls back to a
stop-the-world compaction.
● 2010, Gil Tene, Azul Systems

43. -Xmx500m -XX:+UseZST
C4 @ Zing
pmem.conf -> cgroups enabled

44. ConcMarkSweep GC (-XX:+UseConcMarkSweepGC)
ConcMarkSweep GC collector is designed for applications that prefer
shorter garbage collection pauses and which can afford to share processor
resources with the garbage collector while the application is running. It
makes sense to use such a collector when applications requirements for time
garbage collection pauses are low.
● 2004, Sun Microsystems

45. -Xmx3g -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
+ 4 calls of Full GC via jcmd <pid> GC.run
ConcMarkSweep (Deprecated since Java 9)
Consider using -XX:-ShrinkHeapInSteps

46. Serial GC (-XX:+UseSerialGC)
Serial GC performs garbage collection in a single thread and has the lowest
consumption of memory among all GC types but, at the same time, it makes
long pauses that can lead to application performance degradation.
● 2004, Sun Microsystems

47. -Xmx3g -XX:+UseCompressedOops -XX:+UseSerialGC
+ 4 calls of Full GC via jcmd <pid> GC.run
Serial
Consider using -XX:-ShrinkHeapInSteps

48. OpenJ9
OpenJ9 uses the Generational Concurrent (-Xgcpolicy:gencon) policy by
default, which is best suited to transactional applications that have many
short lived objects. Alternative policies are available, including those that
cater for applications with large Java heaps (-Xgcpolicy:balanced),
applications that are sensitive to response-time (-Xgcpolicy:metronome), or
applications that require high application throughput (-Xgcpolicy:optthruput).
● 2017, Eclipse Foundation

49. -Xmx3g -XX:+UseCompressedOops
-XX:+IdleTuningCompactOnIdle -XX:+IdleTuningGcOnIdle -XX:IdleTuningMinIdleWaitTime=1
-Xjit:waitTimeToEnterDeepIdleMode=1000
Bash command to check the real usage
while true
do
pid=$(pgrep -f java | tail -n1)
used=$(ps -orss --no-headers --pid $pid)
echo "scale=2 ; $used / 1024/1024" | bc
sleep 1
done
OpenJ9 (Not Enough Tuning and Testing Experience*)
Inconsistent behaviour with -XX:+IdleTuningGcOnIdle, mem not released back to OS on Idle

50. Parallel GC (-XX:+UseParallelGC)
Parallel GC is a “stop-the-world” multithreaded Garbage Collector similar to
the serial collector. The primary difference is that multiple threads are used
to speed up garbage collection. By default, both minor and major collections
are executed in parallel to further reduce garbage collection costs.
● 2000, Sun Microsystems

51. -Xmx3g -XX:+UseCompressedOops -XX:+UseParallelGC
+ periodical jcmd <pid> GC.run
Parallel

52. Main Points of Elastic Vertical Scaling
A - initial usage
B - maximum usage
C - growth speed
D - duration before release
E - release speed
F - minimum usage after release
Different GCs provide different results and fine tuning options

53. Running GC
Tests in Kubernetes

54. Auto Testing Package for Kubernetes
https://github.com/jelastic/java-vertical-scaling-test/blob/master/manifest-k8s.yaml

55. Load Testing Logic
java [OPTIONS] -jar app.jar <sleep> <mode>
where
sleep - 100
mode - 2
https://github.com/jelastic/java-vertical-scaling-test/blob/ma
ster/src/com/jelastic/verticalscaling/Load.java#L64

56. G1 in Kubernetes

57. Shenandoah in Kubernetes

58. ZGC @ Oracle OpenJDK in Kubernetes

59. Joint Comparison - Several Load Cycles
RAM CPU

60. Running GC
Tests with Jakarta EE

61. ● simple .war artifact deployed with GlassFish 5
● JSP that sets 1MB attribute in session
● 1 min session timeout
● https://github.com/jelastic/java-vertical-scaling-test/tree/payara/web
app
Load Testing Logic
Load test GETs webapp endpoint n times
for i in {1..n}; do curl -s localhost:8080 > /dev/null; done

62. GlassFish with Shenandoah
-Xmx3g -XX:+UseCompressedOops -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=compact
for i in {1..1000}; do curl -s localhost:8080 > /dev/null; done

63. Resizing Xmx
On the Fly

64. Dynamic Max Heap Resizing - Solving Right Sizing Problem
Restart for Xmx Resize

65. -XX:SoftMaxHeapSize @ ZGC
SoftMaxHeapSize is set for the GC to
strive not to grow heap size beyond the
specified size unless it is highly needed:
● to keep the heap footprint down, while
maintaining the capability to deal with a
temporary increase in heap space
requirement
● with lots of margin, to increase confidence
that you will not run into an allocation stall
because of an unforeseen increase in
allocation rate
Using -XX:SoftMaxHeapSize

66. -XX:G1PeriodicGCInterval and -XX:SoftMaxHeapSize
G1PeriodicGCInterval (G1) / ZCollectionInterval (ZGC) is a time-based
solution which has no direct impact on how much the heap will grow during
the given interval.
SoftMaxHeapSize is a size-based solution which controls how large the heap
can grow, but has no direct relation with uncommit (returning unused
memory back to OS)
Both options can be used, whichever condition is met first will trigger a GC.
ZGC (Java13+) https://bugs.openjdk.java.net/browse/JDK-8222181
G1, Shenandoah (TBD) https://bugs.openjdk.java.net/browse/JDK-8236073

67. -Xsoftmx @ OpenJ9
https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/openj9/xsoftmx/index.html
Runtime adjustable heap size (-Xsoftmx) allows to adjust heap size
dynamically and take advantage of hot-add of memory.
You can set this option on the command line, then modify it at run time by
using the com.ibm.lang.management.MemoryMXBean.setMaxHeapSize().
This option can be useful in virtualized or cloud environments, for example,
where the available memory might change dynamically to meet business
needs.
By default, -Xsoftmx is set to the same value as -Xmx.

68. C4 is fully elastic and can return all empty pages to the OS after each GC cycle.
However, C4 sticks to the Xmx it was given, and avoid doing heavy elastic memory dance,
since relinquishing memory mappings and reestablishing them on Linux kernels is
bandwidth-limited in practice by the rate of page mapping invalidation the kernel can
handle.
C4 goes above Xmx rather than go between Xms and Xmx. JavaMemMax option controls
the true maximum. In the future it will allow both scenarios where above-Xmx is allowed
and where above-Xmx is prohibited.
Two modes:
● Contingency (default mode) - goes above Xmx if it absolutely has to and will work
hard to collect and stay below Xmx.
● Insurance (best effort elasticity) - borrows available memory and goes above Xmx in
order to delay GC whenever possible.
JavaMemMax @ С4 + ZST (Zing System Tools)

69. Xmx can be set higher than the Container Limit. Also both
SoftMaxHeapSize and Container Limit can be adjusted on the fly
without the need to restart JVM or container.
At the moment the heap size can go beyond SoftMaxHeapSize and
there is no guarantee on how much the heap will grow other than up
to Xmx.
The problem arises when Used Heap comes close to Container Limit.
Most likely the JVM will be killed by the OOM Killer as it exceeds the
amount of memory available in the container.
JEP Draft: Dynamic Max Memory Limit @ G1
We can introduce manageable HardMaxHeapSize. In this case JVM will throw OOM Error (losing
one operation) instead of OOM Kill (losing whole JVM).
Xmx
Container Limit
SoftMaxHeapSize

70. JEP 387: Elastic Metaspace (Target Release Java 16)
To deal with potential problems involving virtual memory fragmentation or uncommit speed, we will add a new
production command-line option to control metaspace reclamation behavior:
-XX:MetaspaceReclaimPolicy=(balanced|aggressive|none)
■ balanced: Most applications should see an improvement in metaspace memory footprint while the
negative effects of memory reclamation should be marginal. This mode is the default, and aims for
backward compatibility.
■ 'aggressive': Offers increased memory-reclamation rates at the cost of increased virtual-memory
fragmentation.
■ 'none': Disables memory reclamation altogether.
Applications that use many small class loaders may suffer from
unreasonably high metaspace usage. Applications with heavy class
loading and unloading activity can thus accrue a lot of unused space in
the metaspace freelists. That space can be returned to the operating
system to be used for other purposes if it is not fragmented, but that’s
often not the case.

71. Keep Only Best Java Memories
Learn More
Get In Touch
@jelastic
info@jelastic.com