While Java is praised for taking much of the burden of managing memory off software developers’ backs, it is not immediately obvious how it works under the hood. This post aims to explain the various technicalities of Java’s memory management in a concise way.

Flavors of JVM

There are many implementations of the Java Virtual Machine. The primary one is Hotspot, created by Oracle. In versions of Java earlier than 8 it had competition in the form of JRockit. It has since been merged into Hotspot, however. Nevertheless, you should be aware that details of Java’s memory management can vary between the different JVMs.

Heap vs Stack memory

Not to be confused with their data structure namesakes, the Heap and Stack are two areas of memory used by the Java Virtual Machine.

Stack memory

The stack is used for static memory allocation. Its objects are always stored directly in the memory and access to them is thus very fast. The amount of memory needed to allocate for the stack is calculated during compile time.

It contains methods, as well as local primitives and reference variables (pointers to objects in the heap).

Heap memory

The heap is used for dynamic memory allocation. Its objects’ lifecycle is managed through garbage collection and the allocation process is dependent upon the chosen heap allocator. While it is also stored directly in the memory, its allocation and deallocation is not as trivial as in the case of the stack, so access to it is not as fast.

It contains objects and their fields (including primitives defined as fields).

Setting the heap size

The size of the heap has a direct impact on allocation speed, how often garbage collection occurs and how long it takes.

A heap that is too small will fill up quickly, initiating garbage collection more often. Additionally, it is much more prone to fragmentation, slowing down object allocation over time.

A heap that is too large will take long to garbage collect. Furthermore, a heap that is larger than the available memory will be paged out to disk, dramatically slowing down the application due to long access times.

A larger heap, however, is usually more desirable than a smaller one. Thus, one should generally aim to make the heap size as large as possible, while making sure it stays within the available physical memory.

The default heap size values for Java 11 are:

• Initial heap size: 1/64 of physical memory
• Maximum heap size: 1/4 of physical memory

Note that the default values can vary depending on the version of Java that you are using.

The heap size can be set at runtime using two parameters:

• -Xms:<size>, which sets the initial and minimum heap size
• -Xmx:<size>, which sets the maximum heap size

An example for running SomeApplication with a 1GB (initial and maximum) heap size:

java -Xms:1g -Xmx:1g SomeApplication

It is recommended that both of these parameters are set to the same value if the optimal heap size for the application is already known.

Generations

It has been observed that the most recently created objects are also the most likely to stop being referenced quickly – this is known as infant mortality or generational hypothesis.

Stemming from this hypothesis, garbage collection in the JVM works using a process called generational collection. All objects managed by it are grouped into generations and are promoted as they age (or removed from memory if they are no longer being referenced). The objects’ age, in turn, is measured in how many garbage collections they have survived.

The heap memory is divided into two areas, known as generations – the Young Generation and the Old Generation. There used to be a third generation, stored off-heap, called the Permanent Generation. However, it has since been replaced with Metaspace.

Young Generation (Nursery)

The young generation, also called the Nursery, is where all new objects are created. It itself is divided into three areas: one Eden and two Survivor spaces. Most new objects are allocated in Eden.

Only one survivor space is active at a time (called From Space) and the other is empty (called To Space).

The reason for there being two Survivor spaces instead of one is memory fragmentation – as objects from one space are copied to the other, the holes left by removed objects are closed.

When the Young Generation fills up, it causes a minor garbage collection, where only the Young Generation is collected. Some of the surviving objects are then moved to the Old Generation, provided they have lived long enough.

Additionally, if the Survivor space runs out of memory, an object can be promoted to the Old Generation before it reaches the appropriate age in a process called premature promotion.

Old Generation (Tenured)

The old generation, also called the Tenured Generation, contains objects that have existed long enough to be promoted there. Once an object is promoted to the Old Generation, it stays there until the end of its life.

When the Old Generation fills up, it causes a major garbage collection, where the the Old Generation is collected.

Permanent generation (off-heap) and Metaspace (since Java 8)

In versions of Java up to and including Java 7, the permanent generation stored all the reflective data of the virtual machine, such as class and method objects. It was not stored on the heap but was still considered part of the generational system.

Since Java 8, however, it has been replaced with Metaspace.

Metaspace stores the same type of information as its predecessor, however it now has a dynamic size. Further details are outside of the scope of this post.

Garbage collection

Java uses an automatic garbage collector to manage all objects in the memory. While the developer decides when to create these objects, it is the responsibility of the garbage collector to dispose of them once they are no longer in use. It achieves this by tracking references to them – once an object instance is no longer referenced by other objects, it can be safely removed from memory.

A memory leak may still occur, however, if code is written in such a way that references to objects are kept longer than they are needed.

Note that generally all forms of garbage collection are stop-the-world events, which means that every time a garbage collection of any kind occurs, all threads in the application are paused. This is one of the reasons why a poorly designed application can start to behave sluggishly after a while. Some newer GC implementations, however, such as ZGC and Shenandoah are designed for low-latency garbage collection, making those events take less time.

It is also important to remember that the JVM cannot be forced to perform a garbage collection. A manual invocation of the System.gc() method is merely a suggestion for the JVM to perform garbage collection and is considered a bad practice.

Minor garbage collection

When an object is created, it is allocated in Eden space. When Eden is filled up, a minor garbage collection occurs on it, where unreferenced objects are removed from memory and the survivors are moved to an active Survivor space.

During the next minor garbage collection, the two Survivor spaces switch – the empty space becomes active, and objects from the other are moved to it.

An object’s age at this stage is determined by how many times it has been moved from one survivor space to the other. Those which are deemed old enough are moved to the Old Generation.

Major garbage collection

A major garbage collection removes all unreferenced objects in the Old Generation. Major collections usually take much longer to complete because of the large amount of objects involved.

Full garbage collection

A full garbage collection is one where both the Young Generation and Old Generation are collected. It can either happen when there is a change in region size (when the JVM increases the heap size) or when a minor GC triggers a major GC.

Types of garbage collectors

There is a large variety of garbage collectors available for Java and one should be at least aware of the existence of the most popular ones. The default garbage collector in Java 7 and 8 was the Parallel Collector. Since Java 9, the default collector is the Garbage-first (G1) collector.

Java 11 introduced Epsilon GZ and ZGC, while Java 12 introduced Shenandoah GC as optional choices, bringing the out-of-the-box choice of garbage collectors to 7:

• Serial Garbage Collector
• Parallel Garbage Collector
• Concurrent Mark-Sweep (CMS) Garbage Collector
• Garbage-First (G1) Garbage Collector
• Epsilon Garbage Collector
• Z garbage collector
• Shenandoah Garbage Collector

Serial Collector

The Serial Collector performs garbage collection using a single thread, making it very efficient due to lack of communication overhead between threads. It is suitable for machines with single-core CPUs as well as applications with data sets smaller than 100MB.

Parallel Collector (Throughput Collector)

Intended for applications with medium to large-sized data sets, this collector works in a similar way to the Serial Collector, except it uses multiple threads to speed up garbage collection.

Concurrent Mark-Sweep (CMS) Garbage Collector

Removed in Java 14, this collector was a predecessor to G1. It was a mostly-concurrent collector, which means it performed some (though not all) of the work concurrently to the application, instead of stopping it entirely.

Garbage-first Collector (G1)

The default compiler since Java 9, G1 is a mostly-concurrent collector, like CMS. Intended for machines with a large amount of memory, it prioritizes collecting regions which are most likely to have a large amount of reclaimable objects (hence the name). G1 replaces the deprecated Concurrent Mark-Sweep collector, offering more predictable garbage collection pauses and allowing users to specify desired pause targets.

Epsilon Collector

Introduced as an experimental feature in Java 11, it is a no-op collector, which means that it will not remove objects which are no longer in use, essentially allowing an application to run out of memory and crash. It is useful for measuring application performance or extremely short-lived jobs, removing the impact that garbage collection has on performance.

Z Collector (ZGC)

Introduced as an experimental feature in Java 11, this collector is intended for applications which require low-latency (pauses lasting less than 10 ms) or use a very large heap (on the scale of terabytes). Stop-the-world phases are limited to root scanning, so GC pause times do not increase with the size of the heap or the live set.

Shenandoah

Introduced in Java 12, this collector has similar characteristics to ZGC but uses a different implementation.

Azul C4

Offered by Azul Systems and available with their Zing JVM, Azul C4 is worth briefly mentioning because it is a “Continuously Concurrent Compacting Collector”, that is there are no stop-the-world events when using it.

Selecting a garbage collector

The default garbage collector should be sufficient for a majority of applications. Changing the garbage collector should be decided on a case by case basis or if the application has strict pause-time requirements. If you find that your application is not performing as well as it should, first try to adjust the heap size. For further fine-tuning, consider adjusting the generation sizes as well. If that doesn’t help, Oracle provides the following guidelines as a starting point for selecting a garbage collector:

• If the application has a small data set (up to approximately 100 MB), then select the serial collector with the option -XX:+UseSerialGC.

• If the application will be run on a single processor and there are no pause-time requirements, then select the serial collector with the option -XX:+UseSerialGC.

• If (a) peak application performance is the first priority and (b) there are no pause-time requirements or pauses of one second or longer are acceptable, then let the VM select the collector or select the parallel collector with -XX:+UseParallelGC.

• If response time is more important than overall throughput and garbage collection pauses must be kept shorter than approximately one second, then select a mostly concurrent collector with -XX:+UseG1GC or -XX:+UseConcMarkSweepGC.

• If response time is a high priority, and/or you are using a very large heap, then select a fully concurrent collector with -XX:UseZGC.

HotSpot Virtual Machine Garbage Collection Tuning Guide

Remember that these are meant to only be a starting point – performance is dependent upon the size of the heap, the amount and type of data being processed by the application and the system configuration (such as the speed and number of cores in the CPU) on which the JVM is being run.

Conclusion

The most important takeaway is that Java manages objects in memory using a Garbage Collector which performs generational collection on the heap. Although there are many to choose from, the default garbage collector since Java 9 is G1 and it should be sufficient for most use cases. Garbage collection works by periodically removing from memory objects which are no longer being referenced in code.

Objects in Java are first created in the Young Generation, where minor garbage collection occurs and are promoted to the Old Generation once they have survived enough garbage collections. A major garbage collection, which is performed on the Old Generation, occurs less frequently than a minor one and when a minor and major garbage collection occur at the same time, it is called a full garbage collection.

Leaving the heap size values to their defaults is discouraged, as it will most likely not utilize all of the available memory on the machine, potentially slowing down the application. Sane values should be found through analysis of both the application needs and the system configuration and thoroughly tested before making a final decision.

Java memory management is a surprisingly complex topic and its gradual evolution over the years, along with subtle differences between different JVM and garbage collector implementations, can cause further confusion. However, an understanding of this topic can help us create better software and diagnose any memory-related issues which might crop up during an application’s lifetime.

Photo by alfauzikri