[FIXED] Does G1GC release back memory to the OS even if Xms = Xmx?

Issue

After reading some answers like this and JEP-346, I have realised that the G1 does release memory back to the OS.

However does it release memory back to the OS, even to the point that current memory use can drop below the initial heap memory (i.e before this JEP, in my case JDK11)?

Assume I have a Java 11 VM running with Xms and Xmx set as 5GB, on a 8GB RAM, however I am consuming only around 1GB. Will G1 release enough memory back to the OS?

I didn't find any documentation anywhere which says that the G1 is restricted to releasing keeping the Xms threshold in mind.

I am observing this in Production, the MemAvailable keeps on decreasing till a point, and then after a GC, it jumps upto close to 30-35% on an 8GB box. So I am assuming it is releasing memory, which is why the MemAvailable is jumping back up.

Also what does it exactly mean to release memory to the OS, is it calling free/unmap ?

Solution

Note: I have deleted my previous answer and have studied the sources (also build my own JVM just to find out this particular moment), here is the answer.

The short answer

JVM 11 version (at the moment), will not go below Xms when making the heap smaller.

The long answer

The absolute truth is in the source code. And here is the decision taken to shrink the heap or not. A few lines below, you can see that if we enter that if, there will be a log statement:

Attempt heap shrinking (capacity higher than max desired capacity after Full GC).

So in essence, if we can understand two parameters : capacity_after_gc and maximum_desired_capacity - we can solve this mystery. In general, capacity_after_gc is not something easy to grasp; mainly because it depends on how much garbage there was and how much the current GC could reclaim. For simplicity, I am going to write some code that does not generated any garbage, so that this value is constant.

In such a case, we only need to understand maximum_desired_capacity.

A few lines above, you can see that this is computed as :

maximum_desired_capacity =  MAX2(maximum_desired_capacity, min_heap_size);

Unfortunately, this is where it gets tricky, because it's a lot of code to follow and understand to really see how these ergonomics are set; especially since they depend on various arguments that the JVM has been started with.

For example min_heap_size is set as:

// If the minimum heap size has not been set (via -Xms),
// synchronize with InitialHeapSize to avoid errors with the default value.

Notice, that they even refer to -Xms as minimum; though the documentation says it's initial. You can also notice that it further depends on another two properties :

 reasonable_minimum , InitialHeapSize

This will be difficult to explain further; that is why I will not. Instead, I will show you some simple proof (I did go through the majority of that code...)

---

Suppose you have this very simple code:

public class HeapShrinkExpand {

    public static void main(String[] args) throws InterruptedException {

        for (int i = 0; i < 10; i++) {
            Thread.sleep(500);
            System.gc();
        }
    }
}

And I run it with:

-Xmx22g 
-XX:InitialHeapSize=1g
"-Xlog:heap*=debug" 
"-Xlog:gc*=debug" 
"-Xlog:ergo*=debug" 

In logs, I will see:

[0.718s][debug][gc,ergo,heap   ] GC(0) Attempt heap shrinking (capacity higher than max desired capacity after Full GC). Capacity: 1073741824B occupancy: 8388608B live: 1018816B maximum_desired_capacity: 27962026B (70 %)
[0.719s][debug][gc,ergo,heap   ] GC(0) Shrink the heap. requested shrinking amount: 1045779798B aligned shrinking amount: 1044381696B attempted shrinking amount: 1044381696B

This tells you some stats around how much shrinking is desired, what is the current capacity, etc. The next line will show you how much the heap has gone down, actually:

[0.736s][debug][gc,ihop] GC(0) Target occupancy update: old: 1073741824B, new: 29360128B

The heap did shrink, down to around 29MB.

If I add a single JVM start-up flag: -Xms10g, those GC logs that are responsible for showing how much the heap was shrank to; will not be present anymore.

And in fact if I run my own JMV (with some logging enabled), those two values: capacity_after_gc and maximum_desired_capacity will always have the same values; meaning that if statement will never be entered and heap will never go below -Xms.

---

I have run the same code with JDK-13 and, while the shrinking logs are present there (when -Xms is given as an argument), the underlying heap stays at the -Xms, still. What I find even more interesting is that under java-13, trying to run with:

 -Xmx22g -Xms5g -XX:InitialHeapSize=1g

will correctly error out with:

Incompatible minimum and initial heap sizes specified

Answered By - Eugene
Answer Checked By - Timothy Miller (JavaFixing Admin)