Java in Containers - Memory

Containerization in Java projects has been a widely adopted virtualization technology in many enterprises since Docker became popular. Containerization eases the application development life cycle, increases reliability, portability, scalability, and security, and makes the system less prone to configuration errors.

Java applications can utilize container platforms and orchestrations like Docker, Docker Swarm, Kubernetes (K8s), Red Hat OpenShift, Amazon Elastic Container Service (Amazon ECS), and many more.

Containerization of Java applications is as simple as creating a dedicated file (named Dockerfile or Containerfile) containing instructions required to build a container image. After the container image is ready, the team deploys the image and sets resource limits for a container.

FROM eclipse-temurin:21-jdk-alpine
WORKDIR /app
COPY target/app.jar app.jar

EXPOSE 8080

ENTRYPOINT ["java","-jar","app.jar"]

It is as simple as that! We have a running service!

However, many teams forget about providing configuration for JVM in containers. The default configuration might not be suitable for running a Java application in a container. Therefore, it could cause performance downgrade or waste of resources.

Memory

Java memory layout divides "available" memory into pools. Memory pools are divided into two groups: heap and non-heap.

• Heap is storing your application objects. The heap is always larger than the non-heap pool and may take even gigabytes of memory. Heap is further divided into:
  • Young generation
    • Eden
    • S0 (Survivor 0)
    • S1 (Survivor 1)
  • Old generation (also known as Tenured space)
• Non-Heap memory is everything else, like Metaspace, JIT Code Cache, GC, symbol, shared class space, compiler, logging, arguments, modules, internal, other, threads, and native allocations.

Default values for common pools

Minimum heap size - 8MB or 1/64th of the system memory.

Maximum heap size - The OpenJDK (server VM; since Java 8) defaults to up to 25% of system memory (RAM) if a system has more than 512 MB of system memory. Otherwise, Java will allocate up to 50% of system memory for the heap.

$ docker container run -it -m 128MB openjdk:11-jdk java -XshowSettings:VM -version
VM settings:
    Max. Heap Size (Estimated): 61.88M
    Using VM: OpenJDK 64-Bit Server VM

$ docker container run -it -m 1024MB openjdk:11-jdk java -XshowSettings:VM -version
VM settings:
    Max. Heap Size (Estimated): 247.50M
    Using VM: OpenJDK 64-Bit Server VM

The following table presents the correlation between memory, minimum heap size, and maximum heap size.

Heap size can be set manually using one of the following Java command line options:

• -Xms4g - Sets minimum heap size (has priority over other heap options)
• -Xmx4g - Sets maximum heap size (has priority over other heap options)
• -XX:InitialRAMPercentage=50.0 - (available since Java 8u191) Computes the initial heap size based on the percentage of system memory. Similar to -Xms.
• -XX:MinRAMPercentage=60.0 - (available since Java 8u191) Computes maximum heap size based on the percentage of system memory if system memory is less than ~250MB. Similar to -Xmx.
• -XX:MaxRAMPercentage=60.0 - (available since Java 8u191) Computes maximum heap size based on the percentage of system memory if system memory is more than ~250MB. Similar to -Xmx.

For other Java command line options, you can utilize VM Options Explorer from Chris Newland who wrote a great tool that indexes Java options (defaults and other metadata) in various JDK codebases.

Another issue regarding the Java memory and containers is the container integration feature, which was introduced and enabled by default in Java 10 by providing a new command line option called -XX:+UseContainerSupport. Until Java 9 the JVM did not recognize memory or CPU limits set by the container. Therefore, JVM tried to utilize more memory than was available to the container causing OOMEs.

To illustrate that problem let's assume a host, that is running containers, has 16 gigabytes of system memory and the container's memory is limited to 2 gigabytes. Before Java 10, the JVM by default should allocate 25% of system memory, that is 512 megabytes, for maximum heap size but allocate 25% of the host system memory and that would be 4 gigabytes for maximum heap size in this case.

However, container integration can still be a problem with cgroup v2 because older Java versions use cgroup v1. Support for cgroup v2 was introduced in Java 15.

Metaspace - It is used to manage memory for class metadata. Class metadata are allocated when classes are loaded. The default max size is unlimited. It can be restricted by using the command line option -XX:MaxMetaspaceSize=256m.

JIT Code Cache - The default value for ReservedCodeCacheSize is 240MB and 48MB with disabled tiered compilation (-XX:-TieredCompilation) since Java 8.

Threads' stacks - The default max size value is 1024k (1MB).

Container memory

The recommended approach for running containers is to run one process in a container at best. It simplifies memory management, horizontal scaling, security, patching and upgrading, and many more.

Linux process memory layout divides memory into segments, like stack, heap, BSS, Data, and Text. The Java memory layout may look similar to the process memory layout. But it is not the same. Java memory allocates as a heap of the process memory.

Conclusions

1. Java's maximum heap size should not be equal to the total available system memory. Java application not only uses heap but also utilizes non-heap memory.
2. A good starting point would be to assign up to 75% of a container memory to a Java application heap.
3. Providing command line options for heap will reduce unutilized memory as the Java application has predefined non-optimal default values for memory pools.
4. Double-check the JDK/JRE version for container platform compatibility. Consider upgrading it to versions that support your environment.
5. Monitor and profile your application.