Set up JobManager Memory

    The further described memory configuration is applicable starting with the release version 1.11. If you upgrade Flink from earlier versions, check the migration guide because many changes were introduced with the 1.11 release.

    The simplest way to set up the memory configuration is to configure the for the process. If you run the JobManager process using local execution mode you do not need to configure memory options, they will have no effect.

    As mentioned before in the , another way to set up the memory for the JobManager is to specify explicitly the JVM Heap size (jobmanager.memory.heap.size). It gives more control over the available JVM Heap which is used by:

    • User code executed during job submission (e.g. for certain batch sources) or in checkpoint completion callbacks

    The required size of JVM Heap is mostly driven by the number of running jobs, their structure, and requirements for the mentioned user code.

    Note If you have configured the JVM Heap explicitly, it is recommended to set neither total process memory nor total Flink memory. Otherwise, it may easily lead to memory configuration conflicts.

    The Flink scripts and CLI set the JVM Heap size via the JVM parameters -Xms and -Xmx when they start the JobManager process, see also .

    Configure Off-heap Memory

    The size of this component can be configured by option. This option can be tuned e.g. if the JobManager process throws ‘OutOfMemoryError: Direct buffer memory’, see the troubleshooting guide for more information.

    There can be the following possible sources of Off-heap memory consumption:

    • Flink framework dependencies (e.g. Akka network communication)

    Note If you have configured the and the JVM Heap explicitly but you have not configured the Off-heap memory, the size of the Off-heap memory will be derived as the minus the JVM Heap. The default value of the Off-heap memory option will be ignored.

    If you run Flink locally (e.g. from your IDE) without creating a cluster, then the JobManager memory configuration options are ignored.