Working Directory

    The working directories for the Flink processes are:

    • JobManager working directory:
    • TaskManager working directory: <WORKING_DIR_BASE>/tm_<TM_RESOURCE_ID>

    with <WORKING_DIR_BASE> being the working directory base, <JM_RESOURCE_ID> being the resource id of the JobManager process and <TM_RESOURCE_ID> being the resource id of the TaskManager process.

    It is also possible to configure a JobManager and TaskManager specific <WORKING_DIR_BASE> via process.jobmanager.working-dir and process.taskmanager.working-dir respectively.

    The JobManager resource id can be configured via jobmanager.resource-id. If not explicitly configured, then it will be a random UUID.

    Flink processes will use the working directory to store the following artifacts:

    • Local state if state.backend.local-recovery is enabled
    • RocksDB’s working directory

    The working directory can be used to enable local recovery across process restarts (FLIP-201). This means that Flink does not have to recover state information from remote storage.