Glossary

    A distributed system consisting of (typically) one Flink Master and one or more processes.

    Event

    An event is a statement about a change of the state of the domain modelled by theapplication. Events can be input and/or output of a stream or batch processing application.Events are special types of .

    ExecutionGraph

    see

    Function

    Functions are implemented by the user and encapsulate theapplication logic of a Flink program. Most Functions are wrapped by a corresponding.

    Instance

    The term instance is used to describe a specific instance of a specific type (usually or Function) during runtime. As Apache Flink is mostly written inJava, this corresponds to the definition of Instance or Object in Java. In the context of ApacheFlink, the term parallel instance is also frequently used to emphasize that multiple instances ofthe same or Function type are running in parallel.

    A Flink Job is the runtime representation of a Flink program. A Flink Job can either be submittedto a long running Flink Session Cluster or it can be started as aself-contained .

    JobGraph

    JobManagers are one of the components running in the Flink Master. A JobManager isresponsible for supervising the execution of the of a single job. Historically, thewhole Flink Master was called JobManager.

    Logical Graph

    A logical graph is a directed graph describing the high-level logic of a stream processing program.The nodes are Operators and the edges indicate input/output-relationships ordata streams or data sets.

    Managed State

    Managed State describes application state which has been registered with the framework. ForManaged State, Apache Flink will take care about persistence and rescaling among other things.

    The Flink Master is the master of a . It contains three distinctcomponents: Flink Resource Manager, Flink Dispatcher and one Flink JobManagerper running .

    Operator

    Node of a . An Operator performs a certain operation, which isusually executed by a Function. Sources and Sinks are special Operators for dataingestion and data egress.

    Operator Chain

    An Operator Chain consists of two or more consecutive Operators without anyrepartitioning in between. Operators within the same Operator Chain forward records to each otherdirectly without going through serialization or Flink’s network stack.

    Partition

    Physical Graph

    A physical graph is the result of translating a for execution in adistributed runtime. The nodes are Tasks and the edges indicate input/output-relationshipsor of data streams or data sets.

    Record

    Records are the constituent elements of a data set or data stream. andFunctions receive records as input and emit records as output.

    A long-running Flink Cluster which accepts multiple forexecution. The lifetime of this Flink Cluster is not bound to the lifetime of any Flink Job.Formerly, a Flink Session Cluster was also known as a Flink Cluster in session mode. Compare toFlink Application Cluster.

    State Backend

    For stream processing programs, the State Backend of a Flink Job determines how its is stored on each TaskManager (Java Heap of TaskManager or (embedded)RocksDB) as well as where it is written upon a checkpoint (Java Heap ofFlink Master or Filesystem).

    Sub-Task

    A Sub-Task is a Task responsible for processing a ofthe data stream. The term “Sub-Task” emphasizes that there are multiple parallel Tasks for the sameOperator or .

    Task

    Node of a . A task is the basic unit of work, which is executed byFlink’s runtime. Tasks encapsulate exactly one parallel instance of anOperator or .

    Transformation

    A Transformation is applied on one or more data streams or data sets and results in one or moreoutput data streams or data sets. A transformation might change a data stream or data set on aper-record basis, but might also only change its partitioning or perform an aggregation. WhileOperators and ) are the “physical” parts of Flink’s API,Transformations are only an API concept. Specifically, most - but not all - transformations areimplemented by certain Operators.