Table API & SQL

    The Table API and the SQL interfaces are tightly integrated with each other as well as Flink’s DataStream and DataSet APIs. You can easily switch between all APIs and libraries which build upon the APIs. For instance, you can extract patterns from a DataStream using the CEP library and later use the Table API to analyze the patterns, or you might scan, filter, and aggregate a batch table using a SQL query before running a on the preprocessed data.

    Please note that the Table API and SQL are not yet feature complete and are being actively developed. Not all operations are supported by every combination of [Table API, SQL] and [stream, batch] input.

    The following dependencies are relevant for most projects:

    • flink-table-common: A common module for extending the table ecosystem by custom functions, formats, etc.
    • flink-table-api-java: The Table & SQL API for pure table programs using the Java programming language (in early development stage, not recommended!).
    • flink-table-api-scala: The Table & SQL API for pure table programs using the Scala programming language (in early development stage, not recommended!).
    • : The Table & SQL API with DataStream/DataSet API support using the Scala programming language.
    • flink-table-planner: The table program planner and runtime.
    • flink-table-uber: Packages the modules above into a distribution for most Table & SQL API use cases. The uber JAR file flink-table*.jar is located in the /opt directory of a Flink release and can be moved to if desired.

    The following dependencies must be added to a project in order to use the Table API & SQL for defining pipelines:

    Internally, parts of the table ecosystem are implemented in Scala. Therefore, please make sure to add the following dependency for both batch and streaming applications:

    Extension Dependencies

    If you want to implement a for interacting with Kafka or a set of user-defined functions, the following dependency is sufficient and can be used for JAR files for the SQL Client:

    • SerializationSchemaFactory
    • DeserializationSchemaFactory
    • ScalarFunction
    • TableFunction

    Where to go next?

    • Concepts & Common API: Shared concepts and APIs of the Table API and SQL.
    • : Streaming-specific documentation for the Table API or SQL such as configuration of time attributes and handling of updating results.
    • Connect to External Systems: Available connectors and formats for reading and writing data to external systems.
    • : Supported operations and API for the Table API.
    • SQL: Supported operations and syntax for SQL.
    • : Supported functions in Table API and SQL.
    • SQL Client: Play around with Flink SQL and submit a table program to a cluster without programming knowledge.