Apache NiFi Connector

    Copied to clipboard!

    Note that the streaming connectors are currently not part of the binary distribution. See here for information about how to package the program with the libraries for cluster execution.

    Installing Apache NiFi

    Instructions for setting up a Apache NiFi cluster can be found here.

    Apache NiFi Source

    The connector provides a Source for reading data from Apache NiFi to Apache Flink.

    The class provides 2 constructors for reading data from NiFi.

    • NiFiSource(SiteToSiteConfig config, long waitTimeMs) - Constructs a given the client’s SiteToSiteConfig and the specified wait time (in milliseconds).

    Example:

    Java

    Scala

    Here data is read from the Apache NiFi Output Port called “Data for Flink” which is part of Apache NiFi Site-to-site protocol configuration.

    Apache NiFi Sink

    The class NiFiSink(…) provides a constructor for instantiating a NiFiSink.

    Example:

    Java

    Scala

    More information about Site-to-Site Protocol can be found here