Doris output plugin

    Learn more about Doris Stream Load

    2.compile

    Execute under extension/logstash/ directory

    You will get logstash-output-doris-{version}.gem file in the same directory

    copy logstash-output-doris-{version}.gem to the logstash installation directory

    Executing an order

    ./bin/logstash-plugin install logstash-output-doris-{version}.gem

    Install logstash-output-doris plugin

    Example:

    Create a new configuration file in the config directory and name it logstash-doris.conf

    The specific configuration is as follows:

    Configuration instructions:

    Load configuration:(Reference documents)

    Other configuration:

    Run the command to start the doris output plugin:

    {logstash-home}/bin/logstash -f {logstash-home}/config/logstash-doris.conf --config.reload.automatic

    1> Download the ruby compressed package and go to ruby official website to download it. The version 2.7.1 used here

    2> Compile and install, configure ruby environment variables

    3> Go to the doris source extension/logstash/ directory and execute

    gem build logstash-output-doris.gemspec

    Get the file logstash-output-doris-0.1.0.gem, and the compilation is complete

    2. Install and configure filebeat (here use filebeat as input)

    1> es official website Download the filebeat tar compression package and decompress it

    2> Enter the filebeat directory and modify the configuration file filebeat.yml as follows:

    /tmp/doris.data is the doris data path

    ./filebeat -e -c filebeat.yml -d "publish"

    1> es official website Download the logstash tar compressed package and decompress it

    2> Copy the logstash-output-doris-0.1.0.gem obtained in step 1 to the logstash installation directory

    3> execute

    ./bin/logstash-plugin install logstash-output-doris-0.1.0.gem

    Install the plugin

    4> Create a new configuration file logstash-doris.conf in the config directory as follows:

    The configuration here needs to be configured according to the configuration instructions

    5> Start logstash:

    ./bin/logstash -f ./config/logstash-doris.conf —config.reload.automatic

    4.Test Load

    Add write data to /tmp/doris.data

    echo a,b,c,d,e >> /tmp/doris.data