Monitoring

The monitoring interface can be easily integrated with Prometheus since we support it native format.

To get started, the first step is to enable the HTTP Server from the configuration file:

the above configuration snippet will instruct Fluent Bit to start it HTTP Server on TCP Port 2020 and listening on all network interfaces:

  2. Fluent Bit v1.4.0
  3. * Copyright (C) 2019-2020 The Fluent Bit Authors
  4. * Copyright (C) 2015-2018 Treasure Data
  5. * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
  6. * https://fluentbit.io
  8. [2020/03/10 19:08:24] [ info] [engine] started
  9. [2020/03/10 19:08:24] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020

now with a simple curl command is enough to gather some information:

  1. $ curl -s http://127.0.0.1:2020 | jq
  2. {
  3.   "fluent-bit": {
  4.     "version": "0.13.0",
  5.     "edition": "Community",
  6.     "flags": [
  7.       "FLB_HAVE_METRICS",
  8.       "FLB_HAVE_SQLDB",
  9.       "FLB_HAVE_TRACE",
  10.       "FLB_HAVE_HTTP_SERVER",
  11.       "FLB_HAVE_FLUSH_LIBCO",
  13.       "FLB_HAVE_VALGRIND",
  14.       "FLB_HAVE_FORK",
  15.       "FLB_HAVE_PROXY_GO",
  16.       "FLB_HAVE_REGEX",
  17.       "FLB_HAVE_C_TLS",
  18.       "FLB_HAVE_SETJMP",
  19.       "FLB_HAVE_ACCEPT4",
  20.       "FLB_HAVE_INOTIFY"
  21.     ]
  22.   }
  23. }

Fluent Bit aims to expose useful interfaces for monitoring, as of Fluent Bit v0.14 the following end points are available:

Query the service uptime with the following command:

it should print a similar output like this:

  1. {
  2.   "uptime_sec": 8950000,
  3.   "uptime_hr": "Fluent Bit has been running:  103 days, 14 hours, 6 minutes and 40 seconds"
  4. }

Query internal metrics in JSON format with the following command:

  1. $ curl -s http://127.0.0.1:2020/api/v1/metrics | jq

Metrics in Prometheus format

Query internal metrics in Prometheus Text 0.0.4 format:

    this time the same metrics will be in Prometheus format instead of JSON:

    1. fluentbit_input_records_total{name="cpu.0"} 57 1509150350542
    2. fluentbit_input_bytes_total{name="cpu.0"} 18069 1509150350542
    3. fluentbit_output_proc_records_total{name="stdout.0"} 54 1509150350542
    5. fluentbit_output_errors_total{name="stdout.0"} 0 1509150350542
    6. fluentbit_output_retries_total{name="stdout.0"} 0 1509150350542
    7. fluentbit_output_retries_failed_total{name="stdout.0"} 0 1509150350542

    By default configured plugins on runtime get an internal name in the format plugin_name.ID. For monitoring purposes this can be confusing if many plugins of the same type were configured. To make a distinction each configured input or output section can get an alias that will be used as the parent name for the metric.

    The following example set an alias to the INPUT section which is using the CPU input plugin:

    1. {
    2.   "input": {
    3.     "server1_cpu": {
    4.       "records": 8,
    5.       "bytes": 2536
    6.     }
    7.   },
    8.   "output": {
    9.     "raw_output": {
    10.       "proc_records": 5,
    11.       "proc_bytes": 1585,
    12.       "errors": 0,
    13.       "retries": 0,
    14.       "retries_failed": 0
    15.     }
    16. }