tracing

    doris is responsible for collecting traces and exporting them to a third-party tracing analysis system, which is responsible for the presentation and storage of traces.

    doris currently supports exporting traces directly to zipkin.

    Configuring and starting Doris

    Add configuration to fe.conf

    3. # Configure traces to export to zipkin
    4. trace_export_url = http://127.0.0.1:9411/api/v2/spans

    Add configuration to be.conf

    1. enable_tracing = true
    3. # Configure traces to export to zipkin
    4. trace_export_url = http://127.0.0.1:9411/api/v2/spans
    6. # Queue size for caching spans. span export will be triggered once when the number of spans reaches half of the queue capacity. spans arriving in the queue will be discarded when the queue is full.
    8. # The maximum number of spans to export in a single pass.
    9. max_span_export_batch_size=512
    11. # Maximum interval for exporting span
    12. export_span_schedule_delay_millis=500

    Start fe and be

    1. sh fe/bin/start_fe.sh --daemon
    2. sh be/bin/start_be.sh --daemon

    View zipkin UI

    Use the opentelemetry collector to export traces to other systems such as zipkin, jaeger, skywalking, or to database systems and files. For more details, refer to .

    Meanwhile, opentelemetry collector provides a rich set of operators to process traces. For example, filterprocessor , . For more details, refer to collector processor.

    opentelemetry has released collector core and , contrib provides richer features, here is an example of contrib version.

    Download collector

    Download otelcol-contrib, available on the official website 

    1. wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.55.0/otelcol-contrib_0.55.0_linux_amd64.tar.gz
    3. tar -zxvf otelcol-contrib_0.55.0_linux_amd64.tar.gz

    Generate configuration file

    The following configuration file uses the otlp (OpenTelemetry Protocol) protocol to receive traces data, perform batch processing and filter out traces longer than 50ms, and finally export them to zipkin and file.

    1. cat > otel-collector-config.yaml << EOF
    2. receivers:
    3.   otlp:
    4.     protocols:
    5.       http:
    7. exporters:
    8.   zipkin:
    9.     endpoint: "http://10.81.85.90:8791/api/v2/spans"
    10.   file:
    11.     path: ./filename.json
    13. processors:
    14.   batch:
    15.   tail_sampling:
    16.     policies:
    17.       {
    18.         name: duration_policy,
    19.         type: latency,
    20.       }
    22. extensions:
    25.   pipelines:
    26.     traces:
    27.       receivers: [otlp]
    28.       processors: [batch, tail_sampling]
    29.       exporters: [zipkin, file]
    30. EOF

    Start collector

    1. nohup ./otelcol-contrib --config=otel-collector-config.yaml &

    Configuring and starting Doris

    Add configuration to fe.conf

    Add configuration to be.conf

    1. enable_tracing = true
    3. # enable opentelemetry collector
    4. trace_exporter = collector
    6. # Configure traces export to collector, 4318 is the default port for collector otlp http
    7. trace_export_url = http://127.0.0.1:4318/v1/traces
    9. # Queue size for caching spans. span export will be triggered once when the number of spans reaches half of the queue capacity. spans arriving in the queue will be discarded when the queue is full.
    10. max_span_queue_size=2048
    12. # The maximum number of spans to export in a single pass.
    13. max_span_export_batch_size=512
    15. # Maximum interval for exporting span
    16. export_span_schedule_delay_millis=500

    Start fe and be

    1. sh fe/bin/start_fe.sh --daemon
    2. sh be/bin/start_be.sh --daemon

      View zipkin UI