Pulsar Perf
This example shows how the Pulsar Perf produces messages with default options. For all configuration options available for the pulsar-perf produce
command, see configuration options.
After the command is executed, the test data is continuously output on the Console.
输出
19:53:31.459 [pulsar-perf-producer-exec-1-1] INFO org.apache.pulsar.testclient.PerformanceProducer - Created 1 producers
19:53:31.482 [pulsar-timer-5-1] WARN com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider
19:53:40.861 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 93.7 msg/s --- 0.7 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.575 ms - med: 3.460 - 95pct: 4.790 - 99pct: 5.308 - 99.9pct: 5.834 - 99.99pct: 6.609 - Max: 6.609
19:53:50.909 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100.0 msg/s --- 0.8 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.437 ms - med: 3.328 - 95pct: 4.656 - 99pct: 5.071 - 99.9pct: 5.519 - 99.99pct: 5.588 - Max: 5.588
19:54:10.940 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100.0 msg/s --- 0.8 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.298 ms - med: 3.220 - 95pct: 4.474 - 99pct: 4.926 - 99.9pct: 5.645 - 99.99pct: 5.654 - Max: 5.654
19:54:20.956 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100.1 msg/s --- 0.8 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.308 ms - med: 3.199 - 95pct: 4.532 - 99pct: 4.871 - 99.9pct: 5.291 - 99.99pct: 5.323 - Max: 5.323
19:54:30.972 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100.0 msg/s --- 0.8 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.249 ms - med: 3.144 - 95pct: 4.437 - 99pct: 4.970 - 99.9pct: 5.329 - 99.99pct: 5.414 - Max: 5.414
19:54:40.987 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100.0 msg/s --- 0.8 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.435 ms - med: 3.361 - 95pct: 4.772 - 99pct: 5.150 - 99.9pct: 5.373 - 99.99pct: 5.837 - Max: 5.837
^C19:54:44.325 [Thread-1] INFO org.apache.pulsar.testclient.PerformanceProducer - Aggregated throughput stats --- 7286 records sent --- 99.140 msg/s --- 0.775 Mbit/s
From the above test data, you can get the throughput statistics and the write latency statistics. The aggregated statistics is printed when the Pulsar Perf is stopped. You can press Ctrl+C to stop the Pulsar Perf. After the Pulsar Perf is stopped, the formatted test result appears under your directory. The document looks like perf-producer-1589370810837.hgrm
. You can also check the test result through HdrHistogram Plotter. For details about how to check the test result through , see HdrHistogram Plotter.
You can get all options by executing the bin/pulsar-perf produce -h
command. Therefore, you can modify these options as required.
The following table lists configuration options available for the pulsar-perf produce
command.
This example shows how the Pulsar Perf consumes messages with default options.
bin/pulsar-perf consume my-topic
输出
From the output test data, you can get the throughput statistics and the end-to-end latency statistics. The aggregated statistics is printed after the Pulsar Perf is stopped. You can press Ctrl+C to stop the Pulsar Perf.
Configuration options for pulsar-perf consume
You can get all options by executing the command. Therefore, you can modify these options as required.
The following table lists configuration options available for the pulsar-perf consume
command.
选项 | 说明 | 默认值 |
---|---|---|
acks-delay-millis | 设置确认分组的延迟时间,以毫秒为单位 | 100 ms |
auth-params | 设置认证参数,格式由认证插件类中configure 方法的实现决定的,如”key1:val1,key2:val2” 或 “{“key1”:”val1”,”key2”:”val2”}”。 | N/A |
auth_plugin | 设置认证插件类名 | N/A |
auto_ack_chunk_q_full | 配置如果队列满时,是否自动确认接收器队列中最旧的消息。 | false |
listener-name | 为broker指定监听器名称。 | N/A |
batch-index-ack | 开启或禁用批量索引确认 | false |
busy-wait | 在 Pulsar 客户端启用或禁用 Busy-Wait. | false |
conf-file | 设置配置文件 | N/A |
encryption-key-name | 设置用于加密有效载荷的公钥名称 | N/A |
encryption-key-value-file | 设置包含用于加密有效载荷的公钥文件 | N/A |
help | 配置帮助信息 | false |
expire_time_incomplete_chunked_messages | Set the expiration time for incomplete chunk messages (in milliseconds). | 0 |
max-connections | 设置单个broker的最大TCP连接数 | 100 |
max_chunked_msg | Set the max pending chunk messages. | 0 |
num-consumers | 设定每个主题的消费者数量 | 1 |
num-io-threads | Set the number of threads to be used for handling connections to brokers. | 1 |
num-subscriptions | Set the number of subscriptions (per topic). | 1 |
num-topic | 设置主题数量 | 1 |
pool-messages | Configure whether to use the pooled message. | true |
rate | 模拟一个慢速消费者消费数据(速率 消息/秒) | 0.0 |
receiver-queue-size | 设置接收队列大小 | 1000 |
receiver-queue-size-across-partitions | Set the max total size of the receiver queue across partitions. | 50000 |
replicated | 配置是否复制订阅状态。 | false |
service-url | 设置 Pulsar 服务的 URL | |
stats-interval-seconds | 设置统计间隔。 如果置为0,则不统计。 | 0 |
subscriber-name | 设置订阅者名称前缀 | sub |
subscription-position | Set the subscription position. Valid values are Latest , Earliest . | Latest |
subscription-type | 设置订阅类型 | Exclusive |
test-duration | Set the test duration (in seconds). If the value is 0 or smaller than 0, it keeps consuming messages. | 0 |
tls-allow-insecure | Set the allowed insecure TLS connection. | N/A |
trust-cert-file | 设置受信的TLS证书文件所在路径 |
By default, the Pulsar Perf uses conf/client.conf
as the default configuration and uses conf/log4j2.yaml
as the default Log4j configuration. If you want to connect to other Pulsar clusters, you can update the brokerServiceUrl
in the client configuration.
You can use the following commands to change the configuration file and the Log4j configuration file.
export PULSAR_CLIENT_CONF=<你的配置文件>
export PULSAR_LOG_CONF=<你的日志配置文件>
In addition, you can use the following command to configure the JVM configuration through environment variables:
export PULSAR_EXTRA_OPTS='-Xms4g -Xmx4g -XX:MaxDirectMemorySize=4g'
To check test results through the HdrHistogram Plotter, follow these steps:
从 GitHub 上克隆 HdrHistogram 仓库到本地
切换到HdrHistogram文件夹
cd HdrHistogram
安装HdrHistogram 绘图器
mvn clean install -DskipTests
转换 Pulsar Perf 生成的文件
你将获得两个文件。 将扩展名为 .hgrm 的输出文件上传到HdrHistogram Plotter。