To use message deduplication in Pulsar, you have to your Pulsar brokers and clients.

You can enable or disable message deduplication on a per-namespace basis. By default, it is disabled on all namespaces. You can enable it in the following ways:

  • Enable for specific namespaces with the interface

You can configure message deduplication in Pulsar using the configuration file. The following deduplication-related parameters are available.

Even if you set the value for brokerDeduplicationEnabled, enabling or disabling via Pulsar admin CLI will override the default settings at the broker-level.

Though message deduplication is disabled by default at broker-level, you can enable message deduplication for specific namespaces using the pulsar-admin namespace set-deduplication command. You can use the --enable/-e flag and specify the namespace. The following is an example with <tenant>/<namespace>:

Even if you enable message deduplication at broker-level, you can disable message deduplication for a specific namespace using the command. Use the --disable/-d flag and specify the namespace. The following is an example with :

If you enable message deduplication in Pulsar brokers, you need complete the following tasks for your client producers:

  1. Specify a name for the producer.

Java clients

Python clients

C++ clients

To enable message deduplication on a Java producer, set the producer name using the producerName setter, and set the timeout to 0 using the sendTimeout setter.

To enable message deduplication on a , set the producer name using producer_name, and set the timeout to 0 using send_timeout_millis.