To use message deduplication in Pulsar, you need to configure your Pulsar brokers and clients.

You can enable or disable message deduplication on a per-namespace basis. By default, it is disabled on all namespaces. You can enable it in the following ways:

  • Enable for specific namespaces with the interface

You can configure message deduplication in Pulsar using the configuration file. The following deduplication-related parameters are available.

By default, message deduplication is disabled on all Pulsar namespaces. To enable it by default on all namespaces, set the brokerDeduplicationEnabled parameter to true and re-start the broker.

Though message deduplication is disabled by default at broker-level, you can enable message deduplication for specific namespaces using the pulsar-admin namespace set-deduplication command. You can use the --enable/-e flag and specify the namespace. The following is an example with <tenant>/<namespace>:

Even if you enable message deduplication at broker-level, you can disable message deduplication for a specific namespace using the command. Use the --disable/-d flag and specify the namespace. The following is an example with :

If you enable message deduplication in Pulsar brokers, you need complete the following tasks for your client producers:

  1. Specify a name for the producer.

The instructions for Java, Python, and C++ clients are different.

Python clients

C++ clients

To enable message deduplication on a Java producer, set the producer name using the producerName setter, and set the timeout to 0 using the sendTimeout setter.

To enable message deduplication on a , set the producer name using producer_name, and set the timeout to 0 using send_timeout_millis.