In each section, collect the options that are combined to use with the helm install command.

By default, the Pulsar Helm chart is installed to a namespace called pulsar.

To install the Pulsar Helm chart into a different Kubernetes namespace, you can include this option in the helm install command.

  1. --set namespace=<different-k8s-namespace>

By default, the Pulsar Helm chart doesn’t create the namespace.

  1. namespaceCreate: false

To use the Pulsar Helm chart to create the Kubernetes namespace automatically, you can include this option in the helm install command.

  1. --set namespaceCreate=true

Persistence

By default, the Pulsar Helm chart creates Volume Claims with the expectation that a dynamic provisioner creates the underlying Persistent Volumes.

  1. volumes:
  2.   persistence: true
  3.   # configure the components to use local persistent volume
  4.   # the local provisioner should be installed prior to enable local persistent volume
  5.   local_storage: false

To use local persistent volumes as the persistent storage for Helm release, you can install the and include the following option in the helm install command.

  1. --set volumes.local_storage=true

The Pulsar Helm chart is designed for production use. To use the Pulsar Helm chart in a development environment (such as Minikube), you can disable persistence by including this option in your helm install command.

  1. --set volumes.persistence=false

Affinity

By default, anti-affinity is enabled to ensure pods of the same component can run on different nodes.

  1. affinity:
  2.   anti_affinity: true

To use the Pulsar Helm chart in a development environment (such as Minikue), you can disable anti-affinity by including this option in your helm install command.

The Pulsar Helm chart is designed for production usage. It deploys a production-ready Pulsar cluster, including Pulsar core components and monitoring components.

  1. ## Components
  2. ##
  3. ## Control what components of Apache Pulsar to deploy for the cluster
  4. components:
  5.   # zookeeper
  6.   zookeeper: true
  7.   # bookkeeper
  8.   bookkeeper: true
  9.   # bookkeeper - autorecovery
  10.   autorecovery: true
  11.   # broker
  12.   broker: true
  13.   # functions
  14.   functions: true
  15.   # proxy
  16.   proxy: true
  17.   # toolset
  18.   toolset: true
  19.   # pulsar manager
  20.   pulsar_manager: true
  22. ## Monitoring Components
  23. ##
  24. ## Control what components of the monitoring stack to deploy for the cluster
  25. monitoring:
  26.   # monitoring - prometheus
  27.   prometheus: true
  28.   # monitoring - grafana
  29.   grafana: true

Docker images

The Pulsar Helm chart is designed to enable controlled upgrades. So it can configure independent image versions for components. You can customize the images by setting individual component.

  1. ## Images
  2. ##
  3. images:
  4.   zookeeper:
  5.     repository: apachepulsar/pulsar-all
  6.     tag: 2.5.0
  7.     pullPolicy: IfNotPresent
  8.   bookie:
  9.     repository: apachepulsar/pulsar-all
  10.     tag: 2.5.0
  11.     pullPolicy: IfNotPresent
  12.   autorecovery:
  13.     repository: apachepulsar/pulsar-all
  14.     tag: 2.5.0
  16.   broker:
  17.     repository: apachepulsar/pulsar-all
  18.     tag: 2.5.0
  19.     pullPolicy: IfNotPresent
  20.   proxy:
  21.     repository: apachepulsar/pulsar-all
  22.     tag: 2.5.0
  23.     pullPolicy: IfNotPresent
  24.   functions:
  25.     repository: apachepulsar/pulsar-all
  26.     tag: 2.5.0
  27.   prometheus:
  28.     repository: prom/prometheus
  29.     tag: v1.6.3
  30.     pullPolicy: IfNotPresent
  31.   grafana:
  32.     repository: streamnative/apache-pulsar-grafana-dashboard-k8s
  33.     tag: 0.0.4
  34.     pullPolicy: IfNotPresent
  35.   pulsar_manager:
  36.     repository: apachepulsar/pulsar-manager
  37.     tag: v0.1.0
  38.     pullPolicy: IfNotPresent
  39.     hasCommand: false

TLS

The Pulsar Helm chart can be configured to enable TLS (Transport Layer Security) to protect all the traffic between components. Before enabling TLS, you have to provision TLS certificates for the required components.

Provision TLS certificates using cert-manager

To use the cert-manager to provision the TLS certificates, you have to install the cert-manager before installing the Pulsar Helm chart. After successfully installing the cert-manager, you can set certs.internal_issuer.enabled to true. Therefore, the Pulsar Helm chart can use the cert-manager to generate selfsigning TLS certificates for the configured components.

  1. certs:
  2.   internal_issuer:
  3.     enabled: false
  4.     component: internal-cert-issuer
  5.     type: selfsigning

You can also customize the generated TLS certificates by configuring the fields as the following.

  1. tls:
  2.   # common settings for generating certs
  3.   common:
  4.     # 90d
  5.     duration: 2160h
  6.     # 15d
  7.     renewBefore: 360h
  8.     organization:
  9.       - pulsar
  10.     keySize: 4096
  11.     keyAlgorithm: rsa
  12.     keyEncoding: pkcs8

Enable TLS

After installing the cert-manager, you can set tls.enabled to true to enable TLS encryption for the entire cluster.

  1. tls:
  2.   enabled: false

You can also configure whether to enable TLS encryption for individual component.

  1. tls:
  2.   # settings for generating certs for proxy
  3.   proxy:
  4.     enabled: false
  5.     cert_name: tls-proxy
  6.   # settings for generating certs for broker
  7.     enabled: false
  8.     cert_name: tls-broker
  9.   # settings for generating certs for bookies
  10.   bookie:
  11.     enabled: false
  12.     cert_name: tls-bookie
  13.   # settings for generating certs for zookeeper
  14.   zookeeper:
  16.     cert_name: tls-zookeeper
  17.   # settings for generating certs for recovery
  18.   autorecovery:
  19.     cert_name: tls-recovery
  20.   # settings for generating certs for toolset
  21.   toolset:
  22.     cert_name: tls-toolset

By default, authentication is disabled. You can set auth.authentication.enabled to true to enable authentication. Currently, the Pulsar Helm chart only supports JWT authentication provider. You can set auth.authentication.provider to jwt to use the JWT authentication provider.

  1. # Enable or disable broker authentication and authorization.
  2. auth:
  3.   authentication:
  4.     enabled: false
  5.     provider: "jwt"
  6.     jwt:
  7.       # Enable JWT authentication
  8.       # If the token is generated by a secret key, set the usingSecretKey as true.
  9.       # If the token is generated by a private key, set the usingSecretKey as false.
  10.       usingSecretKey: false
  11.   superUsers:
  12.     # broker to broker communication
  13.     broker: "broker-admin"
  14.     # proxy to broker communication
  15.     proxy: "proxy-admin"
  16.     # pulsar-admin client to broker/proxy communication
  17.     client: "admin"

To enable authentication, you can run prepare helm release to generate token secret keys and tokens for three super users specified in the auth.superUsers field. The generated token keys and super user tokens are uploaded and stored as Kubernetes secrets prefixed with <pulsar-release-name>-token-. You can use the following command to find those secrets.

Authorization

By default, authorization is disabled. Authorization can be enabled only when authentication is enabled.

  1. auth:
  2.   authorization:
  3.     enabled: false

To enable authorization, you can include this option in the helm install command.

  1. --set auth.authorization.enabled=true

CPU and RAM resource requirements

By default, the resource requests and the number of replicas for the Pulsar components in the Pulsar Helm chart are adequate for a small production deployment. If you deploy a non-production instance, you can reduce the defaults to fit into a smaller cluster.

Once you have all of your configuration options collected, you can install dependent charts before installing the Pulsar Helm chart.

Install dependent charts

One of the easiest way to get started is to use the local storage provisioner provided along with the Pulsar Helm chart.

  1. helm repo add streamnative https://charts.streamnative.io
  2. helm repo update
  3. helm install pulsar-storage-provisioner streamnative/local-storage-provisioner

Install cert-manager

The Pulsar Helm chart uses the to provision and manage TLS certificates automatically. To enable TLS encryption for brokers or proxies, you need to install the cert-manager in advance.

For details about how to install the cert-manager, follow the official instructions.

Alternatively, we provide a bash script to install a cert-manager release to the namespace cert-manager.

  1. git clone https://github.com/apache/pulsar
  2. cd pulsar/deployment/kubernetes/helm
  3. ./scripts/cert-manager/install-cert-manager.sh

Once you have install all the dependent charts and collected all of your configuration options, you can run prepare_helm_release.sh to prepare the Helm release.

  1. git clone https://github.com/apache/pulsar
  2. cd pulsar/deployment/kubernetes/helm
  3. ./scripts/pulsar/prepare_helm_release.sh -n <k8s-namespace> -k <helm-release-name>

The prepare_helm_release creates the following resources:

  • A Kubernetes namespace for installing the Pulsar release
  • A secret for storing the username and password of the control center administrator. The username and password can be passed to prepare_helm_release.sh through flags --control-center-admin and --control-center-password. The username and password is used for logging into the Grafana dashboard and Pulsar Manager.
  • JWT secret keys and tokens for three super users: broker-admin, proxy-admin, and admin. By default, it generates an asymmetric pubic/private key pair. You can choose to generate a symmetric secret key by specifying --symmetric.
    • proxy-admin role is used for proxies to communicate to brokers.
    • broker-admin role is used for inter-broker communications.
    • admin role is used by the admin tools.

Deploy Pulsar cluster using Helm

Once you have finished the following three things, you can install a Helm release.

  • Collect all of your configuration options.
  • Install dependent charts.
  • Prepare the Helm release.

In this example, we name our Helm release pulsar.

  1. git clone https://github.com/apache/pulsar
  2. cd pulsar/deployment/kubernetes/helm
  3. helm upgrade --install pulsar pulsar \
  4.     --timeout 10m \
  5.     --set [your configuration options]

You can also use the --version <installation version> option if you want to install a specific version of Pulsar Helm chart.

A list of installed resources are output once the Pulsar cluster is deployed. This may take 5-10 minutes.

The status of the deployment can be checked by running the helm status pulsar command, which can also be done while the deployment is taking place if you run the command in another terminal.

Access Pulsar cluster

The default values will create a ClusterIP for the following resources, which you can use to interact with the cluster.

  • Proxy: You can use the IP address to produce and consume messages to the installed Pulsar cluster.
  • Pulsar Manager: You can access the Pulsar Manager UI at http://<pulsar-manager-ip>:9527.
  1. kubectl get service -n <k8s-namespace>