In each section collect the options that will be combined to use with helm install.

By default, the chart is installed to a namespace called pulsar.

If you decide to install the chart into a different k8s namespace, you can include this option in your Helm install command:

  1. --set namespace=<different-k8s-namespace>

By default, the chart doesn’t create the namespace.

  1. namespaceCreate: false

If you want the chart to create the k8s namespace automatically, you can include this option in your Helm install command.

  1. --set namespaceCreate=true

Persistence

By default the chart creates Volume Claims with the expectation that a dynamic provisioner will create the underlying Persistent Volumes.

  1. volumes:
  2.   persistence: true
  3.   # configure the components to use local persistent volume
  4.   # the local provisioner should be installed prior to enable local persistent volume
  5.   local_storage: false

If you would like to use local persistent volumes as the persistent storage for your Helm release, you can install and include the following option in your Helm install command.

  1. --set volumes.local_storage=true

This chart is designed for production use, To use this chart in a development environment (e.g. minikube), you can disable persistence by including this option in your Helm install command.

  1. --set volumes.persistence=false

Affinity

By default anti-affinity is turned on to ensure pods of same component can run on different nodes.

  1. affinity:
  2.   anti_affinity: true

If you are planning to use this chart in a development environment (e.g. minikue), you can disable anti-affinity by including this option in your Helm install command.

This chart is designed for production usage. It deploys a production-ready Pulsar cluster including Pulsar core components and monitoring components.

  1. ## Components
  2. ##
  3. ## Control what components of Apache Pulsar to deploy for the cluster
  4. components:
  5.   # zookeeper
  6.   zookeeper: true
  7.   # bookkeeper
  8.   bookkeeper: true
  9.   # bookkeeper - autorecovery
  10.   autorecovery: true
  11.   # broker
  12.   broker: true
  13.   # functions
  14.   functions: true
  15.   # proxy
  16.   proxy: true
  17.   # toolset
  18.   toolset: true
  19.   # pulsar manager
  20.   pulsar_manager: true
  22. ## Monitoring Components
  23. ##
  24. ## Control what components of the monitoring stack to deploy for the cluster
  25. monitoring:
  26.   # monitoring - prometheus
  27.   prometheus: true
  28.   # monitoring - grafana
  29.   grafana: true

Docker Images

This chart is designed to enable controlled upgrades. So it provides the capability to configure independent image versions for components. You can customize the images by setting individual component.

  1. ## Images
  2. ##
  3. ## Control what images to use for each component
  4.   zookeeper:
  5.     repository: apachepulsar/pulsar-all
  6.     tag: 2.5.0
  7.     pullPolicy: IfNotPresent
  8.   bookie:
  9.     repository: apachepulsar/pulsar-all
  10.     tag: 2.5.0
  11.     pullPolicy: IfNotPresent
  12.   autorecovery:
  13.     repository: apachepulsar/pulsar-all
  14.     tag: 2.5.0
  15.     pullPolicy: IfNotPresent
  16.   broker:
  17.     repository: apachepulsar/pulsar-all
  18.     tag: 2.5.0
  20.   proxy:
  21.     repository: apachepulsar/pulsar-all
  22.     tag: 2.5.0
  23.     pullPolicy: IfNotPresent
  24.   functions:
  25.     repository: apachepulsar/pulsar-all
  26.     tag: 2.5.0
  27.   prometheus:
  28.     repository: prom/prometheus
  29.     tag: v1.6.3
  30.     pullPolicy: IfNotPresent
  31.   grafana:
  32.     repository: streamnative/apache-pulsar-grafana-dashboard-k8s
  33.     tag: 0.0.4
  34.     pullPolicy: IfNotPresent
  35.   pulsar_manager:
  36.     repository: apachepulsar/pulsar-manager
  37.     tag: v0.1.0
  38.     pullPolicy: IfNotPresent
  39.     hasCommand: false

TLS

This Pulsar Chart can be configured to enable TLS to protect all the traffic between components. Before you enable TLS, you have to provision TLS certificates for the components you have configured to enable TLS.

Provision TLS certs using cert-manager

In order to using cert-manager to provision the TLS certificates, you have to install before installing the Pulsar chart. After successfully install cert manager, you can then set certs.internal_issuer.enabled to true. So the Pulsar chart will use cert-manager to generate selfsigning TLS certs for the configured components.

  1. certs:
  2.   internal_issuer:
  3.     enabled: false
  4.     component: internal-cert-issuer
  5.     type: selfsigning

You can also customize the generated TLS certificates by configuring the fields as the following.

  1. tls:
  2.   # common settings for generating certs
  3.   common:
  4.     # 90d
  5.     duration: 2160h
  6.     # 15d
  7.     renewBefore: 360h
  8.     organization:
  9.       - pulsar
  10.     keySize: 4096
  11.     keyAlgorithm: rsa
  12.     keyEncoding: pkcs8

Enable TLS

After installing cert-manager, you can then set tls.enabled to true to enable TLS encryption for the entire cluster.

  1. tls:
  2.   enabled: false

You can also control whether to enable TLS encryption for individual component.

  1. tls:
  2.   # settings for generating certs for proxy
  3.   proxy:
  4.     enabled: false
  5.     cert_name: tls-proxy
  6.   # settings for generating certs for broker
  7.     enabled: false
  8.     cert_name: tls-broker
  9.   # settings for generating certs for bookies
  10.   bookie:
  11.     enabled: false
  12.     cert_name: tls-bookie
  13.   # settings for generating certs for zookeeper
  14.   zookeeper:
  15.     enabled: false
  16.     cert_name: tls-zookeeper
  18.   autorecovery:
  19.     cert_name: tls-recovery
  20.   # settings for generating certs for toolset
  21.   toolset:
  22.     cert_name: tls-toolset

Authentication is disabled by default. You can set auth.authentication.enabled to true to turn on authentication. Currently this chart only supports JWT authentication provider. You can set auth.authentication.provider to jwt to use JWT authentication provider.

  1. # Enable or disable broker authentication and authorization.
  2. auth:
  3.   authentication:
  4.     enabled: false
  5.     provider: "jwt"
  6.     jwt:
  7.       # Enable JWT authentication
  8.       # If the token is generated by a secret key, set the usingSecretKey as true.
  9.       # If the token is generated by a private key, set the usingSecretKey as false.
  10.       usingSecretKey: false
  11.   superUsers:
  12.     # broker to broker communication
  13.     broker: "broker-admin"
  14.     # proxy to broker communication
  15.     proxy: "proxy-admin"
  16.     # pulsar-admin client to broker/proxy communication
  17.     client: "admin"

If you decide to enable authentication, you can run to generate token secret keys and tokens for three super users specified in auth.superUsers field. The generated token keys and super user tokens are uploaded and stored as kubernetes secrets prefixed with <pulsar-release-name>-token-. You can use following command to find those secrets.

Authorization

Authorization is disabled by default. Authorization can be enabled only if Authentication is enabled.

  1. auth:
  2.   authorization:
  3.     enabled: false

You can include this option to turn on authorization.

  1. --set auth.authorization.enabled=true

CPU and RAM resource requirements

The resource requests, and number of replicas for the Pulsar components in this Chart are set by default to be adequate for a small production deployment. If you are trying to deploy a non-production instance, you can reduce the defaults in order to fit into a smaller cluster.

Once you have all of your configuration options collected, we need to install dependent charts before proceeding to install the Pulsar Chart.

Install Dependent Charts

One of the easiest way to get started is to use the local storage provisioner provided along with the Pulsar Helm chart.

  1. helm repo add streamnative https://charts.streamnative.io
  2. helm repo update
  3. helm install pulsar-storage-provisioner streamnative/local-storage-provisioner

Install Cert Manager

The Pulsar Chart uses cert-manager to automate provisioning and managing TLS certificates. If you decide to enable TLS encryption for brokers or proxies, you need to install cert-manager first.

You can follow the to install cert-manager.

Alternatively, we provide a bash script install-cert-manager.sh to install a cert-manager release to namespace cert-manager.

  1. git clone https://github.com/apache/pulsar-helm-chart
  2. cd pulsar-helm-chart
  3. ./scripts/cert-manager/install-cert-manager.sh

Once you have install all the dependent charts and collected all of your configuration options, you can run to prepare the helm release.

  1. git clone https://github.com/apache/pulsar-helm-chart
  2. cd pulsar-helm-chart
  3. ./scripts/pulsar/prepare_helm_release.sh -n <k8s-namespace> -k <helm-release-name>

The prepare_helm_release creates following resources:

  • A k8s namespace for installing the Pulsar release
  • Create a secret for storing the username and password of control center administrator. The username and password can be passed to prepare_helm_release.sh through flags --control-center-admin and --control-center-password. The username and password is used for logging into Grafana dashboard and Pulsar Manager.
  • Create the JWT secret keys and tokens for three superusers: broker-admin, proxy-admin, and admin. By default, it generates asymmeric pubic/private key pair. You can choose to generate symmeric secret key by specifying --symmetric.
    • proxy-admin role is used for proxies to communicate to brokers.
    • broker-admin role is used for inter-broker communications.
    • admin role is used by the admin tools.

Deploy using Helm

Once you have done the following three things, you can proceed to install a Helm release.

  • Collect all of your configuration options
  • Install dependent charts
  • Prepare the Helm release

In this example, we’ve named our Helm release pulsar.

  1. git clone https://github.com/apache/pulsar-helm-chart
  2. cd pulsar-helm-chart
  3. helm upgrade --install pulsar charts/pulsar \
  4.     --timeout 600 \
  5.     --set [your configuration options]

You can also use --version <installation version> option if you would like to install a specific version of Pulsar Helm chart.

This will output the list of resources installed once the deployment finishes which may take 5-10 minutes.

The status of the deployment can be checked by running helm status pulsar which can also be done while the deployment is taking place if you run the command in another terminal.

Accessing the Pulsar Cluster

The default values will create a ClusterIP for the following resources you can use to interact with the cluster.

  • Proxy: You can use the IP address to produce and consume messages to the installed Pulsar cluster.
  • Pulsar Manager: You can access the pulsar manager UI at http://<pulsar-manager-ip>:9527.
  1. kubectl get service -n <k8s-namespace>