DC/OS (the DataCenter Operating System) is a distributed operating system used for deploying and managing applications and systems on . DC/OS is an open-source tool that Mesosphere creates and maintains .

Apache Pulsar is available as a , which runs multiple applications as manageable sets.

In order to run Pulsar on DC/OS, you need the following:

• DC/OS version 1.9 or higher

• A with at least three agent nodes

• The DC/OS CLI tool installed

• The configuration file from the Pulsar GitHub repo.

Each node in the DC/OS-managed Mesos cluster must have at least:

• 4 CPU
• 60 GB of total persistent disk

Alternatively, you can change the configuration in PulsarGroups.json according to match your resources of DC/OS cluster.

Deploy Pulsar using the DC/OS command interface

You can deploy Pulsar on DC/OS using this command:

$ dcos marathon group add PulsarGroups.json

This command deploys Docker container instances in three groups, which together comprise a Pulsar cluster:

• 3 bookies (1 on each agent node and 1 bookie recovery instance)
• 3 Pulsar (1 broker on each node and 1 admin instance)
• 1 Prometheus instance and 1 instance

When you run DC/OS, a ZooKeeper cluster already runs at master.mesos:2181, thus you do not have to install or start up ZooKeeper separately.

After executing the dcos command above, click on the Services tab in the DC/OS GUI interface, which you can access at in this example. You should see several applications in the process of deploying.

To monitor the status of the BookKeeper cluster deployment, click on the bookkeeper group in the parent pulsar group.

At this point, 3 bookies should be shown as green, which means that the bookies have been deployed successfully and are now running.

You can also click into each bookie instance to get more detailed information, such as the bookie running log.

To display information about the BookKeeper in ZooKeeper, you can visit . In this example, 3 bookies are under the available directory.

The Pulsar broker Group

Similar to the BookKeeper group above, click into the brokers to check the status of the Pulsar brokers.

You can also click into each broker instance to get more detailed information, such as the broker running log.

Broker cluster information in Zookeeper is also available through the web UI. In this example, you can see that the and managed-ledgers directories have been created.

The monitory group consists of Prometheus and Grafana.

Prometheus

Click into the instance of prom to get the endpoint of Prometheus, which is 192.168.65.121:9090 in this example.

If you click that endpoint, you can see the Prometheus dashboard. The http://192.168.65.121:9090/targets URL display all the bookies and brokers.

Click into grafana to get the endpoint for Grafana, which is 192.168.65.121:3000 in this example.

If you click that endpoint, you can access the Grafana dashbaord.

Run a simple Pulsar consumer and producer on DC/OS

Now that you have a fully deployed Pulsar cluster, you can run a simple consumer and producer to show Pulsar on DC/OS in action.

Download and prepare the Pulsar Java tutorial

You can clone a repo. This repo contains a simple Pulsar consumer and producer (you can find more information in the README file of the repo).

Change the SERVICE_URL from pulsar://localhost:6650 to in both ConsumerTutorial.java and . The pulsar://a1.dcos:6650 endpoint is for the broker service. You can fetch the endpoint details for each broker instance from the DC/OS GUI. a1.dcos is a DC/OS client agent, which runs a broker. The client agent IP address can also replace this.

Now, change the message number from 10 to 10000000 in main method of ProducerTutorial.java so that it can produce more messages.

Now compile the project code using the command below:

$ mvn clean package

Execute this command to run the consumer:

Execute this command to run the producer:

$ mvn exec:java -Dexec.mainClass="tutorial.ProducerTutorial"

You can see the producer producing messages and the consumer consuming messages through the DC/OS GUI.

View Grafana metric output

While the producer and consumer run, you can access running metrics information from Grafana.

You can shut down and uninstall the pulsar application from DC/OS at any time in the following two ways:

1. Using the DC/OS GUI, you can choose Delete at the right end of Pulsar group.