Testing tools

This page includes our existing test tool which are part of the Ozone source base.

Note: we have more tests (like TCP-DS, TCP-H tests via Spark or Hive) which are not included here because they use external tools only.

As every almost every java project we have the good old unit tests inside each of our projects.

Integration test (JUnit)

Traditional unit tests are supposed to test only one unit, but we also have higher level unit tests. They use which is a helper method to start real daemons (scm,om,datanodes) during the unit test.

From maven/java point of view they are just simple unit tests (JUnit library is used) but to separate them (and solve some dependency problems) we moved all of these tests to hadoop-ozone/integration-test

We use docker-compose based pseudo-cluster to run different configuration of Ozone. To be sure that the different configuration can be started we implemented acceptance tests with the help of .

The smoketests are available from the distribution () but the robot files defines only the tests: usually they start CLI and check the output.

These definition of the tests are included in the compose directory (check or ./compose/test-all.sh).

For example a simple way to test the distribution package:

Blockade

is a tool to test network failures and partitions (it’s inspired by the legendary Jepsen tests).

Blockade tests are implemented with the help of tests and can be started from the directory of the distribution.

See the README in the blockade directory for more details.

This is a way to get in your machine. It can be started from the source code and a MiniOzoneCluster (which starts real daemons) will be started and killed randomly.

Freon

Freon is a command line application which is included in the Ozone distribution. It’s a load generator which is used in our stress tests.

In randomkeys mode, the data written into ozone cluster is randomly generated. Each key will be of size 10 KB.

The number of volumes/buckets/keys can be configured. The replication type and factor (eg. replicate with ratis to 3 nodes) also can be configured.

For more information use:

bin/ozone freon –help

For example:

Genesis is a microbenchmarking tool. It’s also included in the distribution (ozone genesis) but it doesn’t require real cluster. It measures different part of the code in an isolated way (eg. the code which saves the data to the local RocksDB based key value stores)

Example run: