pull-deps tool

pull-deps has several command line options, they are as follows:

-c or --coordinate (Can be specified multiple times)

Extension coordinate to pull down, followed by a maven coordinate, e.g. org.apache.druid.extensions:mysql-metadata-storage

-h or --hadoop-coordinate (Can be specified multiply times)

Apache Hadoop dependency to pull down, followed by a maven coordinate, e.g. org.apache.hadoop:hadoop-client:2.4.0

--no-default-hadoop

Don’t pull down the default hadoop coordinate, i.e., org.apache.hadoop:hadoop-client:2.3.0. If -h option is supplied, then default hadoop coordinate will not be downloaded.

--clean

Remove existing extension and hadoop dependencies directories before pulling down dependencies.

-l or --localRepository

A local repository that Maven will use to put downloaded files. Then pull-deps will lay these files out into the extensions directory as needed.

-r or --remoteRepository

--no-default-remote-repositories

Don’t use the default remote repositories, only use the repositories provided directly via —remoteRepository.

or --defaultVersion

Version to use for extension coordinate that doesn’t have a version information. For example, if extension coordinate is org.apache.druid.extensions:mysql-metadata-storage, and default version is 24.0.2, then this coordinate will be treated as org.apache.druid.extensions:mysql-metadata-storage:24.0.2

--use-proxy

Use http/https proxy to send request to the remote repository servers. --proxy-host and --proxy-port must be set explicitly if this option is enabled.

--proxy-type

Set the proxy type, Should be either http or https, default value is https.

--proxy-host

Set the proxy host. e.g. proxy.com.

--proxy-port

Set the proxy port number. e.g. 8080.

Set a username to connect to the proxy, this option is only required if the proxy server uses authentication.

--proxy-password

Set a password to connect to the proxy, this option is only required if the proxy server uses authentication.

To run pull-deps, you should

Example:

Suppose you want to download mysql-metadata-storage and hadoop-client(both 2.3.0 and 2.4.0) with a specific version, you can run pull-deps command with -c org.apache.druid.extensions:mysql-metadata-storage:24.0.2, -h org.apache.hadoop:hadoop-client:2.3.0 and -h org.apache.hadoop:hadoop-client:2.4.0, an example command would be:

Because --clean is supplied, this command will first remove the directories specified at druid.extensions.directory and druid.extensions.hadoopDependenciesDir, then recreate them and start downloading the extensions there. After finishing downloading, if you go to the extension directories you specified, you will see

Note that if you specify --defaultVersion, you don’t have to put version information in the coordinate. For example, if you want to use version 24.0.2, you can change the command above to