Hop Server

    Additionally, a Hop Server can also be accessed directly through a number of web services and in combination with the Web Service and metadata types.

    Hop Server is available as a script in your Hop installation directory.

    Run Hop Server without any parameters to display its usage options. On Windows, this is , on Mac and Linux, run hop-server.sh.

    The available Hop Server options are:

    Start Hop Server with command line parameters

    Hop Server can be started with hostname or ip address and port number as unnamed arguments:

    1. hop-server <Interface address> <Port> [-h] [-p <arg>] [-s] [-u <arg>]

    Example startup commands are:

    1. hop-server.sh 127.0.0.1 8080
    1. hop-server.sh 192.168.1.221 8081

    Listen to all interfaces on the server:

    1. hop-server.sh 0.0.0.0 8080

    Specify the xml configuration file as the only argument:

    1. hop-server <Configuration File>

    The syntax of this configuration file is fairly simple:

    Example startup commands with a configuration file are:

    1. hop-server.sh /foo/bar/hop-server-config.xml

    Or with a remote configuration file:

    1. hop-server.sh http://www.example.com/hop-server-config.xml

    You can also enable a project lifecyfle environment for the Hop server:

    1. hop-server.sh -e graph-aws hop-server.xml

    In the sample above the environment contains configuration files with variables which are loaded. With the environment the server also knows the project home folder. The server configuration file is found in the home folder automatically with the implicit relative path.

    SSL configuration

    The 3 main options are:

    • keyStore : the path to the java keystore file, created with keytool

    • keyStorePassword : the password to the keystore file

    • keyPassword : the key password. If this is the same as the keystore password you can omit this option.

    The HTTP protocol used is version 1.1 or HTTP/1.1. The type of keystore read is a Java Keystore or type: JKS. Let’s take a look at how we can generate a sample keystore:

    1. # Generate a new key
    2. #
    3. openssl genrsa -des3 -out hop.key
    5. # Make a new certificate
    6. #
    7. openssl req -new -x509 -key hop.key -out hop.crt
    9. # Create a PKCS12 keystore and import it into a JKS keystore
    10. # The resulting file is: keystore
    11. #
    12. keytool -keystore keystore -import -alias hop -file hop.crt -trustcacerts
    13. openssl req -new -key hop.key -out hop.csr
    14. openssl pkcs12 -inkey hop.key -in hop.crt -export -out hop.pkcs12
    15. keytool -importkeystore -srckeystore hop.pkcs12 -srcstoretype PKCS12 -destkeystore keystore

    Here is an example of the information to include in your server XML:

    1. <hop-server-config>
    2. <hop-server>
    3. ...
    5.       <keyStore>/path/to/keystore</keyStore>
    6.       <keyStorePassword>password</keyStorePassword>
    7.       <keyPassword>keyPassword</keyPassword>
    10.     <!-- Add the following line to support querying over https -->
    11.     <sslMode>Y</sslMode>
    12.   </hop-server>
    13.   ...
    14. </hop-server-config>

    It’s often very convenient to run a Hop docker container since it has all the required software automatically delivered. For a complete description of the standard Hop docker container see the full reference in the technical documentation. Here is an example of how you would start a “long-lived” docker container:

    Stopping Hop Server

    In a testing setup where Hop Server was started from a terminal, the process can be terminated through CTRL-C.

    In headless environments, the same hop-server command used to start the server can be used to stop it:

    1. hop-server.sh 127.0.0.1 8080 -s -u cluster -p cluster

    Verify startup

    Starting a Hop Server on the local machine e.g. on port 8081 will only take 1 or 2 seconds.

    The console output will look similar to what is listed below:

    1. 2020/06/20 18:35:12 - HopServer - Installing timer to purge stale objects after 1440 minutes.
    2. 2020/06/20 18:35:12 - HopServer - Created listener for webserver @ address : localhost:8081

    You can query the new server with another hop-server command:

    1. sh hop-server.sh -gs -u cluster -p cluster 127.0.0.1 8080
    2. Pipelines: 0 found.
    4. Workflows: 0 found.

    Query a pipeline from the command line

    1. sh hop-server.sh -id 375c9113-b538-4559-8e98-ee02a435fbb9 -u cluster -p cluster -ps service-example -j my-project hop-server.xml
    2. 2021/10/01 13:27:04 - HopServer - Enabling project 'my-project'
    3.   ID: 375c9113-b538-4559-8e98-ee02a435fbb9
    4.       Name:     service-example
    5.       Status:   Finished
    6.       Start:    2021/10/01 13:26:45.128
    7.       End:      2021/10/01 13:26:45.220
    8.       Log date: 2021/10/01 13:27:04.363
    9.       Errors:   0
    10.       Transforms: 4 found.
    11.         1
    12.           Name:      a,b
    13.           Copy:      0
    14.           Status:    Finished
    15.           Input:     0
    16.           Output:    0
    17.           Read:      1
    18.           Written:   1
    19.           Rejected:  0
    20.           Updated:   0
    21.           Errors:    0
    22.         2
    23. ...
    24.         3
    25. ...
    26. ...
    28.           2021/10/01 13:26:45 - service-example - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
    29.           2021/10/01 13:26:45 - service-example - Execution started for pipeline [service-example]
    30.           2021/10/01 13:26:45 - a,b.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    31.           2021/10/01 13:26:45 - c,d.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    32.           2021/10/01 13:26:45 - build JSON.0 - Finished processing (I=0, O=1, R=1, W=1, U=0, E=0)
    33.           2021/10/01 13:26:45 - OUTPUT.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    34.           2021/10/01 13:26:45 - service-example - Pipeline duration : 0.092 seconds [  0.092" ]
    35.           2021/10/01 13:26:45 - service-example - Execution finished on a local pipeline engine with run configuration 'local'
    1. sh hop-server.sh -ws test-workflow -id e24b4549-edf0-4d77-987e-f103b630b4cc -u cluster -p cluster localhost 8181
    2.   ID: e24b4549-edf0-4d77-987e-f103b630b4cc
    3.       Name:     test-workflow
    4.       Status:   Finished
    5.       Log date: 2021/10/01 14:27:45.891
    6.       Result:   true
    7.       Errors:   0
    8.       Logging:
    9.           2021/10/01 14:27:45 - test-workflow - Start of workflow execution
    10.           2021/10/01 14:27:46 - test-workflow - Starting action [sample]
    11.           2021/10/01 14:27:46 - sample - Using run configuration [remote-8181]
    12.           2021/10/01 14:27:46 - sample - Executing this pipeline using the Remote Pipeline Engine with run configuration 'remote-8181'
    13.           2021/10/01 14:27:46 - sample - 2021/10/01 14:27:46 - sample - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
    14.           2021/10/01 14:27:46 - sample - 2021/10/01 14:27:46 - sample - Execution started for pipeline [sample]
    15.           2021/10/01 14:27:47 - sample - 2021/10/01 14:27:47 - 1M.0 - Finished processing (I=0, O=0, R=0, W=1000000, U=0, E=0)
    16.           2021/10/01 14:27:47 - sample - 2021/10/01 14:27:47 - someString,someInt.0 - Finished processing (I=0, O=0, R=1000000, W=1000000, U=0, E=0)
    17.           2021/10/01 14:27:47 - sample - 2021/10/01 14:27:47 - id.0 - Finished processing (I=0, O=0, R=1000000, W=1000000, U=0, E=0)
    18.           2021/10/01 14:27:47 - sample - 2021/10/01 14:27:47 - sample - Pipeline duration : 0.977 seconds [  0.977" ]
    19.           2021/10/01 14:27:47 - sample - 2021/10/01 14:27:47 - sample - Execution finished on a local pipeline engine with run configuration 'local'
    20.           2021/10/01 14:27:47 - sample - Execution finished on a remote pipeline engine with run configuration 'remote-8181'
    21.           2021/10/01 14:27:48 - test-workflow - Starting action [true]
    22.           2021/10/01 14:27:48 - test-workflow - Starting action [false]
    23.           2021/10/01 14:27:48 - test-workflow - Starting action [log-something]
    24.           2021/10/01 14:27:48 - Subject - Message
    25.           2021/10/01 14:27:48 - test-workflow - Starting action [Success]
    26.           2021/10/01 14:27:48 - test-workflow - Finished action [Success] (result=[true])
    27.           2021/10/01 14:27:48 - test-workflow - Finished action [log-something] (result=[true])
    28.           2021/10/01 14:27:48 - test-workflow - Finished action [false] (result=[true])
    29.           2021/10/01 14:27:48 - test-workflow - Finished action [true] (result=[true])
    30.           2021/10/01 14:27:48 - test-workflow - Finished action [sample] (result=[true])
    31.           2021/10/01 14:27:48 - test-workflow - Workflow execution finished
    32.           2021/10/01 14:27:48 - test-workflow - Workflow duration : 2.715 seconds [  2.714" ]

    Connect to the Hop Server UI

    To connect to the previously started server, point your browser to [http://localhost:8081](http://localhost:8081).

    You’ll be prompted for your username and password. The default is for both the username and password. The defaults obviously should be changed in any environment that goes beyond a simple local developer setup.

    For each of the options in the pipeline and workflow dialogs described below, select a pipeline and workflow from the list and select the desired option.

    The header bar for workflows and pipelines is almost identical (from left to right).