Upgrading to 1.6.9

    • ACL System in Legacy Mode - You can find information about legacy configuration options and differences between modes here.
    • - You can find more details around legacy ACL and new ACL configuration options here. Legacy ACL config options will be listed as deprecates as of 1.4.0.

    In this guide, we will be using an example with two datacenters (DCs) and will be referring to them as DC1 and DC2. DC1 will be the primary datacenter.

    Requirements

    • All Consul servers should be on a version of Consul >= 1.2.4 and < 1.6.9.

    This guide makes the following assumptions:

    • You have at least two datacenters configured and have ACL replication enabled. If you are not using multiple datacenters, you can follow along and simply skip the instructions related to replication.
    • You have not already performed the ACL token migration. If you have, please skip all related steps.

    Considerations

    There are quite a number of changes between releases. Notable changes are called out in our Specific Version Details page. You can find more granular details in the full . Looking through these changes prior to upgrading is highly recommended.

    Two very notable items are:

    • 1.6.2 introduced more strict JSON decoding. Invalid JSON that was previously ignored might result in errors now (e.g., in service definitions). See [GH#6680].
    • 1.6.3 introduced the limit. This defaults to 200. Prior to this, connections per client were unbounded. [GH#7159]

    1. Check the replication status of the primary datacenter (DC1) by issuing the following curl command from a consul server in that DC:

    You should receive output similar to this:

    1. {
    2.   "Enabled": false,
    3.   "Running": false,
    4.   "SourceDatacenter": "",
    5.   "ReplicatedIndex": 0,
    6.   "LastSuccess": "0001-01-01T00:00:00Z",
    7. }

    The primary datacenter (indicated by acl_datacenter) will always show as having replication disabled, so this is normal even if replication is happening.

    2. Check replication status in DC2 by issuing the following curl command from a consul server in that DC:

    1. curl -s -H "X-Consul-Token: $MASTER_TOKEN" localhost:8500/v1/acl/replication?pretty

    You should receive output similar to this:

    1. {
    2.   "Enabled": true,
    3.   "Running": true,
    4.   "SourceDatacenter": "dc1",
    5.   "ReplicatedIndex": 9,
    6.   "LastSuccess": "2020-09-10T21:16:15Z",
    7.   "LastError": "0001-01-01T00:00:00Z"
    8. }

    3. Upgrade DC2 agents to version 1.6.9 by following our . Leave all DC1 agents at 1.2.4. You should start observing log messages like this after that:

    4. Check that replication is still working in DC2.

    From a Consul server in DC2:

    1. curl -s -H "X-Consul-Token: $MASTER_TOKEN" localhost:8500/v1/acl/replication?pretty
    2. curl -s -H "X-Consul-Token: $MASTER_TOKEN" localhost:8500/v1/acl/list?pretty

    Take note of the ReplicatedIndex value.

    Create a new file containing the payload for creating a new token named test-ui-token.json with the following contents:

    1. {
    2.   "Name": "UI Token",
    3.   "Type": "client",
    4.   "Rules": "key \"\" { policy = \"write\" } node \"\" { policy = \"read\" } service \"\" { policy = \"read\" }"
    5. }

    test-ui-token.json

    From a Consul server in DC1, create a new token using that file:

    1. curl -X PUT -H "X-Consul-Token: $MASTER_TOKEN" -d @test-ui-token.json localhost:8500/v1/acl/create

    From a Consul server in DC2:

    ReplicatedIndex should have incremented and you should find the new token listed. If you try using CLI ACL commands you will receive this error:

    1. Failed to retrieve the token list: Unexpected response code: 500 (The ACL system is currently in legacy mode.)

    This is because Consul in legacy mode. ACL CLI commands will not work and you have to hit the old ACL HTTP endpoints (which is why curl is being used above rather than the consul CLI client).

    Once this is complete, you should observe a log entry like this from your server agents:

      6. Confirm that replication is still working in DC2 by issuing the following curl command from a consul server in that DC:

      1. curl -s -H "X-Consul-Token: $MASTER_TOKEN" localhost:8500/v1/acl/replication?pretty

      You should receive output similar to this:

      6. Migrate your legacy ACL tokens to the new ACL system by following the instructions in our ACL Token Migration guide.

      This step must be completed before upgrading to a version higher than 1.6.x.

      Post-Upgrade Configuration Changes

      When moving from a pre-1.4.0 version of Consul, you will find that several of the ACL-related configuration options were renamed. Backwards compatibility is maintained in the 1.6.x release series, so you are old config options will continue working after upgrading, but you will want to update those now to avoid issues when moving to newer versions.

      These are the changes you will need to make:

      • acl_datacenter is now named primary_datacenter (review our for more info)

      • acl_default_policy, acl_down_policy, , acl_*_token and enable_acl_replication options are now specified like this (review our for more info):

        1. acl {
        2.   enabled                  = true/false
        3.   default_policy           = "..."
        4.   down_policy              = "..."
        5.   policy_ttl               = "..."
        6.   role_ttl                 = "..."
        7.   enable_token_replication = true/false
        8.   enable_token_persistence = true/false
        9.   tokens {
        10.     master       = "..."
        11.     agent        = "..."
        12.     agent_master = "..."
        13.     replication  = "..."
        14.     default      = "..."
        15.   }
        16. }

      You can make sure your config changes are valid by copying your existing configuration files, making the changes, and then verifying them by using .

      Once your config is passing the validation check, replace your old config files with the new ones and slowly roll your cluster again one server at a time – leaving the leader agent for last in each datacenter.