Compatible Terraform Modules for Network Infrastructure Automation

Compatible modules for Consul-Terraform-Sync follow the standard module structure. Modules can use syntax supported by Terraform version 0.13 and newer.

Below are the two required elements for module compatibility with Consul-Terraform-Sync

Root module - Terraform has one requirement for files in the root directory of the repository to function as the primary entrypoint for the module. It should encapsulate the core logic to be used by Consul-Terraform-Sync for task automation. is the recommended filename for the main file where resources are created.
- Consul-Terraform-Sync requires all modules to have the following input variable declared within the root module. The declaration of the services variable can be included at the top of the suggested variables.tf file where other input variables are commonly declared. This variable functions as the response object from the Consul catalog API and surfaces network information to be consumed by the module. It is structured as a map of objects.

Optional Input Variables

In addition to the required services input variable, Consul-Terraform-Sync provides additional, optional input variables to be used within your module. Support for an optional input variable requires two changes:

Updating the Terraform Module to declare the input variable in the suggested variables.tf
Adding configuration to the Consul-Terraform-Sync task block to define the source input values that should be provided to the input variables

See below sections for more information on and declaring optional input variables in your Terraform module.

A source input allows for a task to satisfy the input requirements defined by the Terraform Module’s , and is configured alongside a task’s condition. Both the source input and condition define objects to be monitored, but for differing reasons.

The condition defines monitored objects with criteria. When this criteria is satisfied, Consul-Terraform-Sync will then trigger a task.

The source input however, defines monitored objects with the intent of providing values or metadata about these objects to the Terraform Module. The source input and condition objects can be the same, such as when task.services is provided without a source input block and condition block, but they can also be defined using separate blocks. In this way the source input does not need to be tied to the provided condition in order to satisfy the Terraform Module.

There are a few ways that a source input can be defined:

services list - The list of services to act as a source input.
source_includes_var condition field - If the condition supports this field, and it is set to true, then the condition’s objects will be used as a source input. For example, if a module is defined supporting then, this field can be set to true in the catalog-services condition.
- This block specifically defines a source input.

Multiple ways of defining a source input adds configuration flexibility, and allows for optional additional input variables to be supported by Consul-Terraform-Sync alongside the services input variable.

These optional input variables include:

catalog_services variable

Services Source Input

Tasks configured with a services source input monitor for changes to services. Monitoring is either performed on a configured list of services or on any services matching a provided regex.

Sample rendered services input:

terraform.tfvars

In order to configure a task with the services source input, the list of services that will be used for the input must be configured in one of the following ways:

the task’s
a condition “services” block configured with regexp and source_includes_var set to true
a configured with regexp

The services source input operates by monitoring the Health List Nodes For Service API and provides the latest service information to the input variables. A complete list of service information that would be provided to the module is expanded below:

Below is an example configuration for a task that will execute on a schedule and provide information about the services matching the regexp parameter to the task’s module. Note that because regexp is set, task.services is omitted, and since a schedule is being used to trigger task execution, a condition "services" block cannot be used.

task {
  name        = "services_condition_task"
  description = "execute on changes to services whose name starts with web"
  providers   = ["my-provider"]
  source      = "path/to/services-condition-module"
  condition "schedule" {
    cron = "* * * * Mon"
  }
    regexp = "^web.*"
  }
}

Consul KV Source Input

Tasks configured with a Consul KV source input monitor Consul KV for changes to KV pairs that satisfy the provided configuration. The Consul KV source input operates by monitoring the Consul KV API and provides these key values to the task’s module.

Sample rendered consul KV input:

consul_kv = {
  "my-key" = "some value"
}

To configure a task with the Consul KV source input, the KVs which will be used for the input must be configured in one of the following ways:

a configured with the source_includes_var set to true.
a source_input “consul-kv” block.

Below is a similar example to the one provided in the section. However, the difference in this example is that instead of triggering based on a change to Consul KV, this task will instead execute on a schedule. Once execution is triggered, Consul KV information is then provided to the task’s module.

Catalog Services Source Input

Tasks configured with a Catalog Services source input monitors for service and tag information provided by the . The source input is a map of service names to a list of tags.

Sample rendered catalog-services input:

catalog_services = {
  "consul" = []
  "web"    = ["blue", "green"]
}

terraform.tfvars

To configure a task with the Catalog Services source input, the catalog services which will be used for the input must be configured in one of the following ways:

a condition “catalog-services” block configured with source_includes_var set to true.

Note: Currently there is no support for a source_input “catalog-services” block.

Example of a catalog-services condition which supports source input through source_includes_var:

task {
  name        = "catalog_services_condition_task"
  description = "execute on registration/deregistration of services"
  providers   = ["my-provider"]
  services    = ["web-api"]
  source      = "path/to/catalog-services-module"
  condition "catalog-services" {
    datacenter          = "dc1"
    namespace           = "default"
    regexp              = "web.*"
    source_includes_var = true
    node_meta {
      key = "value"
    }
  }
}

How to Create a Compatible Terraform Module

You can read more on how to create a module or work through a . Consul-Terraform-Sync is designed to integrate with any module that satisfies the specifications in the following section.

The repository hashicorp/consul-terraform-sync-template-module can be cloned and used as a starting point for structuring a compatible Terraform module. The template repository has the files described in the next steps prepared.

First, create a directory to organize Terraform configuration files that make up the module. You can start off with creating two files main.tf and variables.tf and expand from there based on your module and network infrastructure automation needs.

The main.tf is the entry point of the module and this is where you can begin authoring your module. It can contain multiple Terraform resources related to an automation task that uses Consul service discovery information, particularly the required . The code example below shows a resource using the services variable. When this example is used in automation with Consul-Terraform-Sync, the content of the local file would dynamically update as Consul service discovery information changes.

main.tf

Something important to consider before authoring your module is deciding the condition under which it will execute. This will allow you to potentially include other types of Consul-Terraform-Sync provided input variables in your module. It will also help inform your documentation and how users should configure their task for your module.

Services Variable

To satisfy the specification requirements for a compatible module, copy the services variable declaration to the variables.tf file. Your module can optionally have other variable declarations and in addition to var.services.

variable "services" {
  description = "Consul services monitored by Consul-Terraform-Sync"
  type = map(
    object({
      id        = string
      name      = string
      kind      = string
      port      = number
      meta      = map(string)
      tags      = list(string)
      namespace = string
      status    = string
      node                  = string
      node_id               = string
      node_datacenter       = string
      node_tagged_addresses = map(string)
      node_meta             = map(string)
      cts_user_defined_meta = map(string)
    })
  )
}

variables.tf

Keys of the services map are unique identifiers of the service across Consul agents and data centers. Keys follow the format service-id.node.datacenter (or service-id.node.namespace.datacenter for Consul Enterprise). A complete list of attributes available for the services variable is included in the documentation for Consul-Terraform-Sync tasks.

variable "services" {
  description = "Consul services monitored by Consul-Terraform-Sync"
  type = map(
    object({
      id           = string
      name         = string
      node_address = string
      port         = number
      status       = string
    })
  )
}

variables.tf

If you are creating a module for a , then you have the option to add the catalog_services variable, which contains service registration and tag information. If your module would benefit from consuming this information, you can copy the catalog_services variable declaration to your variables.tf file in addition to the other variables.

variables.tf

The keys of the catalog_services map are the names of the services that are registered with Consul at the given datacenter. The value for each service name is a list of all known tags for that service.

We recommend that if you make a module with with a catalog-services condition, that you document this in the README. This way, users that want to configure a task with your module will know to configure a catalog-services condition block.

Similarly, if you include the catalog_services variable in your module, we recommend that you also document this usage in the README. Users of your module will then know to set the catalog-services condition configuration to be true. When this field is set to true, Consul-Terraform-Sync will declare the catalog_services variable in the generated root module, and pass the variable to a child module. Therefore, if this field is configured inconsistently, Consul-Terraform-Sync will error and exit.

Consul KV Variable

If you are creating a module for a , then you have the option to add the consul_kv variable, which contains a map of the keys and values for the Consul KV pairs. If your module would benefit from consuming this information, you can copy the consul_kv variable declaration to your variables.tf file in addition to the other variables.

variable "consul_kv" {
    description = "Keys and values of the Consul KV pairs monitored by Consul-Terraform-Sync"
    type        = map(string)
}

variables.tf

If your module contains the consul_kv variable, we recommend documenting the usage in the README file so that users know to set the source_includes_var configuration to true in the consul-kv condition. Setting the field to true instructs Consul-Terraform-Sync to declare the consul_kv variable in the generated root module and pass the variable to a child module. Therefore, if this field is configured inconsistently, Consul-Terraform-Sync will error and exit.

Network infrastructure differs vastly across teams and organizations, and the automation needs of practitioners are unique based on their existing setup. can be used to serve as customization parameters to the module for practitioners.

Identify areas in the module where practitioners could tailor the automation to fit their infrastructure.
Declare input variables and insert the use of variables throughout module resources to expose these options to practitioners.
Include descriptions to capture what the variables are and how they are used, and specify custom validation rules for variables to provide context to users the expected format and conditions for the variables.
Set reasonable default values for variables that are optional, or omit default values for variables that are required module arguments.

Terraform is an explicit configuration language and requires variables to be declared, typed, and passed explicitly through as module arguments. Consul-Terraform-Sync abstracts this by creating intermediate variables at the root level from the source input. These values are configured by practitioners within the . Value assignments are parsed to interpolate the corresponding variable declaration and are written to the appropriate Terraform files. A few assumptions are made for the intermediate variables: the variables users provide Consul-Terraform-Sync are declared and supported by the module, matching name and type.

Module Guidelines

This section covers guidelines for authoring compatible Consul-Terraform-Sync modules.

Scope

We recommend scoping the module to a few related resources for a provider. Small modules are easier and more flexible for end users to adopt for Consul-Terraform-Sync. It allows them to iteratively combine different modules and use them as building blocks to meet their unique network infrastructure needs.

Complexity

Consider authoring modules with low complexity to reduce the run time for Terraform execution. Complex modules that have a large number of dependencies may result in longer runs, which adds delay to the near real time network updates.

Providers

The Terraform module must declare which providers it requires within the terraform.required_providers block. We suggest to also include a version constraint for the provider to specify which versions the module is compatible with.

Aside from the required_providers block, provider configurations should not be included within the sharable module for network integrations. End users will configure the providers through Consul-Terraform-Sync, and Consul-Terraform-Sync will then translate provider configuration to the generated root module appropriately.

Documentation

Modules for Consul-Terraform-Sync are Terraform modules and can effectively run independently from the consul-terraform-sync daemon and Consul environment. They should be written and designed with Terraform best practices and should be clear to a Terraform user what the module does and how to use it. Module documentation should be named or README.md. The description should capture what the module should be used for and the implications of running it in automation with Consul-Terraform-Sync.