Auto-Create Multiple Blocks in a Terraform Resource

Learn how to automatically create multiple Terraform resources, and multiple blocks within one Terraform resource, using the for_each meta-argument and dynamic blocks.

Creating Multiple Resources

When using Terraform in a realistic environment (e.g. not a lab or tutorial), you often need to create automatically create multiple resources based on a list or map of data. This article reviews the options for creating resources based on data.There are two general methods to choose from when creating multiple resources:

• for_each creates multiple resources from data defined in a map or set of strings
• count creates the number of resources you specify

The count meta-argument is the simplest option if you need multiple resources that are identical except for a numeric identifier. For example, you can create “n” identical web servers in a pool.

The for_each meta-argument is necessary if there’s any difference between the resources other than the numeric identifier. For example, you might need to create multiple buckets in Amazon S3 or Cloud Storage, each with a unique bucket name, and each might have a unique URL to access it.

Check out this article for a detailed explanation of count and for_each in Terraform. The article is from Spacelift.io, which provides a cloud-based SaaS alternative to Terraform Cloud or Atlantis. Spacelift looks interesting, but I have not used it, and I have no affiliation with them.

Creating Multiple Blocks Within a Resource

Sometimes, you want to create a single resource but populate it with data from a map. In this case, you cannot use for_each, because it would create multiple resources. For example, consider a load balancer that handles requests for multiple domains and routes each domain to a different backend. How can you define the domains and backends in data, rather than hard-coding them in the load balancer resource?

Dynamic Blocks allow you to use the for_each meta-argument inside of a resource. The documentation around Dynamic Blocks is a little confusing, so here’s an example of using Dynamic Blocks to define host rules and path matchers from a map for a Google Cloud Load Balancer. The goal is to define multiple subdomains and map each subdomain to a Google Cloud Storage backend bucket. First, let’s define the data we need in a map:

backend_buckets = {
  "default" = {
    url = "default-subdomain.example.com"
  },
  "another-subdomain" = {
    url = "somethingelse.example.com"
  },
}

I’m using a map here because the subdomains don’t follow a simple counting pattern (1,2,3,…). For this module, we’ll assume the buckets have been defined in a map called buckets_map that is output from another module. Now, we’ll use for_each to create multiple backend buckets from the map:

resource "google_compute_backend_bucket" "backend" {
  for_each    = var.buckets_map
  name        = each.key
  bucket_name = each.value.name
  enable_cdn  = var.enable_cdn
}

When you use for_each to create resources from a map, you get resources that are arranged like a map, and you access them with map syntax, as shown below. Now, we’ll create ONE load balancer, with multiple host_rule and path_matcher blocks:

resource "google_compute_url_map" "load_balancer" {
  name            = "load-balancer"
  default_service = google_compute_backend_bucket.backend["default"].id

  # Create one host rule for each bucket and URL
  dynamic "host_rule" {
    for_each = var.backend_buckets
    content {
      hosts        = [host_rule.value.url]
      path_matcher = host_rule.key
    }
  }

  # Create one path matcher for each bucket and URL
  dynamic "path_matcher" {
    for_each = var.backend_buckets
    content {
      name            = path_matcher.key
      default_service = google_compute_backend_bucket.backend[path_matcher.key].id
    }
  }
}

Let’s see the resource that these dynamic blocks generates:

resource "google_compute_url_map" "load_balancer" {
    creation_timestamp = "2021-11-10T13:02:46.040-08:00"
    default_service    = REDACTED
    id                 = REDACTED
    map_id             = REDACTED
    name               = "load-balancer"
    project            = REDACTED
    self_link          = REDACTED

    host_rule {
        hosts        = [
            "default-subdomain.example.com",
        ]
        path_matcher = "default"
    }
    host_rule {
        hosts        = [
            "somethingelse.example.com",
        ]
        path_matcher = "another-subdomain"
    }

    path_matcher {
        default_service = REDACTED
        name            = "default"
    }
    path_matcher {
        default_service = REDACTED
        name            = "another-subdomain"
    }
}

As the Terraform docs warn, be careful when using for_each or Dynamic Blocks to ensure that your code remains readable.