Terraform - Use nested loops with count

Question

I am trying to use a nested loop in terraform. I have two list variables list_of_allowed_accounts and list_of_images, and looking to iterate over list list_of_images and then iterate over list list_of_allowed_accounts.

Here is my terraform code.

variable "list_of_allowed_accounts" {
  type    = "list"
  default = ["111111111", "2222222"]
}

variable "list_of_images" {
  type    = "list"
  default = ["alpine", "java", "jenkins"]
}

data "template_file" "ecr_policy_allowed_accounts" {
  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    count = "${length(var.list_of_allowed_accounts)}"
    account_id = "${element(var.list_of_allowed_accounts, count.index)}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${length(var.list_of_images)}"
  repository = "${element(aws_ecr_repository.images.*.id, count.index)}"
  count = "${length(var.list_of_allowed_accounts)}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.rendered}"
}

This is a bash equivalent of what I am trying to do.

for image in alpine java jenkins
do 
  for account_id in 111111111 2222222
  do 
    // call template here using variable 'account_id' and 'image'
  done
done

Answer 1

Terraform doesn't have direct support for this sort of nested iteration, but we can fake it with some arithmetic.

variable "list_of_allowed_accounts" {
  type = "list"
  default = ["1111", "2222"]
}

variable "list_of_images" {
  type = "list"
  default = ["alpine", "java", "jenkins"]
}

data "template_file" "ecr_policy_allowed_accounts" {
  count = "${length(var.list_of_allowed_accounts) * length(var.list_of_images)}"

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = "${var.list_of_allowed_accounts[count.index / length(var.list_of_images)]}"
    image      = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${data.template_file.ecr_policy_allowed_accounts.count}"

  repository = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.*.rendered[count.index]}"
}

Since we want to create a policy template for every combination of account and image, the count on the template_file data block is the two multiplied together. We can then use the division and modulo operations to get back from count.index to the separate indices into each list.

Since I didn't have a copy of your policy template I just used a placeholder one; this configuration thus gave the following plan:

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.0
    policy:     "policy allowing 1111 to access alpine"
    repository: "alpine"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.1
    policy:     "policy allowing 1111 to access java"
    repository: "java"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.2
    policy:     "policy allowing 1111 to access jenkins"
    repository: "jenkins"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.3
    policy:     "policy allowing 2222 to access alpine"
    repository: "alpine"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.4
    policy:     "policy allowing 2222 to access java"
    repository: "java"

+ aws_ecr_respository_policy.repo_policy_allowed_accounts.5
    policy:     "policy allowing 2222 to access jenkins"
    repository: "jenkins"

Each policy instance applies to a different pair of account id and image, covering all combinations.

Answer 2

The answers here do work (I used them initially), but I think I have a better solution using Terraform's setproduct function. I haven't seen many examples of it used around the interwebs, but setproduct takes two sets (or more importantly, two lists) and produces a list of sets with every permutation of the inputs. In my case I am creating SSM parameters:

variable "list1" {
  type    = "list"
  default = ["outer1", "outer2"]
}

variable "list2" {
  type    = "list"
  default = ["inner1", "inner2", "inner3"]
}

locals {
  product = "${setproduct(var.list1, var.list2)}"
}

resource "aws_ssm_parameter" "params" {
  count     = "${length(var.list1) * length(var.list2)}"
  name      = "/${element(local.product, count.index)[0]}/${element(local.product, count.index)[1]}"
  type      = "String"
  value     = "somevalue"
  overwrite = false
  lifecycle { ignore_changes = ["value"] }
}

This creates SSM parameters named:

/outer1/inner1
/outer1/inner2
/outer1/inner3
/outer2/inner1
/outer2/inner2
/outer2/inner3

My wimpy little brain can parse this a little easier than the modulo magic in the other answers!

Answer 3

FYI if anyone comes here from Google, if you are using terraform 0.12, you will need to use the floor function anywhere you do divide, or else you will get an error about partial indexes.

account_id = var.list_of_allowed_accounts[floor(count.index / length(var.list_of_images))]

Answer 4

While solutions with setproduct and using modulo work in some cases, there is a more elegant solution: using terraform modules (not to be confused with the modulo operator). By using a terraform module an extra opportunity arises to use the count meta-argument.

This solution is especially useful if not all combinations of inputs apply. For example, I used this solution to handle a list of projects, where each project has a list of unique roles.

For example, on the top-level you can use this:

variable "account_images" {
  type = map(list(string))
  default = {
    "1111" = ["alpine", "java"],
    "2222" = ["jenkins"]
  }
}

module "account" {
  source = "../accounts"
  count = length(var.account_images)
  account = keys(var.account_images)[count.index]
  images = var.account_images[keys(var.account_images)[count.index]]
}

And then have an additional terraform module in the ../accounts directory with this content.

variable "account" {
  type = string
}

variable "images" {
  type = list(string)
}


data "template_file" "ecr_policy_allowed_accounts" {
  count = length(var.images)}

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = var.account
    image      = var.images[count.index]
  }
}

I find this approach easier to understand.

Answer 5

Basically the problem is in the data "template_file", the account_id can not be set the way you think it will since the count in your case is just another var that never gets incremented/changed. Just saying since I miss to see what is your question exactly.

Answer 6

I don't have enough reputation points to add a comment to the answer provided by @Martin Atkins, so I am posting his answer with a slight modification, which works around Terraform issue 20567

variable "list_of_allowed_accounts" {
  type = "list"
  default = ["1111", "2222"]
}

variable "list_of_images" {
  type = "list"
  default = ["alpine", "java", "jenkins"]
}

# workaround for TF issue https://github.com/hashicorp/terraform/issues/20567
locals {
  policy_count = "${length(var.list_of_allowed_accounts) * length(var.list_of_images)}"
}

data "template_file" "ecr_policy_allowed_accounts" {
  count = "${local.policy_count}"

  template = "${file("${path.module}/ecr_policy.tpl")}"

  vars {
    account_id = "${var.list_of_allowed_accounts[count.index / length(var.list_of_images)]}"
    image      = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  }
}

resource "aws_ecr_repository_policy" "repo_policy_allowed_accounts" {
  count = "${local.policy_count}"

  repository = "${var.list_of_images[count.index % length(var.list_of_images)]}"
  policy = "${data.template_file.ecr_policy_allowed_accounts.*.rendered[count.index]}"
}