Iterating over local files with Terraform

This has come up a couple of times in client work, so I thought I’d write down how I go about doing it. This technique can be useful when you have enviroment-specific lists of resources - such as users - that need to be created.

Consider a file users.txt:

Luke
Bo
Daisy
Jesse

This file contains a list of objects we wish to create. In this example they will be of type aws_iam_user, but the same technique can be applied for any kind of resource or provider. The file is just a list of strings, specifying nested properties is outside the scope of this example.

main.tf

We can use the file() function in Terraform to read the file into a string, then split() the string on '\n' to extract the individual lines into a list. The compact() function removes empty elements from a list, which is useful in the event that the last line of the file is terminated with a carrriage return, resulting in an empty element.

The foreach() function1 requires a map or a set to iterate over, so we convert the list with toset().

locals {
  names = toset(compact((split("\n", file("./users.txt") ))))
}


resource "aws_iam_user" "dukes" {
  for_each = local.names
  name     = each.key
    path = "/Dukes/"
 }
 

Try it out

Create the input file

$ cat <<EOF > users.txt
Luke
Bo
Daisy
Jesse
EOF

Apply the configuration

$ terraform apply -auto-approve
...
Plan: 4 to add, 0 to change, 0 to destroy.
aws_iam_user.dukes["Daisy"]: Creating...
aws_iam_user.dukes["Luke"]: Creating...
aws_iam_user.dukes["Jesse"]: Creating...
aws_iam_user.dukes["Bo"]: Creating...
aws_iam_user.dukes["Jesse"]: Creation complete after 2s [id=Jesse]
aws_iam_user.dukes["Luke"]: Creation complete after 2s [id=Luke]
aws_iam_user.dukes["Daisy"]: Creation complete after 2s [id=Daisy]
aws_iam_user.dukes["Bo"]: Creation complete after 2s [id=Bo]

Update the list in users.txt

$ cat <<EOF > users.txt
Vance
Coy
Daisy
Jesse
EOF

Apply the updated configuration

$ terraform apply -auto-approve
...
Plan: 2 to add, 0 to change, 2 to destroy.
aws_iam_user.dukes["Bo"]: Destroying... [id=Bo]
aws_iam_user.dukes["Luke"]: Destroying... [id=Luke]
aws_iam_user.dukes["Vance"]: Creating...
aws_iam_user.dukes["Coy"]: Creating...
aws_iam_user.dukes["Bo"]: Destruction complete after 2s
aws_iam_user.dukes["Vance"]: Creation complete after 2s [id=Vance]
aws_iam_user.dukes["Luke"]: Destruction complete after 2s
aws_iam_user.dukes["Coy"]: Creation complete after 2s [id=Coy]

Notes

For this to work, the users.txt file must already exist at the start of the Terraform run. Assuming that the list of users isn’t stored in the same place as the Terraform code, this means that whatever CI tool you are using will have to make sure that this file is correctly assembled and in the right place before the start of the Terraform step. The exact steps to achieve this vary widely by platform, so they are omitted here.


  1. Yes, I realise it isn’t really a function ↩︎