Azure Deployments via Terraform with Circle CI

In a previous post I detailed hosting Foundry VTT in Azure. I didn’t go into detail about how the actual deployment was executed, as I thought it would make a post of its own, particularly as I ended up throwing Circle CI into the mix. So here is that post.

Although this particular example is for deploying Foundry, it shows how to apply any Terraform to Azure via Circle CI.

Configuring Azure authentication for the Terraform Orb

In order for Terraform to authenticate with Azure RM, we need credentials for a Service Principal, and to define these in ARM_* environment variables.

Azure RM Service Principal Credentials

Run the following Azure CLI command to generate a Service Principal:

$ az ad sp create-for-rbac --scopes /subscriptions/<Subscription ID> --name <Service Principal Name>
{
  "appId": "<redacted>",
  "displayName": "<redacted>",
  "name": "http://<redacted>",
  "password": "<redacted>",
  "tenant": "<redacted>"
}

More info, including how to reduce the scope is in the official documentation: https://learn.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli

The output from the output JSON above will give you the values you need for the environment variables Terraform uses to authenticate with Azure:

• ARM_CLIENT_ID - taken from appId
• ARM_CLIENT_SECRET - taken from password
• ARM_TENANT_ID - taken from tenant
• ARM_SUBSCRIPTION_ID - your Subscription ID

The official documentation for this is here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret.

Installing Azure CLI

We don’t need to worry about installing Azure CLI, as my Circle CI config will use the cimg/azure:2023.03 image, which includes it.

Provisioning the Terraform Backend

I have used the Azure RM backend to persist the Terraform state in between applies, so you will need to provision an Azure Storage Account for it, and change the appropriate values in https://github.com/mowen/foundryvtt-azure/blob/main/terraform/main.tf:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.9.0"
    }
  }

  backend "azurerm" {
    resource_group_name  = "<your resource group>"
    storage_account_name = "<your terraform backend storage account>"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Foundry VTT Authentication and License Key

We also need some FOUNDRY_* environment variables for the containers to authenticate with Foundry. I pass these in as inputs variables to Terraform:

• TF_VAR_FOUNDRY_ADMIN_KEY
• TF_VAR_FOUNDRY_PASSWORD
• TF_VAR_FOUNDRY_USERNAME
• TF_VAR_FOUNDRY_LICENSE_KEY

They are defined as inputs here:

variable "FOUNDRY_ADMIN_KEY" {
  type      = string
  sensitive = true
}

variable "FOUNDRY_PASSWORD" {
  type      = string
  sensitive = true
}

variable "FOUNDRY_USERNAME" {
  type      = string
  sensitive = true
}

variable "FOUNDRY_LICENSE_KEY" {
  type      = string
  sensitive = true
}

And then used in foundry.tf:

secure_environment_variables = {
  "FOUNDRY_USERNAME"    = var.FOUNDRY_USERNAME
  "FOUNDRY_PASSWORD"    = var.FOUNDRY_PASSWORD
  "FOUNDRY_ADMIN_KEY"   = var.FOUNDRY_ADMIN_KEY
  "FOUNDRY_LICENSE_KEY" = var.FOUNDRY_LICENSE_KEY
}

Required Environment Variables

Once we have values for all of environment variables, they should be added to the Project Settings in Circle CI:

The Terraform Config

Without going into too much detail about what Terraform is deploying in this example, I have two .tf files, one for persistent resources, and one for ephemeral ones. In the deploy step both are included, and in the destroy step only the persistent one is, so that the ephemeral ones are destroyed.

The Terraform code is publicly available here: https://github.com/mowen/foundryvtt-azure/tree/main/terraform

Circle CI Workflows

The Circle CI config.yml defines two workflows:

• deploy-foundry - to provision the Foundry Container Group
• destroy-foundry - to destroy the Foundry Container Group

So you run deploy-foundry before starting your game, and then once it is finished you run destroy-foundry to destroy the compute resources and save costs.

# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1

orbs:
    terraform: circleci/terraform@3.2.0

jobs:
    terraform-plan:
        docker:
            - image: cimg/azure:2023.03
        steps:
            - checkout
            - terraform/init:
                  path: terraform
            - terraform/plan:
                  path: terraform

    terraform-apply:
        docker:
            - image: cimg/azure:2023.03
        steps:
            - checkout
            - terraform/init:
                  path: terraform
            - terraform/apply:
                  path: terraform

    terraform-destroy-foundry:
        docker:
            - image: cimg/azure:2023.03
        steps:
            - checkout
            - terraform/init:
                  path: terraform
            - run: rm terraform/foundry.tf
            - terraform/apply:
                  path: terraform

workflows:
    deploy-foundry:
        jobs:
            - terraform-plan
            - hold:
                  type: approval
                  requires:
                      - terraform-plan
            - terraform-apply:
                  requires:
                      - hold
    destroy-foundry:
        jobs:
            - hold:
                  type: approval
            - terraform-destroy-foundry:
                  requires:
                      - hold

Not the - run: rm terraform/foundry.tf line which removes the foundry.tf file prior to the terraform apply in the terraform-destroy-foundry job.

Final Circle CI Pipeline

The pipeline in the Circle CI UI should look like this:

Both the terraform-apply in deploy-foundry and the terraform-destroy-foundry in destroy-foundry have a hold step before them, so you can via the Terraform plan in the UI and confirm it is as expected before proceeding.

So with this simple pipeline I can deploy my Foundry instance when I want to run a game, and destroy it straight after, all via the Circle CI web UI.