How to Structure Terraform Modules for an AI Platform

AI platform infrastructure grows quickly: networks, clusters, GPU node pools, model endpoints, vector databases, secrets and monitoring. Kept in one Terraform folder with one state file, every plan takes minutes, and a small dashboard change risks touching the network.

A clear module structure keeps changes small and safe.

Step 1: Split by layer and rate of change

infra/
  modules/
    network/
    cluster/
    gpu-nodepool/
    vector-db/
    model-access/
  live/
    prod/
      network/      # own state
      cluster/      # own state
      platform/     # vector-db, model-access, monitoring
    staging/

Things that rarely change, like networks, get their own state. Things that change often, like model access and application resources, live in a separate layer. A mistake in one layer can't destroy another.

Step 2: Keep modules small and focused

A module should do one job with a handful of inputs. gpu-nodepool takes an instance family, a maximum GPU count and some labels. It doesn't also create the cluster. Small modules are easier to review and reuse.

Step 3: Pass values between layers explicitly

Read values from other layers with remote state data sources, or look resources up by name or tag. Avoid IDs copied by hand between folders.

Step 4: Pin versions

Pin the Terraform version, provider versions and module versions:

terraform {
  required_version = "~> 1.9"
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 6.0" }
  }
}

Upgrade them on purpose, one layer at a time.

Step 5: Use the same modules everywhere

Staging and production call the same modules with different variables. If staging needs something production doesn't, add an input to the module rather than copying it.

Step 6: Run plans in CI

Every pull request runs terraform plan for the layers it touches and posts the output for review. Applies to production run from the pipeline, never from a laptop.

Things to watch

• State locking. Use a remote backend with locking so two applies can't run at the same time.
• Secrets in state. State files can contain sensitive values. Encrypt them and restrict who can read them.
• Terraform or OpenTofu. Both work with this structure. Choose one for the organization and stay consistent.

If you've inherited one large folder, move the network into its own state first. It's the layer where mistakes hurt the most.