Organization Migration in Terraform Cloud

Streamlining Infrastructure Organization Migration in Terraform Cloud

Picture this: You’ve been tasked with setting up the infra for a startup. You’ve meticulously crafted your infrastructure in Terraform and Terraform Cloud in your personal account, finely tuned to match the requirements. But now, a new chapter unfolds as you need to migrate it to the client’s organization. The challenge? Preserving every resource, every configuration, and every crafted detail while ensuring a flawless transition. Fear not, this article will help you maintain the integrity of your infrastructure while you seamlessly integrate it into its new home.

Terraform, developed by HashiCorp, is a robust tool for defining and provisioning infrastructure as code (IaC). Using the HashiCorp Configuration Language (HCL), Terraform allows you to specify your desired infrastructure state in configuration files, covering resources like virtual machines, networks, storage, and more. Once configured, Terraform automatically manages the creation, updating, and maintenance of your infrastructure to ensure it aligns with your defined state.

For teams collaborating on infrastructure projects, Terraform Cloud offers essential support. It provides a centralized platform for managing Terraform runs, ensuring consistency and reliability across environments. With features such as shared state and secret management, access controls, and a private registry for Terraform modules, Terraform Cloud streamlines collaboration and enhances security.

In scenarios where infrastructure needs to transition between environments, such as moving from a personal account to a client’s organization, maintaining continuity and integrity is crucial. This is where leveraging Terraform Cloud’s organizational capabilities and preserving Terraform state become essential.

The following steps, will guide you through the process of transitioning your infrastructure to a new organization within Terraform Cloud, preserving your existing resources and ensuring a smooth migration.

0. Lock Your Workspace

To avoid any unexpected plan and apply running by you or a teammate, first lock your workspace. Locking prevents all applies (and many kinds of plans) from proceeding, and affects runs created via UI, CLI, API, and automated systems. You can find the option to do so in the top right of your workspace page:

1. Extract Your Current State

Initiate by pulling your current state into a local file:

terraform state pull > state-old-org.tfstate

The command above pulls the current state stored remotely in Terraform Cloud and writes it to state-old-org.tfstate.

2. Modify Backend Configuration

The Cloud block of your Terraform code specifies which organization and workspace to use. It might currently look something like this:

cloud {
  organization = "old-organization"
  hostname     = "app.terraform.io"
  workspaces {
      name = "my-workspace"
  }
}

You need to adjust the organization field to specify your new organization:

cloud {
  organization = "new-organization"
  hostname     = "app.terraform.io"
  workspaces {
      name = "my-workspace"
  }
}

You can find the value at top of the page in your Terraform Cloud:

After making any changes to the Cloud configuration block Terraform requires a re-initialization:

terraform init

This will also create a workspace if you haven’t manually created it for the new organization.

3. Migrate State

Now that at this point, you have already saved the state from the old organization and changed the code to point to the new organization. The next step is pushing the state:

terraform state push state-old-org.tfstate

This command pushes the local state saved in state-old-org.tfstate to your new organization in Terraform Cloud. Note that if the new organization is not a clean slate and there are previous states in it, this command will fail since the lineage of the state would be different (see the official docs for more info on this).

Verify the state migration by checking the States tab in your Terraform Cloud workspace:

Here you should see a new state with the same content as state-old-org.tfstate file.

4. Align Workspace Settings and Validate Migration

Ensure the new workspace mirrors the settings of its counterpart in the old organization.

For plain variables, you can easily copy the key values from the UI; for secrets, you need to either generate them again or obtain them from where you did for the original organization.

Now execute a “Plan-only” run in the new organization. If there are no errors, the state is migrated successfully and the variables are correct. Also, your plan should not yield any changes (unless the planned changes also happen when you do the Plan run in the old organization – denoting a mismatch between Terraform state and your resources.)

If you are not using version control connections, you can skip the next step.

5. Change Version Control Connections

If you are using version control integration for your workspace, you need to connect your version control system to the new organization. If unfamiliar, you can read on how to do this in the Terraform documentation here.

Before doing so, to avoid a dichotomy, you have to break the connection for the old organization. What you can do is switch the workflow to “CLI-Driven Workflow” by going to the “Version Control” setting, changing the source (“Change source” as the image below):

And then choosing the Cli-Driven Workflow:

And, that’s it!

Congratulations! You did it! You’ve navigated the twists and turns of Terraform Cloud with finesse. With each step, you’ve preserved the integrity of your infrastructure, like a skilled conductor guiding a symphony through a flawless performance. So, take a bow, maestro of the cloud! Here’s to smooth transitions, organized infrastructures, and many more adventures in the ever-evolving landscape of cloud computing!