HashiConf The full conference agenda's now LIVE. Buy your pass and plan your schedule. Register
Terraform
Terraform Home

You are viewing documentation for pre-release version v1.14.x (beta). View latest version.

Import existing resources in bulk

You can configure queries that instruct Terraform to search your existing infrastructure for unmanaged resources. Terraform can also generate configuration for importing the resources it finds so that you can import them to your Terraform workspace in bulk. For information about importing single resources or small batches of resources, refer to Import single resources for instructions.

Introduction

For organizations with large sets of infrastructure resources, manually identifying and importing them is tedious and labor intensive, even when using third-party tools or custom scripts. To alleviate this burden, you can write HCL-based queries and run them with the Terraform CLI to retrieve unmanaged resources so that you can import them in bulk.

Complete the following steps to find and import resources in bulk:

  • Search for resources: Create list blocks to search for existing resources.
  • Generate configuration: Use the Terraform command to generate resource and import blocks for importing the resources. The configuration also includes the resource identity.
  • Import resources: Copy the generated configuration into your main.tf file and run a terraform apply command to import the resources.
  • After importing resources, you can remove the generated import block or keep it as an historical record.

Define a query

To search for resources, you must create a standard .tf configuration and a .tfquery.hcl configuration.

Standard configurations

If your .tf configuration does not already have a required_providers block, add it so that Terraform can install all provider plugins required to create and manage resources specified in the configuration. Refer to the required_providers reference for details on how to configure this block.

Query configurations

Create a file with a .tfquery.hcl extension and add list blocks to define queries that read existing infrastructure and return lists of unmanaged resources.

Each list block requires an inline argument that specifies the type of resource you are querying. The type is specific to your provider. The block also requires an inline argument that declares a local name for the list of results returned by the query. Refer to the list block reference for configuration details.

The following example defines a list block named prod that queries aws_instances:

list "aws_instance" "prod" {
  # . . .
}

Add the following arguments to your list block:

  • provider: This argument is required. It specifies which provider configuration Terraform should use to perform the query. The list block retrieves the provider configuration from the terraform block in your main.tf configuration, but you can declare a provider block in your query file with an alternate configuration and reference it in the provider argument in your list block.

  • config: Add the config block to your list block and define provider-specific arguments to build your query. Refer to your provider documentation for supported arguments and values.

  • include_resource: By default, Terraform retrieves only resource identities, but you can set the include_resource argument to true so that Terraform can also retrieve all available resource attributes. To reference resource attributes retrieved by the list block, you must enable the include_resource argument. Setting this argument to true may affect performance.

  • limit: By default, Terraform retrieves up to 100 resources per list block, but you can use the limit argument to specify a higher or lower number of results.

The list block also supports several Terraform meta-arguments, which are arguments built into Terraform configuration language that configure how Terraform creates and manages infrastructure objects. For example, you can use the count meta-argument to create multiple instances of the resource list returned by the query. Refer to Meta-arguments for more information.

In the following example, Terraform applies provder arguments to query for aws_instance resources in the us-east-2 region. The query also filters results prefixed with prod- and stage- and that are in a running state. The limit argument, which is not a provider-specific argument, instructs Terraform to return the first 50 results:

list "aws_instance" "prod" {
  provider = aws
  limit = 50
  config {
    region = "us-east-2"
    filter {
      name   = "tag:Name"
      values = ["prod-*", "staging-*"]
    }
    filter {
        name = "instance-state-name"
        values = ["running"]
    }
  }
}

Parameterize your query configuration

You can parameterize your .tfquery.hcl file so that you can reuse it with a different set of inputs. Refer to Set configuration parameters for instructions on how to parameterize configurations.

Add the following blocks to your query file to parameterize the configuration:

  • variable: Add the variable block to your query file to define variables that people who run the query configuration can provide at runtime. Refer to the variable block reference for configuration details.
  • locals: Add the locals block to your query file to define temporary variables scoped to the query configuration. Refer to the locals block reference for configuration details.

Specify a custom provider configuration

The query file checks the main.tf configuration for provider configurations, but you can also declare one or more provider blocks in the query configuration file to define alternate provider configurations. To use one of the alternate provider configurations, add the provider argument to your list block and reference the name of the provider configuration. Refer to the provider block reference for information about how to configure provider blocks.

Query the infrastructure

Run the terraform query command to retrieve the resource types specified in your list blocks. The command prints the results to your console. By default, each result includes the following information:

  • Reference to the list block that queried for resources formatted as list.<type>.<label>.
  • Identity of a discovered resource.

The provider may also include other information, such as a description of the resource. Refer to your provider documentation for details.

The number of results depends on the provider, as well as any query constraints you configured in the query configuration file.

To generate machine-readable results, you can include the -json flag:

$ terraform query -json

Generate configuration for importing results

To generate configuration, run the terraform query command and add the -generate-config-out flag. The flag expects a path where Terraform stores the generated configuration as a file named generated.tf. The generated.tf file contains the resource and import blocks, including resource identities, necessary for importing results.

The following example generates configuration to the working directory:

$ terraform query -generate-config-out=generated.tf

After generating the results file, you must remove the file before rerunning the command to generate a new results file. Rerunning the command when a generated.tf already exists at the specified path results in an error.

Import resources

Copy the import and resource blocks from the generated.tf file to your main.tf configuration and run the terraform apply command to import the resources.

Next steps

After importing your resources, you can discard the query configuration file and the generated.tf file, but you can archive them to maintain a historical record of the import.

Edit this page on GitHub