Configure Redis data store connection

This topic describes how to configure Terraform Enterprise connection to an externally-managed Redis data store. This step is only necessary when operating Terraform Enterprise in an active-active operational mode. To allow Terraform Enterprise to self-manage Redis, configure Terraform Enterprise to run in disk operational mode on a compatible runtime platform, such as Docker or Podman. Refer to Configure operational mode for additional information.

Introduction

Terraform Enterprise uses Redis to cache and manage the background job scheduler queue across available hosts. Redis server configuration is required for any runtime platform configured to operate in active-active mode. You can operate Terraform Enterprise in active-active mode on the following runtime platforms:

Kubernetes
OpenShift
Nomad

Requirements

Before proceeding, ensure that your environment meets the following requirements.

Redis server

Either a cloud-managed or self-hosted Redis server is required.
For self-hosted servers, Terraform Enterprise supports the following verions:
- Redis 6.2 or later
- Redis 7. We recommend using version 7.
Redis Cluster is not supported.

Example Redis servers:

Your managed Redis server may require additional configuration. For example, Azure Managed Redis is based on Redis Enterprise, which has additional authentication configuration requirements. Refer to the documentation for your distribution for more information.

Secure Redis servers

For secure Redis servers, create a user with read and write access. The Redis user must have sufficient permissions to authenticate with Terraform Enterprise.

TLS requirements

Verify that you meet the following requirements when TLS is required to connect to the Redis server:

A valid TLS certificate and key are configured on the Redis server.
The Redis server is properly configured to accept TLS connections.
The Terraform Enterprise client is configured to use TLS when connecting to Redis. Refer to the TLS configuration reference for additional information.

You can configure Terraform Enterprise to use Redis client certificates to authenticate with the server for Redis standalone, Enterprise, and Sentinel. When providing client certificates, the password is not required.

Configure the following settings to use Redis client certificates:

Set the TFE_REDIS_USE_MTLS variable to true.
Specify the path to the CA certificate file in the TFE_REDIS_CA_CERT_PATH variable.
Specify the path to the client certificate file in the TFE_REDIS_CLIENT_CERT_PATH variable.
Specify the path to the client key in the TFE_REDIS_CLIENT_KEY_PATH variable.

For detailed information on configuring TLS for Redis, refer to the official Redis documentation on encryption.

Azure requirements

For Azure, you must provision a Redis instance for the main Redis service and an additional instance for Sidekiq. This is because the managed services do not support numbered databases.

Configure both the standard Redis environment variables and their corresponding Sidekiq equivalents to ensure both instances are properly configured with identical settings.

For example, a configuration that includes the TFE_REDIS_HOST and TFE_REDIS_USE_TLS variables must also define TFE_REDIS_SIDEKIQ_HOST and TFE_REDIS_SIDEKIQ_USE_TLS.

Refer to the Redis configuration reference for all available settings.

Specify Redis connection settings

You can connect to either a Redis standalone instance or a Redis Enterprise instance in non-clustering mode.

Authentication

You can using the following methods to configure authentication.

Default user and password

You can configure Redis to use the default user and require a password.

requirepass <your password>

When using the default user, configure the following settings for Terraform Enterprise:

TFE_REDIS_USE_AUTH: true
TFE_REDIS_PASSWORD: <your password>

ACL

You can also configure Redis to use an ACL. Refer to the Redis documentation for more information.

user <your user> on ><your password> ~* &* +@all

Warning

Redis users must have sufficient permissions to authenticate with Terraform Enterprise. We succesfully tested a user with permissions set to ~* &* +@all.

When using an ACL, configure the following environment variables in Terraform Enterprise:

TFE_REDIS_USE_AUTH: true
TFE_REDIS_USER: <your user>
TFE_REDIS_PASSWORD: <your password>

Azure MSI

You can also configure Terraform Enterprise to use Azure MSI for authentication.

You can use both system-assigned and user-assigned managed identities to authenticate to Azure Cache for Redis. For more information, refer to the Azure documentation. When using Azure MSI authentication, the Redis password is not required.

Configure the following settings to use Azure MSI:

Set the TFE_REDIS_PASSWORDLESS_AZURE_USE_MSI variable to true.
Set the TFE_REDIS_SIDEKIQ_PASSWORDLESS_AZURE_USE_MSI variable to true if your Sidekiq Redis instance also requires Azure MSI for authentication with Terraform Enterprise.
Set the TFE_REDIS_PASSWORDLESS_AZURE_CLIENT_ID to the user-assigned managed identity. Leave blank to use the system-assigned MSI. This is required to generate a token with Microsoft Entra ID.
Set the TFE_REDIS_SIDEKIQ_PASSWORDLESS_AZURE_CLIENT_ID to the user-assigned managed identity. Leave blank to use the system-assigned MSI. Defaults to the same value as TFE_REDIS_PASSWORDLESS_AZURE_CLIENT_ID if not set.
Set the TFE_REDIS_USER to the managed identity OID. Redis cache expects this while issuing the AUTH command.
Set the TFE_REDIS_SIDEKIQ_USER to the managed identity OID. Redis cache expects this while issuing the AUTH command. Defaults to the same value as TFE_REDIS_USER if not set.
Make sure to set TFE_REDIS_USE_TLS and TFE_REDIS_SIDEKIQ_USE_TLS variables to true if your Redis server requires TLS. TFE_REDIS_SIDEKIQ_USE_TLS will default to TFE_REDIS_USE_TLS if not set.

AWS IAM

You can also configure Terraform Enterprise to use AWS IAM to authenticate to Redis.

For more information, refer to the AWS documentation. When using AWS IAM authentication, the Redis password is not required.

Configure the following settings to use AWS IAM:

Set the TFE_REDIS_PASSWORDLESS_AWS_USE_INSTANCE_PROFILE variable to true.
Set the TFE_REDIS_SIDEKIQ_PASSWORDLESS_AWS_USE_INSTANCE_PROFILE variable to true if your Sidekiq Redis instance also requires AWS IAM for authentication with Terraform Enterprise.
Set the TFE_REDIS_PASSWORDLESS_AWS_REGION variable to the AWS region of the Redis resource.
Set the TFE_REDIS_SIDEKIQ_PASSWORDLESS_AWS_REGION variable to the AWS region of the Redis resource. Defaults to the same value as TFE_REDIS_PASSWORDLESS_AWS_REGION if not set.
Set the TFE_REDIS_PASSWORDLESS_AWS_HOST_NAME to the name of the Redis instance.
Set the TFE_REDIS_SIDEKIQ_PASSWORDLESS_AWS_HOST_NAME to the name of the Redis instance. Defaults to the same value as TFE_REDIS_PASSWORDLESS_AWS_HOST_NAME if not set.
Set the TFE_REDIS_USER to the user with IAM enabled.
Set the TFE_REDIS_SIDEKIQ_USER to the user with IAM enabled. Defaults to the same value as TFE_REDIS_USER if not set.
Make sure to set TFE_REDIS_USE_TLS and TFE_REDIS_SIDEKIQ_USE_TLS variables to true if your Redis server requires TLS. TFE_REDIS_SIDEKIQ_USE_TLS defaults to TFE_REDIS_USE_TLS if not set.

Redis Standalone

Add the following settings to your Terraform Enterprise configuration:

Set the TFE_REDIS_HOST variable to the location of your Redis server. Format the location as HOST[:PORT], for example redis.example.com or redis.example.com:6379.
Set the TFE_REDIS_USE_TLS variable to true if your Redis server requires TLS. Defaults to false.
Set the TFE_REDIS_USE_AUTH variable to true if your Redis server requires authentication.
Set the TFE_REDIS_PASSWORD variable to the password for the user.
Set the TFE_REDIS_USE_MTLS variable to use mtls with Redis standalone. Defaults to false. When this is true, both TFE_REDIS_USE_TLS and TFE_REDIS_USE_AUTH must be false.
Set the TFE_REDIS_CLIENT_CERT_PATH variable to set the path to the client certificate file to be used for mTLS authentication with Redis server.
Set the TFE_REDIS_CA_CERT_PATH variable to set the path to the Certificate Authority file user to validate the certificate.
Set the TFE_REDIS_CLIENT_KEY_PATH variable to set the path to the private key file corresponding to the client certificate.

Refer to Redis settings in the configuration reference for additional information.

Redis Enterprise

Terraform Enterprise can use Redis Enterprise in non-clustering mode as its Redis service. To do so, you must configure a separate Redis endpoint for sidekiq, an internal component. This requirement exists because sidekiq and other components that rely on Redis must be kept separate. In normal operation, this is accomplishing using numbered Redis databases, which are not supported in Redis Enterprise. By defining a separate endpoint for sidekiq usage, Terraform Enterprise will use the default database 0 while still maintaining separation between sidekiq and other components.

Add the following settings to your Terraform Enterprise configuration:

Set the TFE_REDIS_SIDEKIQ_HOST variable to the location of your Redis server. Format the location as HOST[:PORT], for example redis.example.com or redis.example.com:6379.
Set the TFE_REDIS_SIDEKIQ_USE_TLS variable to true if your Redis server requires TLS. Defaults to false.
Set the TFE_REDIS_SIDEKIQ_USE_AUTH variable to true if your Redis server requires authentication.
Set the TFE_REDIS_SIDEKIQ_PASSWORD variable to the password for the user.
Set the TFE_REDIS_SIDEKIQ_USE_MTLS variable to use mtls with Redis Enterprise. Defaults to false.
Set the TFE_REDIS_SIDEKIQ_CLIENT_CERT_PATH variable to set the path to the client certificate file to be used for mTLS authentication with Redis server.
Set the TFE_REDIS_SIDEKIQ_CA_CERT_PATH variable to set the path to the Certificate Authority file user to validate the certificate.
Set the TFE_REDIS_SIDEKIQ_CLIENT_KEY_PATH variable to set the path to the private key file corresponding to the client certificate.

Redis Sentinel

Warning

Redis Sentinel is not supported for Terraform Enterprise on Replicated.

Terraform Enterprise can use Redis Sentinel as a highly available Redis service. Read more about highly available Redis services with Redis Sentinel in the Redis Sentinel documentation.

Terraform Enterprise queries Redis Sentinel instances to determine which Redis instance is active master. Terraform Enterprise performs queries while Redis replicates transaction data to other replicas.

You cannot use Redis Sentinel and Redis Enterprise in the same Terraform Enterprise deployment.

Use the following settings in your Terraform Enterprise configuration to use Redis Sentinel. Refer to the Configuration reference for information about all configuration settings:

Set the TFE_REDIS_SENTINEL_ENABLED variable to true in order to use Redis Sentinel.
Set the TFE_REDIS_SENTINEL_HOSTS variable to a comma separated list of the locations of Redis Sentinel hosts. Format the locations as HOST[:PORT],HOST[:PORT],..., for example redis-sentinel-1.example.com,redis-sentinel-2.example.com:26379.
Set the TFE_REDIS_SENTINEL_LEADER_NAME variable to the name of a service, such as main. Terraform Enterprise queries Redis Sentinel for the service to discover an active Redis host. This name should return a valid Redis service location when issuing a SENTINEL GET-MASTER-ADDR-BY-NAME <TFE_REDIS_SENTINEL_LEADER_NAME> command to Redis Sentinel.
Set the TFE_REDIS_SENTINEL_USERNAME variable to the username for the Redis Sentinel user. This setting is optional and is used to authenticate with Redis Sentinel instances.
Set the TFE_REDIS_SENTINEL_PASSWORD variable to the password for the Redis Sentinel user. This setting is optional and is used to authenticate with Redis Sentinel instances.
Set the TFE_REDIS_USER variable to the username for the Redis user. This setting is optional and is used to authenticate with Redis instances.
Set the TFE_REDIS_PASSWORD variable to the password for the Redis Sentinel user. This setting is optional and is used to authenticate with Redis instances.
Set the TFE_REDIS_USE_MTLS variable to use mtls with Redis Sentinel. Defaults to false. When this is true, both TFE_REDIS_USE_TLS and TFE_REDIS_USE_AUTH must be false.
Set the TFE_REDIS_CLIENT_CERT_PATH variable to set the path to the client certificate file to be used for mTLS authentication with Redis server.
Set the TFE_REDIS_CA_CERT_PATH variable to set the path to the Certificate Authority file user to validate the certificate.
Set the TFE_REDIS_CLIENT_KEY_PATH variable to set the path to the private key file corresponding to the client certificate.

Failover benchmarks for Redis Sentinel

We tested failover performance for a Terraform Enterprise deployment connected to a Redis Sentinel cluster. The cluster consisted of three Sentinel instances and three Redis instances. We observed the following outcomes:

Failover events that were manually triggered through Redis Sentinel showed no observable down time in Terraform Enterprise. This suggests a high degree of safety in planned maintenance operations.
The recovery time objective (RTO) during manually triggered failover events ranged from 47s to 2m10s.
The average RTO was 1m16s across successful failovers.
One out of 16 failovers experienced issues that required Terraform Enterprise node restarts to resolve continued run failures.

Terraform Enterprise performance when using Redis Sentinel depends on how Redis Sentinel is configured to monitor and recover from Redis instance problems.