HashiConf 2024 Now streaming live from Boston! Attend for free
Consul
Consul Home

You are viewing documentation for version v1.13.x. View latest version.

Vault as a Service Mesh Certificate Authority

Consul can be used with Vault to manage and sign certificates. The Vault CA provider uses the Vault PKI secrets engine to generate and sign certificates.

This page documents the specifics of the Vault CA provider. Please read the certificate management overview page first to understand how Consul manages certificates with configurable CA providers.

Tip: Complete the tutorial to learn how to configure Vault as the Consul Connect service mesh Certification Authority.

Requirements

Prior to using Vault as a CA provider for Consul, the following requirements must be met:

  • Vault 0.10.3 or later. Consul uses URI SANs in the PKI engine which were introduced in Vault 0.10.3. Prior versions of Vault are not compatible with Connect.

Configuration

The Vault CA is enabled by setting the CA provider to "vault" and setting the required configuration values.

The configuration may either be provided in the agent's configuration file using the ca_provider and ca_config options, or configured using the /connect/ca/configuration API endpoint.

Example configurations are shown below:

Connect CA configuration

/etc/consul.d/config.hcl

# ...
connect {
  enabled = true
  ca_provider = "vault"
  ca_config {
    address = "http://localhost:8200"
    token = "..."
    root_pki_path = "connect-root"
    intermediate_pki_path = "connect-intermediate"
  }
}

The configuration options are listed below.

NOTE: The first key is the value used in API calls, and the second key (after the /) is used if you are adding the configuration to the agent's configuration file.

  • Address / address (string: <required>) - The address of the Vault server.

  • Token / token (string: "") - A token for accessing Vault. This is write-only and will not be exposed when reading the CA configuration. This token must have proper privileges for the PKI paths configured. In Consul 1.8.5 and later, if the token has the renewable flag set, Consul will attempt to renew its lease periodically after half the duration has expired.

    Warning: You must either provide a token or configure an auth method below.

  • AuthMethod / auth_method (map: nil) - Vault auth method to use for logging in to Vault. Please see Vault Auth Methods for more information on how to configure individual auth methods. If auth method is provided, Consul will obtain a new token from Vault when the token can no longer be renewed.

    • Type/ type (string: "") - The type of Vault auth method.

    • MountPath/ mount_path (string: <AuthMethod.Type>) - The mount path of the auth method. If not provided the auth method type will be used as the mount path.

    • Params/params (map: nil) - The parameters to configure the auth method. Please see Vault Auth Methods for information on how to configure the auth method you wish to use. If using the Kubernetes auth method, Consul will read the service account token from the default mount path /var/run/secrets/kubernetes.io/serviceaccount/token if the jwt parameter is not provided.

  • RootPKIPath / root_pki_path (string: <required>) - The path to a PKI secrets engine for the root certificate.

    If the path does not exist, Consul will mount a new PKI secrets engine at the specified path with the RootCertTTL value as the root certificate's TTL. If the RootCertTTL is not set, a max_lease_ttl of 87600 hours, or 10 years is applied by default as of Consul 1.11 and later. Prior to Consul 1.11, the root certificate TTL was set to 8760 hour, or 1 year, and was not configurable. The root certificate will expire at the end of the specified period.

    When WAN Federation is enabled, each secondary datacenter must use the same Vault cluster and share the same root_pki_path with the primary datacenter.

    To use an intermediate certificate as the primary CA in Consul initialize the RootPKIPath in Vault with a PEM bundle. The first certificate in the bundle must be the intermediate certificate that Consul will use as the primary CA. The last certificate in the bundle must be a root certificate. The bundle must contain a valid chain, where each certificate is followed by the certificate that authorized it.

  • RootPKINamespace / root_pki_namespace (string: <optional>) - The absolute namespace that the RootPKIPath is in. Setting this parameter overrides the Namespace option for the RootPKIPath. Introduced in 1.12.3.

  • IntermediatePKIPath / intermediate_pki_path (string: <required>) - The path to a PKI secrets engine for the generated intermediate certificate. This certificate will be signed by the configured root PKI path. If this path does not exist, Consul will attempt to mount and configure this automatically.

    When WAN Federation is enabled, every secondary datacenter must specify a unique intermediate_pki_path.

  • IntermediatePKINamespace / intermediate_pki_namespace (string: <optional>) - The absolute namespace that the IntermediatePKIPath is in. Setting this parameter overrides the Namespace option for the IntermediatePKIPath. Introduced in 1.12.3.

  • CAFile / ca_file (string: "") - Specifies an optional path to the CA certificate used for Vault communication. If unspecified, this will fallback to the default system CA bundle, which varies by OS and version.

  • CAPath / ca_path (string: "") - Specifies an optional path to a folder containing CA certificates to be used for Vault communication. If unspecified, this will fallback to the default system CA bundle, which varies by OS and version.

  • CertFile / cert_file (string: "") - Specifies the path to the certificate used for Vault communication. If this is set, then you also need to set key_file.

  • KeyFile / key_file (string: "") - Specifies the path to the private key used for Vault communication. If this is set, then you also need to set cert_file.

  • TLSServerName / tls_server_name (string: "") - Specifies an optional string used to set the SNI host when connecting to Vault via TLS.

  • TLSSkipVerify / tls_skip_verify (bool: false) - Specifies if SSL peer validation should be enforced.

  • Namespace / namespace (string: <optional>) - The Vault Namespace that the Token and PKI Certificates are a part of. Vault Namespaces are a Vault Enterprise feature. Added in Consul 1.11.0

Common CA Config Options

The following configuration options are supported by all CA providers:

  • CSRMaxConcurrent / csr_max_concurrent (int: 0) - Sets a limit on the number of Certificate Signing Requests that can be processed concurrently. Defaults to 0 (disabled). This is useful when you want to limit the number of CPU cores available to the server for certificate signing operations. For example, on an 8 core server, setting this to 1 will ensure that no more than one CPU core will be consumed when generating or rotating certificates. Setting this is recommended instead of csr_max_per_second when you want to limit the number of cores consumed since it is simpler to reason about limiting CSR resources this way without artificially slowing down rotations. Added in 1.4.1.

  • CSRMaxPerSecond / csr_max_per_second (float: 50) - Sets a rate limit on the maximum number of Certificate Signing Requests (CSRs) the servers will accept. This is used to prevent CA rotation from causing unbounded CPU usage on servers. It defaults to 50 which is conservative – a 2017 MacBook can process about 100 per second using only ~40% of one CPU core – but sufficient for deployments up to ~1500 service instances before the time it takes to rotate is impacted. For larger deployments we recommend increasing this based on the expected number of server instances and server resources, or use csr_max_concurrent instead if servers have more than one CPU core. Setting this to zero disables rate limiting. Added in 1.4.1.

  • LeafCertTTL / leaf_cert_ttl (duration: "72h") - The upper bound on the lease duration of a leaf certificate issued for a service. In most cases a new leaf certificate will be requested by a proxy before this limit is reached. This is also the effective limit on how long a server outage can last (with no leader) before network connections will start being rejected. Defaults to 72h. This value cannot be lower than 1 hour or higher than 1 year.

    This value is also used when rotating out old root certificates from the cluster. When a root certificate has been inactive (rotated out) for more than twice the current leaf_cert_ttl, it will be removed from the trusted list.

  • RootCertTTL / root_cert_ttl (duration: "87600h") The time to live (TTL) for a root certificate. Defaults to 10 years as 87600h. This value, if provided, needs to be higher than the intermediate certificate TTL.

    This setting applies to all Consul CA providers.

    For the Vault provider, this value is only used if the backend is not initialized at first.

  • IntermediateCertTTL / intermediate_cert_ttl (duration: "8760h") The time to live (TTL) for any intermediate certificates signed by root certificate of the primary datacenter. This field is only valid in the primary datacenter. Defaults to 1 year as 8760h.

    This setting applies to all Consul CA providers.

    For the Vault provider, this value is only used if the backend is not initialized at first.

  • PrivateKeyType / private_key_type (string: "ec") - The type of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored. Currently supported options are ec or rsa. Default is ec.

    It is required that all servers in a datacenter have the same config for the CA. It is recommended that servers in different datacenters use the same key type and size, although the built-in CA and Vault provider will both allow mixed CA key types.

    Some CA providers (currently Vault) will not allow cross-signing a new CA certificate with a different key type. This means that if you migrate from an RSA-keyed Vault CA to an EC-keyed CA from any provider, you may have to proceed without cross-signing which risks temporary connection issues for workloads during the new certificate rollout. We highly recommend testing this outside of production to understand the impact, and suggest sticking to same key type where possible.

    Note: This only affects CA keys generated by the provider. Leaf certificate keys are always EC 256 regardless of the CA configuration.

  • PrivateKeyBits / private_key_bits (string: "") - The length of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored.

    Currently supported values are:

Root and Intermediate PKI Paths

The Vault CA provider uses two separately configured PKI secrets engines for managing Connect certificates.

The RootPKIPath is the PKI engine for the root certificate. Consul will use this root certificate to sign the intermediate certificate. Consul will never attempt to write or modify any data within the root PKI path.

The IntermediatePKIPath is the PKI engine used for storing the intermediate signed with the root certificate. The intermediate is used to sign all leaf certificates and Consul may periodically generate new intermediates for automatic rotation. Therefore, Consul requires write access to this path.

If either path does not exist, then Consul will attempt to mount and initialize it. This requires additional privileges by the Vault token in use. If the paths already exist, Consul will use them as configured.

Vault ACL Policies

Vault PKI can be managed by either Consul or by Vault. If you want to manually create and tune the PKI secret engines used to store the root and intermediate certificates, use Vault Managed PKI Paths. If you want to have the PKI automatically managed for you, use Consul Managed PKI Paths.

Vault Managed PKI Paths

The following Vault policy allows Consul to use pre-existing PKI paths in Vault. Consul is granted read-only access to the PKI mount points and the Root CA, but is granted full control of the Intermediate or Leaf CA for Connect clients.

In this example the RootPKIPath is connect_root and the IntermediatePKIPath is connect_inter. These values should be updated for your environment.

vault-managed-pki-policy.hcl

# Existing PKI Mounts
path "/sys/mounts" {
  capabilities = [ "read" ]
}

path "/sys/mounts/connect_root" {
  capabilities = [ "read" ]
}

path "/sys/mounts/connect_inter" {
  capabilities = [ "read" ]
}

# Needed for Consul 1.11+
path "/sys/mounts/connect_inter/tune" {
  capabilities = [ "update" ]
}

path "/connect_root/" {
  capabilities = [ "read" ]
}

path "/connect_root/root/sign-intermediate" {
  capabilities = [ "update" ]
}

path "/connect_inter/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "auth/token/renew-self" {
    capabilities = [ "update" ]
}

path "auth/token/lookup-self" {
    capabilities = [ "read" ]
}

Consul Managed PKI Paths

The following Vault policy allows Consul to create and manage the PKI paths in Vault. Consul is granted the ability to create the PKI mount points and given full control of the Root and Intermediate or Leaf CA for Connect clients.

In this example the RootPKIPath is connect_root and the IntermediatePKIPath is connect_inter. These values should be updated for your environment.

consul-managed-pki-policy.hcl

# Consul Managed PKI Mounts
path "/sys/mounts" {
  capabilities = [ "read" ]
}

path "/sys/mounts/connect_root" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "/sys/mounts/connect_inter" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Needed for Consul 1.11+
path "/sys/mounts/connect_inter/tune" {
  capabilities = [ "update" ]
}

path "/connect_root/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "/connect_inter/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}