Enable API gateway for Kubernetes

The Consul API gateway ships with Consul and can be installed directly when you install Consul on Kubernetes.

Before Consul 1.21.x, API gateways were managed by the API group gateway.networking.k8s.io. This configuration is referred to as Standard API gateway.

Consul 1.21.x introduces the Consul API gateway controller and the new API gateway CRDs under the API Group consul.hashicorp.com. This configuration is referred to as Consul API gateway.

Both controllers are supported by Consul's Helm chart.

If your environment supports it, we recommend the standard API gateway installation approach, which is the default installation choice.

Before you begin the installation process, verify that the environment you are deploying Consul and the API gateway in meets the requirements listed in the Technical Specifications. Refer to the Release Notes for any additional information about the version you are deploying.

Select your API gateway implementation

If your environment supports it, we recommend the standard API gateway installation approach, which is the default installation choice.

Use the Consul API gateway controller for the platforms that manage gateway CRDs under the API group gateway.networking.k8s.io in version v1 and is below 1.6.0 package version of the API group gateway.networking.k8s.io.

If the platform manages the gateway CRDs under the API group gateway.networking.k8s.io or if any other application or user externally installed these CRDs, set global.installK8sNetworkingCRDs: false.

Create a values.yaml file for configuring your Consul API gateway deployment and include the following settings:

values.yaml

global:
  installK8sNetworkingCRDs: true
  crds:
    enableTcpRoute: true
    consulapi:
      enabled: false
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: true

values.yaml

global:
  installK8sNetworkingCRDs: true
  openshift:
    enabled: true
    crds:
      enableTcpRoute: true
      consulapi:
        enabled: false
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: true
  cni:
    enabled: true
    logLevel: info
    multus: true
    cniBinDir: "/var/lib/cni/bin"
    cniNetDir: "/etc/kubernetes/cni/net.d"

values.yaml

global:
  installK8sNetworkingCRDs: false
  openshift:
    enabled: true
    crds:
      enableTcpRoute: false
      consulapi:
        enabled: false
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: true
  cni:
    enabled: true
    logLevel: info
    multus: true
    cniBinDir: "/var/lib/cni/bin"
    cniNetDir: "/etc/kubernetes/cni/net.d"

values.yaml

global:
  installK8sNetworkingCRDs: false
  openshift:
    enabled: true
    crds:
      enableTcpRoute: false
      consulapi:
        enabled: true
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: true
  cni:
    enabled: true
    logLevel: info
    multus: true
    cniBinDir: "/var/lib/cni/bin"
    cniNetDir: "/etc/kubernetes/cni/net.d"

By default, GKE Autopilot installs Gateway API resources.

The following working example enables both Consul service mesh and the Consul API gateway on GKE Autopilot. Refer to connectInject.agiGateway in the Helm chart reference for additional information. To support TCP route CRDs use Consul API gateway by setting global.crds.consulapi.enabled: true.

values.yaml

global:
  installK8sNetworkingCRDs: false
  crds:
    consulapi:
      enabled: false
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: false
    manageNonStandardCRDs: true

To support TCP route CRDs use Consul API gateway by setting global.crds.consulapi.enabled: true.

values.yaml

global:
  installK8sNetworkingCRDs: false
  crds:
    consulapi:
      enabled: false
connectInject:
  enabled: true
  apiGateway:
    manageExternalCRDs: false
    manageNonStandardCRDs: true

The Consul Helm chart deploys the Standard/Consul API gateway using the configuration specified in the values.yaml file.

Refer to Helm Chart Configuration - connectInject.apiGateway for information about the Helm chart configuration options.

API gateway reconciliation

Consul only reconciles API gateway resources that meet specific requirements. For Consul to manage an API gateway, the gateway must:

Include the required labels in the gateway metadata

labels:
  component: api-gateway

Specify the correct controller name that matches the gateway type.

Consul validates both criteria before reconciling the gateway object.

Standard API gateway

For the standard Kubernetes Gateway API that uses API Group gateway.networking.k8s.io/v1, the gateway must:

Set the gatewayClassName to reference a GatewayClass with controller name consul.hashicorp.com/gateway-controller
Include the label component: api-gateway in the metadata

Example configuration:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: api-gateway # can be modified 
  namespace: consul # can be modified
  labels:
    component: api-gateway # Do not modify this

Consul API gateway

For the Consul Gateway resource that uses API Group consul.hashicorp.com/v1beta1, the gateway must:

Set the gatewayClassName to reference a GatewayClass with controller name consul.hashicorp.com/gateway-controller-consul
Include the label component: api-gateway-consul in the metadata

Example configuration:

apiVersion: consul.hashicorp.com/v1beta1
kind: Gateway
metadata:
  name: api-gateway-consul # can be modified
  namespace: consul # can be modified
  labels:
    component: api-gateway-consul # Do not modify this

Use multiple gateway classes

Every API gateway sets the required spec.gatewayClassName field to select a gateway class. The gateway class points to a GatewayClassConfig that defines deployment settings such as the number of gateway instances, CPU and memory resources, and the Kubernetes service type. Gateways that use the same gateway class share the same GatewayClassConfig.

When you install Consul with Helm, Consul creates one default gateway class and one default GatewayClassConfig for each API gateway implementation that you enable:

API gateway implementation	Default gateway class	Default `GatewayClassConfig`
Standard API gateway	`GatewayClass` named `consul`	`consul-api-gateway`
Consul API gateway	`CustomGatewayClass` named `consul-custom-class`	`consul-api-gateway-custom`

The Consul API gateway defaults are created when you set global.crds.consulapi.enabled to true. The Consul API gateway requires consul-k8s 1.8.14 or later.

To change the number of instances, resources, or other deployment settings for an individual gateway without affecting other gateways, create an additional gateway class and GatewayClassConfig, and then set the gateway's spec.gatewayClassName field to the new gateway class.

Gateway class relationships

The following resources determine how a gateway is deployed:

Resource	Description
`Gateway`	Declares the gateway and its listeners. The `spec.gatewayClassName` field references a gateway class.
`GatewayClass` or `CustomGatewayClass`	Selects the controller that manages the gateway with the `spec.controllerName` field. The `spec.parametersRef` field references a `GatewayClassConfig`.
`GatewayClassConfig`	Defines the deployment settings for gateways that use the gateway class. This cluster-scoped resource is the same for both API gateway implementations.

The gateway class resource type and the controller name must match the API gateway implementation:

API gateway implementation	Gateway class resource	`controllerName`
Standard API gateway	`GatewayClass` in `gateway.networking.k8s.io`	`consul.hashicorp.com/gateway-controller`
Consul API gateway	`CustomGatewayClass` in `consul.hashicorp.com`	`consul.hashicorp.com/gateway-controller-consul`

Warning: Each controller reconciles only its own gateway class resource type. A GatewayClass that specifies the consul.hashicorp.com/gateway-controller-consul controller is never accepted, and Consul does not deploy gateways that reference it.

Create a GatewayClassConfig

Create a GatewayClassConfig that defines the settings for your gateway. The following example runs two instances of each gateway that uses it and sets the CPU and memory resources.

gatewayclassconfig-app1.yaml

apiVersion: consul.hashicorp.com/v1alpha1
kind: GatewayClassConfig
metadata:
  name: consul-api-gateway-app1
spec:
  deployment:
    defaultInstances: 2
    minInstances: 1
    maxInstances: 4
    resources:
      requests:
        cpu: 200m
        memory: 200Mi
      limits:
        cpu: 200m
        memory: 200Mi
  serviceType: LoadBalancer

The deployment block configures the following settings:

defaultInstances: Specifies the number of instances that Consul deploys when it creates the gateway.
minInstances and maxInstances: Specify the lower and upper limits for the number of instances. Consul raises or lowers the number of instances to stay within these limits.
resources: Specifies the CPU and memory requests and limits for each gateway instance.

Create a gateway class

Create a gateway class that references the GatewayClassConfig in spec.parametersRef.

gatewayclass-app1.yaml

apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: consul-app1
spec:
  controllerName: consul.hashicorp.com/gateway-controller
  parametersRef:
    group: consul.hashicorp.com
    kind: GatewayClassConfig
    name: consul-api-gateway-app1

customgatewayclass-app1.yaml

apiVersion: consul.hashicorp.com/v1beta1
kind: CustomGatewayClass
metadata:
  name: consul-app1
spec:
  controllerName: consul.hashicorp.com/gateway-controller-consul
  parametersRef:
    group: consul.hashicorp.com
    kind: GatewayClassConfig
    name: consul-api-gateway-app1

Apply the GatewayClassConfig and the gateway class to your cluster.

$ kubectl apply -f gatewayclassconfig-app1.yaml
$ kubectl apply -f gatewayclass-app1.yaml

For the Consul API gateway, apply customgatewayclass-app1.yaml instead of gatewayclass-app1.yaml.

Create a gateway that uses the gateway class

Set spec.gatewayClassName to the name of the gateway class. Gateways that set spec.gatewayClassName to a different gateway class, such as the default class, are not affected.

gateway-app1.yaml

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: api-gateway-app1
  namespace: app1
  labels:
    component: api-gateway
spec:
  gatewayClassName: consul-app1
  listeners:
    - protocol: HTTP
      port: 80
      name: http
      allowedRoutes:
        namespaces:
          from: All
    - protocol: TCP
      port: 8090
      name: tcp-8090
      allowedRoutes:
        namespaces:
          from: All

gateway-app1.yaml

apiVersion: consul.hashicorp.com/v1beta1
kind: Gateway
metadata:
  name: api-gateway-app1
  namespace: app1
  labels:
    component: api-gateway-consul
spec:
  gatewayClassName: consul-app1
  listeners:
    - protocol: HTTP
      port: 80
      name: http
      allowedRoutes:
        namespaces:
          from: All

Create the app1 namespace if it does not already exist, and then apply the gateway.

$ kubectl create namespace app1
$ kubectl apply -f gateway-app1.yaml

Verify the gateway class

Verify that Consul accepted the gateway class. The ACCEPTED column must show True.

$ kubectl get gatewayclass consul-app1

Verify that the gateway deployment uses the instance count and resources from the GatewayClassConfig.

$ kubectl get deployment api-gateway-app1 --namespace app1 --output jsonpath='{.spec.replicas}{"\n"}{.spec.template.spec.containers[0].resources}{"\n"}'

Repeat these steps to create additional gateway classes and GatewayClassConfig resources for other gateways.