Multiport services

This page describes how to configure and deploy services that utilize multiple ports in a Consul service mesh.

In a Consul service mesh, you may configure services to use multiple ports. This configuration lets you run different types of traffic through the same service while still benefiting from Consul's service discovery and traffic management features. For example, a web application might have one port for HTTP traffic and another for gRPC traffic. With Consul service mesh, you may configure your services to use multiple ports while still leveraging the benefits of service mesh features such as secure communication, traffic routing, and observability.

Configuring multiport services

Configure a multiport service in Consul service mesh by specifying the ports in the service registration. Each port should have a unique name, port number, and whether it is the default port.

This example demonstrates a web application with both HTTP and gRPC ports defined in HCL and JSON formats.

service {
  name = "api"
  id   = "api-1"
  ports = [
    {
      name = "http"
      port = 8080
      default = true
    },
    {
      name = "grpc"
      port = 9090
    }
  ]
}

{
  "name": "api",
  "id": "api-1",
  "ports": [
    {
      "name": "http",
      "port": 8080,
      "default": true
    },
    {
      "name": "grpc",
      "port": 9090
    }
  ]
}

Addressing multiport services

Non-transparent proxy

In non-transparent proxy mode, downstream services explicitly configure upstreams by defining these parameters:

• A service destination_name
• The local_bind_port
• The destination_port, which is specific to the multiport use case

This configuration lets the downstream service specify which port on the upstream service it wants to connect to. The proxy handles routing the traffic accordingly. Refer to the upstream configuration reference for details.

This example configuration is for a non-transparent proxy service with multiport upstream services and illustrates the following:

1. The web service connecting to localhost:9191 is rerouted to the single port db:9191 service.
2. The web service connecting to localhost:9192 is rerouted to the multiport destination of api:8080.
3. The web service connecting to localhost:9193 is rerouted to the multiport destination of api:9090.

service {
  name = "web"
  port = 8080
  connect {
    sidecar_service {
      proxy {
        upstreams = [
          {
            destination_name = "db"
            local_bind_port = 9191
          },
          {
            destination_name = "api"
            destination_port = "http"
            local_bind_port = 9192
          },
          {
            destination_name = "api"
            destination_port = "grpc"
            local_bind_port = 9193
          }
        ]
        config {
          handshake_timeout_ms = 1000
        }
      }
    }
  }
}

{
  "name": "web",
  "port": 8080,
  "connect": {
    "sidecar_service": {
      "proxy": {
        "upstreams": [
          {
            "destination_name": "db",
            "local_bind_port": 9191
          },
          {
            "destination_name": "api",
            "destination_port": "http",
            "local_bind_port": 9192
          },
          {
            "destination_name": "api",
            "destination_port": "grpc",
            "local_bind_port": 9193
          }
        ],
        "config": {
          "handshake_timeout_ms": 1000
        }
      }
    }
  }
}

Transparent proxy

In transparent proxy mode, Consul uses virtual DNS to resolve the correct port for an upstream service without needing to specify the port in the configuration.

Consul generates the virtual DNS entries based on the service name and port name, following the naming convention <port-name>.<service-name>.virtual.consul.

Consul's internal DNS returns a Virtual IP (VIP) unique to that specific service-port combination. For example, if you have a service named api with two ports named http and grpc, the virtual DNS entries would be http.api.virtual.consul and grpc.api.virtual.consul.

Multiport services across cluster peers

You can route traffic to a specific named port on a multiport service that you export to a cluster peer. Peered multiport routing supports both non-transparent and transparent proxy modes.

Peered multiport routing is an experimental feature. The peering-multiport-upstreams feature gate controls peered multiport routing. The gate is disabled by default. Multiport services within a single cluster, and peered services that use only their default port, do not depend on this feature gate.

Requirements

Before you enable peered multiport routing, verify that your environment meets the following requirements:

• Every Consul server in both peered clusters runs Consul Enterprise v2.1.0 or later. The feature gate stays disabled until all servers in the cluster meet this minimum version.
• The mesh gateways, client agents, and sidecar proxies that serve the exporting and consuming services also run Consul Enterprise v2.1.0 or later. Refer to Upgrade considerations.
• The clusters are peered and each cluster has a mesh gateway. If the exporting service is in an admin partition, register the mesh gateway in the same partition.
• The exporting service is registered as a multiport service and has a sidecar proxy.

Enable the feature gate

Enable the peering-multiport-upstreams feature gate in both the exporting cluster and the consuming cluster. Run the following command on each cluster:

$ consul operator feature set peering-multiport-upstreams enabled

To verify the state of the feature gate, run the consul operator feature get command. The output shows both the state you requested and the state that is in effect.

$ consul operator feature get peering-multiport-upstreams

You can also enable the feature gate when you create a datacenter, and you can manage it with the HTTP API. Refer to Feature gates for more information, including the required ACL permissions.

Export the service and create an intention

Export the multiport service from the exporting cluster with an exported-services configuration entry. You export the service as a whole. You do not specify individual ports.

The following configuration entry exports the api service to the dc1-peer peer:

Kind = "exported-services"
Name = "default"

Services = [
  {
    Name = "api"
    Consumers = [
      {
        Peer = "dc1-peer"
      }
    ]
  }
]

{
  "Kind": "exported-services",
  "Name": "default",
  "Services": [
    {
      "Name": "api",
      "Consumers": [
        {
          "Peer": "dc1-peer"
        }
      ]
    }
  ]
}

When you export from an admin partition, the Name field must match the name of the partition.

Then, create a service intention in the exporting cluster that allows the downstream service in the peer to connect.

The following intention allows the web service in the dc1-peer peer to connect to the api service:

Kind = "service-intentions"
Name = "api"

Sources = [
  {
    Name   = "web"
    Peer   = "dc1-peer"
    Action = "allow"
  }
]

{
  "Kind": "service-intentions",
  "Name": "api",
  "Sources": [
    {
      "Name": "web",
      "Peer": "dc1-peer",
      "Action": "allow"
    }
  ]
}

Intentions apply to the service and not to individual ports. An intention that allows a source to connect to a multiport service allows that source to connect to every named port on the service.

Address peered multiport services

You can address the ports of a peered multiport service in two ways. The method you use depends on whether the downstream service runs in transparent proxy mode:

• Peered services in non-transparent proxy mode. Configure an upstream for each port and connect to the local bind port.
• Peered services in transparent proxy mode. Connect to the virtual DNS name of each port.

Peered services in non-transparent proxy mode

In non-transparent proxy mode, configure an upstream for each port. Specify the peer with the destination_peer parameter and the port with the destination_port parameter. If you omit destination_port, the upstream routes traffic to the service's default port.

This example configuration illustrates the following:

1. Consul routes traffic that the web service sends to localhost:9192 to the http port of the api service in the dc2-peer peer.
2. Consul routes traffic that the web service sends to localhost:9193 to the grpc port of the api service in the dc2-peer peer.
3. Consul routes traffic that the web service sends to localhost:9194 to the default port of the api service in the dc2-peer peer.

service {
  name = "web"
  port = 8080
  connect {
    sidecar_service {
      proxy {
        upstreams = [
          {
            destination_name = "api"
            destination_peer = "dc2-peer"
            destination_port = "http"
            local_bind_port  = 9192
          },
          {
            destination_name = "api"
            destination_peer = "dc2-peer"
            destination_port = "grpc"
            local_bind_port  = 9193
          },
          {
            destination_name = "api"
            destination_peer = "dc2-peer"
            local_bind_port  = 9194
          }
        ]
      }
    }
  }
}

{
  "name": "web",
  "port": 8080,
  "connect": {
    "sidecar_service": {
      "proxy": {
        "upstreams": [
          {
            "destination_name": "api",
            "destination_peer": "dc2-peer",
            "destination_port": "http",
            "local_bind_port": 9192
          },
          {
            "destination_name": "api",
            "destination_peer": "dc2-peer",
            "destination_port": "grpc",
            "local_bind_port": 9193
          },
          {
            "destination_name": "api",
            "destination_peer": "dc2-peer",
            "local_bind_port": 9194
          }
        ]
      }
    }
  }
}

Peered services in transparent proxy mode

In transparent proxy mode, applications address each port of a peered service with a virtual DNS name that includes the peer name. Consul assigns each named port its own virtual IP address. Virtual DNS names for peered services use the following format:

[<port-name>.]<service>.virtual.<peer>.peer.<domain>

For example, if the dc2-peer peer exports the api service, the virtual DNS entries are as follows:

If the service is in a non-default admin partition or namespace of the consuming cluster, add the partition and namespace labels. For example, the virtual DNS name for the grpc port in namespace ns1 and partition ap1 is grpc.api.virtual.ns1.ns.ap1.ap.dc2-peer.peer.consul. For more information about the virtual DNS format, refer to Service virtual IP lookups.

Consul assigns the virtual IP addresses for named ports after it imports the exported service. Peered resources can take several minutes to synchronize, so a lookup that you make immediately after you export the service may not resolve. Retry the lookup before you troubleshoot.

A virtual DNS name for a port that does not exist on a multiport service returns an NXDOMAIN response.

Considerations

Consider the following limitations when you use multiport services across cluster peers:

• Service-level discovery chains apply to the default port only. If you configure a service resolver, service router, service splitter, failover policy, or sameness group for the exported service, the configuration applies to the default port. For TCP services, requests to named ports still reach the correct port.
• HTTP services can route named ports to the default port. When the exported service's protocol is http and the service has a discovery chain that is not the default, requests to a named port can reach the default port and return a successful response instead of an error. Consul logs a warning that names the service, the affected ports, and the protocol. If you see this warning, remove the discovery chain configuration or set the service protocol to tcp.
• All ports share one protocol. You set the protocol for the whole service in its service-defaults configuration entry. You cannot set a protocol for each port.
• Intentions do not apply per port. Refer to Export the service and create an intention.
• Removing a named port requires updating consumers. If you remove a port from a service registration, an upstream in the consuming cluster that still names the removed port can continue to reach the default port instead of failing. Update or remove the upstreams and DNS references for the port before you remove it.
• Port names cannot contain ., :, /, or whitespace. Consul rejects service registrations that use these characters in a port name.
• WAN-federated datacenters do not support multiport services. Use cluster peering to connect multiport services across datacenters.

Upgrade considerations

Upgrade every component in both peered clusters to Consul Enterprise v2.1.0 or later before you enable the feature gate. Complete the upgrade in the following order:

1. Upgrade the Consul servers.
2. Upgrade the client agents, mesh gateways, and sidecar proxies.
3. Enable the peering-multiport-upstreams feature gate in the exporting and consuming clusters.
4. Add or update the upstreams that name a port.

Enable the feature gate in both clusters. If you disable it in the consuming cluster, Consul removes that cluster's per-port Envoy listeners and Envoy clusters, and requests to named ports fail. If you disable it in the exporting cluster, Consul removes the mesh gateway's per-port routing.

Roll back

To stop routing to peered named ports, disable the feature gate in both clusters:

$ consul operator feature set peering-multiport-upstreams disabled

Consul removes the per-port resources, and requests to named ports fail instead of reaching another port. Peered traffic to the default port and services that use a single port keep working. When you enable the feature gate again, Consul restores the per-port resources. Expect a short period of failed requests while the proxies update.

Multiport functionality in Consul services

The following functionality has partial support for Consul Multiport:

• Mesh gateways
• Proxy and Service defaults
• Consul DNS

Note that multiport in WAN Federation is not supported.

Next steps

Multiport services in Consul service mesh let you run different types of traffic through the same service while still benefiting from Consul's service discovery and traffic management features. By configuring your services to use multiple ports, you can ensure that your applications can communicate effectively while still leveraging the benefits of a service mesh.

For more information about the topics covered in this guide, refer to the following resources:

• Consul service mesh documentation
• Consul service configuration reference
• Consul proxy configuration reference
• Consul DNS documentation