Service Mesh Observability Overview

In order to take advantage of Connect's L7 observability features you will need to:

• Deploy sidecar proxies that are capable of emitting metrics with each of your services. We have first class support for Envoy.
• Define where your proxies should send metrics that they collect.
• Define the protocols for each of your services.
• Define the upstreams for each of your services.

If you are using Envoy as your sidecar proxy, you will need to enable gRPC on your client agents. To define the metrics destination and service protocol you may want to enable configuration entries and centralized service configuration.

Kubernetes

If you are using Kubernetes, the Helm chart can simplify much of the configuration needed to enable observability. See our Kubernetes observability docs for more information.

Metrics Destination

For Envoy the metrics destination can be configured in the proxy configuration entry's config section.

kind = "proxy-defaults"
name = "global"
config {
   "envoy_dogstatsd_url": "udp://127.0.0.1:9125"
}

Find other possible metrics syncs in the Connect Envoy documentation.

Service Protocol

You can specify the service protocol in the service-defaults configuration entry. You can override it in the service registration. By default, proxies only give you L4 metrics. This protocol allows proxies to handle requests at the right L7 protocol and emit richer L7 metrics. It also allows proxies to make per-request load balancing and routing decisions.

Service Upstreams

You can set the upstream for each service using the proxy's upstreams sidecar parameter, which can be defined in a service's sidecar registration.