Observe your service mesh

This topic provides an overview of Consul's observability features. Consul can be configured with Prometheus to provide visualizations about agent operations and service traffic between proxies in your service mesh.

Introduction

Consul supports observability functions that enable you to understand the state of your service mesh while it is running. By monitoring agent, client, dataplane, gateway, mesh proxy, and sidecar proxy requests and their L7 traffic, you can diagnose network issues and effectively mitigate communication failures in your service mesh. Consul can be configured to expose three kinds of telemetry features to help you observe your service mesh:

• Access logs
• Distributed traces
• Service mesh telemetry metrics

Access logs

Consul can emit access logs to record application connections and requests that pass through proxies in a service mesh, including sidecar proxies and gateways. When you enable access logs in your proxy defaults, you can use it to diagnose and troubleshoot network issues, detect threats to your network, and audit traffic for security compliance.

You can also configure the OpenTelemetry Envoy extension to capture and stream access logs. Refer to access logs for more information on enabling access logs in Consul.

Distributed tracing

Distributed tracing tracks and correlates requests between microservices, including proxies and gateways, in your service mesh. By implementing a tracing library such as Jaeger or Zipkin, you can trace the spans a request follows to investigate failures between service calls.

Refer to distributed tracing for more information.

Service mesh telemetry metrics

Consul agents, dataplanes, gateways, and sidecar proxies emit metrics that you can access and visualize using tools such as Prometheus and Grafana. Consul's UI can display visualizations of service mesh topologies and L7 metrics when they are enabled and configured in Consul. HCP Consul Central enables additional visualizations of cluster insights using dashboards and configurations managed by HashiCorp.

Refer to service mesh telemetry on virtual machines (VMs) or service mesh telemetry on Kubernetes for more information.

Guidance

To get started with the features described on this page, refer to the tutorials in the Observability tutorial collection:

• Monitor application health and performance with Consul proxy metrics
• Debug service mesh events and errors with Consul proxy access logs
• Monitor Consul server health and performance with metrics and logs
• Visualize Service Mesh Communication in the Consul UI
• Observe Consul service mesh traffic with Prometheus
• Layer 7 observability with Prometheus, Grafana, and Kubernetes
• HashiConf 2023 Learn Lab: Deep observability insights at any scale with Consul

Runtime specific usage documentation

For runtime-specific guidance, refer to the following pages:

• Observe service mesh telemetry on virtual machines (VMs)
• Observe service mesh telemetry on Kubernetes

Reference documentation

For reference material related to Consul's observability functions, refer to the following pages:

• Telemetry parameters in Consul agent configuration reference
• global.metrics parameters in Kubernetes Helm chart reference
• Proxy defaults configuration reference
• Envoy proxy configuration reference
• OpenTelemetry Access Logging extension configuration reference

Constraints, limitations, and troubleshooting

If you experience errors when configuring Consul's observability features, refer to the following list of technical constraints.

• Consul's observability features are not enabled by default on Community and Enterprise deployments. A HashiCorp-managed observability service is available through HCP Consul Central.
• Consul deploys Envoy proxies by default. Tracing spans that are incompatible with Envoy proxies do not function with Consul.