OpenTelemetry tracing in Traefik doesn’t actually add new traces; it exports the traces Traefik already generates to an OpenTelemetry collector.
Let’s see Traefik in action with OpenTelemetry tracing enabled. Imagine a client request hitting Traefik, which then forwards it to a backend service.
# traefik.yml
api:
insecure: true
dashboard: true
entryPoints:
web:
address: ":80"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
log:
level: DEBUG
tracing:
openTelemetry:
endpoint: "http://otel-collector:4318" # Or your collector's address
tls:
insecureSkipVerify: true
With this configuration, when a request comes in, Traefik automatically creates a trace for the request’s lifecycle within Traefik itself. This includes the time spent in routing, middleware processing, and forwarding to the backend. If your backend services are also instrumented, their spans can be linked to Traefik’s initial span, building a complete end-to-end trace.
Here’s how it works internally: Traefik uses an internal tracing mechanism. When you enable OpenTelemetry tracing, you’re essentially telling Traefik to take these internally generated spans and send them over the wire to a specified endpoint. This endpoint is typically an OpenTelemetry Collector, which then processes and forwards these traces to a backend like Jaeger, Zipkin, or a cloud-native observability platform.
The primary levers you control are the endpoint where Traefik sends traces and TLS configurations for secure transport. You can also control the sampling rate if your OpenTelemetry collector supports it, though Traefik itself doesn’t directly expose sampling configuration for its own traces.
The most surprising thing about this setup is that Traefik’s own routing and middleware processing are automatically traced without any application code changes. You get visibility into the "edge" of your system for free.
When you look at your tracing backend (e.g., Jaeger UI), you’ll see a trace for each request that Traefik processes. This trace will have a root span representing the incoming request to Traefik, followed by child spans for each step Traefik takes, like applying a RateLimit middleware or forwarding to a specific backend service. If your backend service also emits spans with the same trace_id, those spans will appear as further children, creating a unified view of the request’s journey.
The configuration for tls.insecureSkipVerify: true is convenient for testing but should be avoided in production. For production, you’ll want to configure proper TLS certificates for the connection between Traefik and your OpenTelemetry collector, ensuring that traces are sent securely and that the collector can verify Traefik’s identity.
The next step in distributed tracing is often understanding how to propagate trace context between Traefik and your backend services.