Traefik’s buffering middleware can store incoming requests and outgoing responses in memory or on disk, allowing services to handle traffic spikes gracefully or to retry failed requests.
Let’s see this in action. Imagine you have a critical API service that sometimes gets overwhelmed. You can use the buffering middleware to absorb these spikes and prevent your API from returning 5xx errors.
http:
routers:
my-api-router:
rule: "Host(`myapi.example.com`)"
service: "my-api-service"
middlewares:
- "buffer-middleware"
middlewares:
buffer-middleware:
buffering:
# Stores up to 100 requests in memory
maxRequests: 100
# If maxRequests is reached, store up to 10MB of requests on disk
maxResponseBytes: "10MB"
# If disk is full, reject new requests
whenEmpty: "error"
# Buffers responses for up to 1 minute
maxAge: "1m"
# Retry failed requests up to 3 times
retry:
attempts: 3
initialInterval: "1s"
# Exponential backoff with a maximum of 10 seconds
maxInterval: "10s"
services:
my-api-service:
loadBalancer:
servers:
- url: "http://192.168.1.100:8080"
In this setup, when myapi.example.com receives a request, Traefik first checks if the buffer-middleware is configured. If it is, Traefik will attempt to buffer the request. It will try to hold up to 100 requests in memory. If that fills up, it will spill over to disk, buffering up to 10MB of request data. If both memory and disk are full, Traefik will return an error (whenEmpty: "error").
Crucially, the buffering middleware isn’t just about holding requests; it’s also about managing responses. The maxAge: "1m" setting means that Traefik will hold onto a buffered response for up to one minute. If your backend service becomes temporarily unavailable, Traefik can serve this cached response, preventing a cascade of errors to your users.
The retry block is where the real resilience comes in. If a request to your backend service fails (e.g., a connection error or a 503 Service Unavailable), Traefik will automatically retry the request. It will try attempts: 3 times, starting with an initialInterval: "1s" between retries. If those retries also fail, the interval between subsequent retries will increase exponentially, up to a maxInterval: "10s". This prevents hammering a struggling service while giving it time to recover.
The buffering middleware’s core purpose is to decouple the request rate from the service’s processing capacity and to provide a buffer against transient backend failures. It acts as a shock absorber for your services. The maxRequests and maxResponseBytes settings define the capacity of the in-memory and on-disk buffers, respectively. maxAge dictates how long a buffered response is considered fresh, while retry configurations determine the strategy for re-attempting requests to unhealthy services.
What often gets overlooked is how Traefik manages the actual data for buffering. When using disk buffering (maxResponseBytes is set and memory is exhausted), Traefik creates temporary files within its data directory. These files are managed by the buffering middleware and are cleaned up once the buffered request or response is no longer needed (either served, retried successfully, or expired). If the disk becomes full, the whenEmpty: "error" setting ensures that Traefik stops accepting new requests rather than crashing or exhibiting unpredictable behavior, providing a clear failure mode.
The next step in building resilient services with Traefik is exploring its circuit breaker middleware, which can prevent requests from even reaching a service that is persistently failing.