SQS Backpressure Pattern: Slow Down Producers (2026)

SQS backpressure isn’t about slowing down producers; it’s about allowing them to slow down gracefully when downstream consumers can’t keep up.

Here’s a typical SQS setup:

{
  "Queues": [
    {
      "Name": "my-processing-queue",
      "Attributes": {
        "VisibilityTimeout": "30",
        "MessageRetentionPeriod": "345600",
        "ApproximateNumberOfMessages": "5",
        "ApproximateNumberOfMessagesNotVisible": "2"
      }
    }
  ],
  "Configurations": {
    "Producer": {
      "MaxMessagesPerSecond": 100,
      "BatchSize": 10
    },
    "Consumer": {
      "MaxMessagesPerSecond": 50,
      "BatchSize": 10
    }
  }
}

Imagine my-processing-queue is getting hammered. Producers are blasting messages at 100/sec, but the consumers, due to complex processing or external dependencies, can only handle 50/sec. Without a backpressure mechanism, this leads to a growing queue. Messages sit in ApproximateNumberOfMessages longer and longer, increasing processing latency. Eventually, consumers might start failing, or producers might hit their own rate limits and error out, leading to a cascade of failures.

The "backpressure pattern" here is about detecting this overload and signaling the producers to throttle themselves. SQS itself doesn’t enforce backpressure; it provides the signals. The most common signal is the ApproximateNumberOfMessages metric. When this count climbs beyond a predefined threshold, it’s an indicator that consumers are falling behind.

How to implement it:

1. Monitor ApproximateNumberOfMessages: This is your primary indicator. Set up CloudWatch alarms on this metric. A common threshold might be when ApproximateNumberOfMessages exceeds a value that represents, say, 15 minutes of backlog at the consumer’s current processing rate. For our example, if consumers process 50 messages/sec, a backlog of 15 minutes is 50 messages/sec * 60 sec/min * 15 min = 45,000 messages. So, an alarm could trigger when ApproximateNumberOfMessages is greater than 45000.

2. Producer Throttling Logic: The producer application needs to actively check this metric (or react to the CloudWatch alarm).

   • Polling: The producer can periodically (e.g., every 30 seconds) query SQS for ApproximateNumberOfMessages or query CloudWatch for the alarm status.
   • Alarm Notification: A more reactive approach is to have the CloudWatch alarm trigger an SNS topic. The producer application (or a dedicated microservice) can subscribe to this SNS topic. When the alarm is triggered, the SNS message directly informs the producer to slow down.

3. Adjusting Producer Rate: Once the backpressure signal is received, the producer application must reduce its sending rate.

   • Rate Limiting: Implement logic within the producer to limit the number of messages sent per second. If the current rate is 100/sec and backpressure is detected, reduce it to, say, 25/sec.
   • Exponential Backoff: For more dynamic adjustment, use exponential backoff. If the queue is still growing at the reduced rate, further decrease the sending rate. For example, if 25/sec isn’t enough, try 10/sec, then 5/sec, and so on.
   • MaxNumberOfMessages in ReceiveMessage (for consumers): While this isn’t directly producer backpressure, consumers can also signal overload by reducing their MaxNumberOfMessages polled from SQS. This reduces the load on SQS itself and can indirectly help. However, the primary backpressure signal is the queue depth.

4. Resuming Production: When ApproximateNumberOfMessages drops below a lower threshold (e.g., 50% of the trigger threshold, or 22,500 messages in our example), the producer can gradually increase its sending rate back towards the original target. This prevents aggressive cycling between full throttle and no throttle.

Why it works:

By monitoring the queue depth and having producers react to it, you prevent the queue from growing indefinitely. The producers effectively "listen" to the consumers’ ability to process by observing the SQS queue. When the queue grows, it’s a signal that consumers are overwhelmed. Producers then reduce their sending rate, giving consumers time to catch up. When the queue shrinks, it means consumers are handling the load, and producers can ramp back up.

The critical part is that the producer application implements this logic. SQS just provides the metrics and the queue itself as the buffer. The VisibilityTimeout is crucial here too; if it’s too short, messages might be returned to the queue before they’re processed, creating a false sense of backlog. If it’s too long, a genuinely failed consumer might hold onto messages for an extended period, also masking the true backlog.

The next problem you’ll hit is managing the transition between throttled and unthrottled states, specifically preventing oscillations where the queue depth rapidly fluctuates, causing the producer to constantly speed up and slow down.