SQS delay queues let you postpone the delivery of a message to a queue for a specified amount of time.
Here’s a quick look at how it works with a sample message and configuration:
import boto3
sqs = boto3.client('sqs')
queue_url = 'YOUR_QUEUE_URL' # Replace with your actual SQS queue URL
# Send a message with a 30-second delay
response = sqs.send_message(
QueueUrl=queue_url,
MessageBody='This message will be delivered after a delay.',
DelaySeconds=30
)
print(f"Message sent with ID: {response['MessageId']}")
When you send a message to an SQS queue, it’s immediately available for consumers to retrieve. However, by setting DelaySeconds on the sendMessage API call, you tell SQS to hold onto the message for that duration. After the delay expires, the message becomes visible and can be processed by your consumers just like any other message.
You can also configure a default delay for an entire SQS queue. This is set at the queue level and applies to all messages sent to that queue unless you explicitly override it with DelaySeconds in the sendMessage call.
The default delay for a queue can be set via the AWS Management Console:
- Navigate to the SQS service.
- Select your queue.
- Go to "Queue attributes."
- Edit the
DelaySecondsattribute.
Or using the AWS CLI:
aws sqs set-queue-attributes \
--queue-url YOUR_QUEUE_URL \
--attributes DelaySeconds=60
This command sets the default delay for YOUR_QUEUE_URL to 60 seconds. Any message sent without an explicit DelaySeconds will wait for 60 seconds before becoming visible.
The primary use case for delay queues is scheduling tasks. Imagine you have a system that needs to perform an action a few minutes after a specific event. Instead of having a worker constantly polling for "time to act," you can send a message to an SQS queue with a DelaySeconds value set to that future time. When the delay expires, the message is delivered, triggering your worker to perform the action. This is far more efficient than polling.
Another common pattern is to use delay queues for retry mechanisms with exponential backoff. If a consumer fails to process a message, it can be sent back to the queue with an increased DelaySeconds value for the next attempt. This prevents overwhelming a service with immediate retries after a failure.
While DelaySeconds is set per message, the queue-level DelaySeconds attribute acts as a default. If a message is sent with DelaySeconds=0 or without the DelaySeconds parameter, and the queue has a default DelaySeconds of, say, 10 seconds, that message will still wait for 10 seconds. If a message is sent with DelaySeconds=5 and the queue’s default delay is 10 seconds, the message will only wait for 5 seconds because the per-message value takes precedence. The actual delay for a message is the maximum of the per-message DelaySeconds and the queue’s default DelaySeconds.
The maximum delay you can set for a message or a queue is 15 minutes (900 seconds). If you need longer delays, you’ll need to implement a different strategy, perhaps by using a separate queue or a timestamp in the message body and a scheduled Lambda function to re-queue it.
When a message is in its delayed state, it’s not visible to consumers. It resides within SQS and will automatically become visible once the delay period has elapsed. You cannot "peek" at a delayed message or retrieve it until its delay has expired.
The next concept you’ll likely encounter is how to handle messages that cannot be processed after multiple retries, leading you to explore Dead-Letter Queues (DLQs).