SQS batching is a powerful optimization that lets you grab and ditch multiple messages in a single API call, drastically cutting down on latency and cost.
Here’s how it looks in the wild. Imagine you’ve got a Go application pulling messages from an SQS queue. Instead of looping and calling ReceiveMessage 10 times, you call it once with MaxNumberOfMessages=10.
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/sqs"
"github.com/aws/aws-sdk-go-v2/service/sqs/types"
)
func main() {
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// handle error
}
client := sqs.NewFromConfig(cfg)
queueURL := "https://sqs.us-east-1.amazonaws.com/123456789012/my-processing-queue"
maxMessages := int32(10)
receiveParams := &sqs.ReceiveMessageInput{
QueueUrl: &queueURL,
MaxNumberOfMessages: maxMessages,
WaitTimeSeconds: 10, // Long polling
}
resp, err := client.ReceiveMessage(context.TODO(), receiveParams)
if err != nil {
// handle error
}
if len(resp.Messages) > 0 {
fmt.Printf("Received %d messages\n", len(resp.Messages))
// Process each message
for _, msg := range resp.Messages {
fmt.Printf("Processing message: %s\n", *msg.MessageId)
// Your message processing logic here...
}
// Batch delete messages
deleteEntries := make([]types.DeleteMessageBatchRequestEntry, len(resp.Messages))
for i, msg := range resp.Messages {
deleteEntries[i] = types.DeleteMessageBatchRequestEntry{
Id: msg.MessageId,
ReceiptHandle: msg.ReceiptHandle,
}
}
deleteParams := &sqs.DeleteMessageBatchInput{
QueueUrl: &queueURL,
Entries: deleteEntries,
}
_, err := client.DeleteMessageBatch(context.TODO(), deleteParams)
if err != nil {
// handle error, potentially retry individual deletes
}
fmt.Println("Batch deleted messages.")
}
}
The core problem SQS batching solves is the overhead of individual API calls. Each ReceiveMessage and DeleteMessage call involves network round trips, authentication, and SQS processing. When you’re dealing with high volumes, this adds up to significant latency and cost. By grouping messages, you reduce the number of these expensive operations.
Internally, SQS treats each batch operation as a single transaction against the queue. When you call ReceiveMessage with MaxNumberOfMessages=10, SQS finds up to 10 available messages, locks them with a visibility timeout, and returns them in one go. For DeleteMessageBatch, SQS attempts to delete all the specified messages. If some fail, it returns a list of failed entries, allowing you to retry them individually.
The key levers you control are:
MaxNumberOfMessages: This parameter inReceiveMessageInputdictates the maximum number of messages you want to retrieve in a single call. The valid range is 1 to 10. Setting this to 10 is generally optimal for throughput.WaitTimeSeconds: InReceiveMessageInput, this enables long polling. Instead of returning immediately if no messages are available, SQS will hold the connection open for up to this duration, waiting for messages. This dramatically reduces the number of emptyReceiveMessagecalls, saving cost and reducing latency. A value between 10 and 20 seconds is common.Entries: InDeleteMessageBatchInput, this is a slice ofDeleteMessageBatchRequestEntrystructs. Each entry specifies theId(message ID) andReceiptHandlefor a message to be deleted.
A common pitfall when using DeleteMessageBatch is assuming all messages will be deleted successfully. The DeleteMessageBatchOutput contains an Failed field, which is a slice of BatchResultErrorEntry if any deletions failed. You must inspect this field and implement retry logic for those specific messages. A common pattern is to extract the MessageId from the failed entries and either re-queue them or attempt to delete them again after a short delay.
The ReceiptHandle is ephemeral and tied to the visibility timeout. If a message’s visibility timeout expires before you delete it, its ReceiptHandle becomes invalid. This is why it’s critical to process messages quickly and delete them within their visibility window, or to extend the visibility timeout if processing takes longer.
The next step after mastering batch receive and delete is understanding SQS dead-letter queues (DLQs) for handling messages that repeatedly fail processing.