SQS message filtering lets you define rules to select which messages a consumer should receive, rather than having the consumer process all messages and then discard unwanted ones.
Imagine you have a single SQS queue receiving events from multiple sources, like user sign-ups, order placements, and product updates. Without filtering, your order processing service would get all these messages and have to check each one to see if it’s an order-related event. With filtering, you can tell SQS, "Only send me messages that look like orders."
Here’s a simple setup. Let’s say we have a queue named my-event-queue. We want a consumer service to only process messages that have a messageType attribute equal to ORDER_PLACED.
First, create a RedrivePolicy for your queue if you don’t have one. This is good practice for dead-letter queues.
{
"maxReceiveCount": 10,
"deadLetterTargetArn": "arn:aws:sqs:us-east-1:123456789012:my-event-queue-dlq"
}
Now, let’s define the filter policy. This is attached to the SQS queue. You can do this via the AWS CLI or the AWS Management Console.
Using the AWS CLI:
aws sqs set-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-event-queue \
--attributes '{
"RedrivePolicy": "{\"maxReceiveCount\": 10, \"deadLetterTargetArn\": \"arn:aws:sqs:us-east-1:123456789012:my-event-queue-dlq\"}",
"FilterPolicy": "{\"messageType\": [\"ORDER_PLACED\"]}"
}'
In this FilterPolicy, messageType is the attribute name, and [\"ORDER_PLACED\"] is a list of values that will match. This means only messages with an attribute named messageType and a value exactly equal to ORDER_PLACED will be delivered to the consumer.
Let’s simulate sending some messages.
Message 1: Will be delivered
{
"MessageAttributes": {
"messageType": {
"DataType": "String",
"StringValue": "ORDER_PLACED"
},
"orderId": {
"DataType": "String",
"StringValue": "ORD12345"
}
},
"MessageBody": "{\"details\": \"Customer placed a new order.\"}"
}
Message 2: Will NOT be delivered
{
"MessageAttributes": {
"messageType": {
"DataType": "String",
"StringValue": "USER_SIGNED_UP"
},
"userId": {
"DataType": "String",
"StringValue": "USR67890"
}
},
"MessageBody": "{\"details\": \"New user registered.\"}"
}
Message 3: Will NOT be delivered
{
"MessageAttributes": {
"eventType": {
"DataType": "String",
"StringValue": "PRODUCT_UPDATED"
},
"productId": {
"DataType": "String",
"StringValue": "PROD001"
}
},
"MessageBody": "{\"details\": \"Product pricing changed.\"}"
}
When your consumer calls ReceiveMessage on my-event-queue, it will only receive Message 1. Messages 2 and 3 will remain in the queue, invisible to this specific consumer.
The core problem SQS message filtering solves is reducing the load on your consumers and the network traffic between SQS and your application. Instead of a general-purpose consumer pulling everything and then doing its own routing and discarding, SQS itself acts as a smart router. This is especially powerful when you have many different types of messages flowing through a single queue and multiple consumers, each interested in a specific subset.
The filtering works by evaluating the message’s attributes against the FilterPolicy you define. SQS compares the attribute names and values in the message against the rules in the policy. If the message attributes match any of the filter policies associated with the queue, the message is considered a match and delivered. If multiple filter policies are defined, a message only needs to match one of them to be delivered.
Here’s where it gets interesting: you can define multiple filter policies on a single queue. For example, you might have another consumer interested in all order-related events, not just ORDER_PLACED. You could add a second filter policy for that consumer, targeting messages with orderId present.
"FilterPolicy": "{\"messageType\": [\"ORDER_PLACED\"]}"
If you wanted to add a policy for another consumer that receives any message with an orderId, you’d add that as a separate policy. SQS treats these as OR conditions. So, a message would be delivered if it matches messageType equals ORDER_PLACED OR if it has an orderId attribute.
"FilterPolicy": "{\"messageType\": [\"ORDER_PLACED\"], \"orderId\": [\"*\"]}"
The [\"*\"] syntax is a wildcard that matches any non-empty string value for the orderId attribute.
You can also use Exists or NotExists conditions. For example, to receive messages that don’t have a userId attribute:
"FilterPolicy": "{\"userId\": [\"!=\", \"null\"]}"
This is a bit counterintuitive; it’s a list of conditions where the first element is the operator (!= for not equals) and the second is the value to compare against (null is a special keyword here). This filters for messages where the userId attribute is not present or is explicitly null.
The most surprising thing is that SQS doesn’t actually remove messages from the queue that don’t match a filter policy if that policy is associated with a specific subscription (like in SNS to SQS). However, when you apply a FilterPolicy directly to an SQS queue, messages that don’t match are simply never delivered to the ReceiveMessage call. They remain in the queue until they are either matched by a future ReceiveMessage call (if the filter policy changes) or eventually expire. This means the queue can grow with messages that a particular consumer might never see if the filter policy is too strict for its needs.
The next thing you’ll likely run into is how to handle messages that do match your filter but still fail processing by your consumer. This is where the RedrivePolicy and dead-letter queues become crucial.