KEYS in Valkey is a dangerous command because it can block your entire Redis instance, and therefore your application, for an indeterminate amount of time.
Let’s see it in action. Imagine you have a Valkey instance with a few million keys.
# On a test instance with some data
redis-cli --scan --pattern '*' | wc -l
# This will start listing keys and then count them.
# On a large dataset, this can take seconds, minutes, or even hours.
# Now, let's simulate a real problem.
# In one terminal:
redis-cli
127.0.0.1:6379> KEYS *
# This command will start scanning and won't return until it's done.
# Your application connected to this instance will now hang.
# In another terminal:
redis-cli
127.0.0.1:6379> SET mykey myvalue
# This command will time out or hang because the Redis server is busy with KEYS.
# PING
# PING
# ... your application's health checks will fail.
The problem is that KEYS performs a blocking, full-table scan of your entire key space. It iterates through every single key, collects them into a list, and then returns that list to the client. On an instance with millions of keys, this operation consumes significant CPU and memory on the server, and its duration is directly proportional to the number of keys. During this time, the Redis server cannot process any other commands, including critical ones like SET, GET, DEL, or even PING. This effectively grinds your application to a halt.
The SCAN command, introduced in Redis 3.2, is the solution. It’s an iterative, non-blocking command that allows you to traverse your key space without freezing your server. SCAN works by maintaining a cursor on the server and returning a small batch of keys with each call. You then use the returned cursor to make the next call, gradually sweeping through your keys.
Here’s how you’d use SCAN to achieve the same goal as KEYS * but safely:
# Initialize cursor to 0
cursor=0
# Loop until cursor returns to 0
while [ "$cursor" != "0" ]; do
# Execute SCAN command, passing the current cursor
# The --raw option makes redis-cli output raw strings,
# and the 'replace' argument to xargs splits on whitespace
# to correctly parse the cursor and keys.
read cursor keys < <(redis-cli --raw SCAN $cursor COUNT 1000)
# Process keys here (e.g., count them, filter them)
echo "Processing ${#keys[@]} keys..."
done
echo "Scan complete."
The COUNT 1000 parameter is a hint to the server about how many keys to return per iteration. The server is not obligated to return exactly that many keys; it will return a number of keys that is at least the cursor’s value plus the count, but it might return more or fewer depending on internal implementation details and the distribution of keys. The crucial part is that SCAN returns a cursor value for the next iteration, allowing the process to continue.
The primary reason SCAN is non-blocking is its cursor-based, incremental approach. Instead of performing one massive operation, it breaks the traversal into many small, quick operations. Each SCAN call returns a subset of keys and a new cursor. The server is free to process other commands between these SCAN calls. This means that even during a full traversal of your key space, your application remains responsive.
SCAN also offers a MATCH pattern, similar to KEYS, but it’s applied server-side and efficiently.
cursor=0
while [ "$cursor" != "0" ]; do
read cursor keys < <(redis-cli --raw SCAN $cursor MATCH "user:*" COUNT 1000)
echo "Processing ${#keys[@]} keys matching 'user:*'..."
# Process the keys returned
done
echo "Scan for 'user:*' complete."
The COUNT parameter in SCAN is often misunderstood. It’s not a guarantee of how many keys will be returned, but rather a server-side hint for the effort the server should put into finding keys for that iteration. A higher COUNT might result in fewer SCAN calls overall but could potentially lead to slightly longer individual SCAN operations if the server has to work harder to find enough matching keys. Conversely, a low COUNT might result in more SCAN calls, each returning fewer keys, but with less work per call. The optimal COUNT depends on your workload and instance characteristics, but 1000 is a common starting point.
The real power of SCAN lies in its ability to be used in production environments without fear of downtime. You can use it for tasks like:
- Key Deletion/Cleanup: Iteratively find and delete old or unused keys.
- Data Migration: Exporting or transforming data by iterating through keys.
- Monitoring: Checking for specific key patterns or estimating key counts.
- Rebalancing/Sharding: Identifying keys that need to be moved.
The final piece of the puzzle is understanding the MATCH parameter’s behavior. While it looks like KEYS’s pattern matching, the server’s implementation is different. SCAN with MATCH returns keys that match the pattern, but the iteration order and the number of keys returned are still subject to the cursor and COUNT parameters. It’s not a simple glob expansion; the server iterates and filters efficiently.
If you find yourself needing to delete a large number of keys matching a pattern, using SCAN in a loop with DEL is the production-safe way.
cursor=0
while [ "$cursor" != "0" ]; do
read cursor keys < <(redis-cli --raw SCAN $cursor MATCH "temp:*" COUNT 1000)
if [ ${#keys[@]} -gt 0 ]; then
echo "Deleting ${#keys[@]} keys matching 'temp:*'..."
redis-cli DEL $keys
fi
done
echo "Cleanup complete."
The next logical step after mastering SCAN is to explore how to use it with Lua scripts for even more atomic and efficient operations, especially when dealing with complex key relationships or needing to perform multiple actions within a single server-side execution.