Valkey, when used as a cache, can easily balloon in memory usage, turning your carefully planned infrastructure costs into a runaway train.

Here’s how a Valkey cluster handles memory, and how you can rein it in.

How Valkey Uses Memory

Valkey stores data in memory, but it’s not just raw data. Each key-value pair has overhead associated with it. This includes:

  • Key Name: The string representing your key.
  • Value: The actual data you’re storing.
  • Key Metadata: This is the crucial part for optimization. It includes:
    • Encoding: How the value is stored (e.g., int, raw, ziplist, listpack).
    • Data Structure: Whether it’s a string, list, set, hash, or sorted set.
    • Internal Structures: For complex data types like lists, sets, hashes, and sorted sets, Valkey uses internal data structures that have their own memory footprint.

The Memory Profile

You can get a detailed breakdown of Valkey’s memory usage with the INFO memory command.

redis-cli INFO memory

This output is your primary tool for understanding what’s consuming RAM. Look for sections like used_memory, used_memory_human, peak_memory, and the breakdowns by object type (mem_fragmentation_ratio, bytes_per_key).

Common Culprits for High Memory Usage

  1. Inefficient Data Structures: Storing a large number of small items in separate keys when a single complex data structure could be used.

    • Diagnosis: INFO memory will show a high count of keys. If you suspect this, you might also see a high bytes_per_key if individual keys are small but numerous.
    • Fix: Convert many small string keys into a single hash, list, or set. For example, instead of user:1:name, user:1:email, user:1:age, use a hash user:1 with fields name, email, age.
    • Why it works: Valkey optimizes storage for complex types. A hash, for instance, uses ziplist or listpack encoding for smaller hashes, which is far more memory-efficient than individual string keys.
  2. Large Keys and Values: Storing massive amounts of data directly in Valkey.

    • Diagnosis: Review the output of INFO memory. If used_memory is high and bytes_per_key is also high, this is a likely cause. You might need to inspect individual keys using SCAN and TYPE if you suspect specific large ones.
    • Fix: Use Valkey as a cache for frequently accessed, smaller data. For large, infrequently accessed data, store it in a persistent store (like a database or object storage) and use Valkey to cache its presence or metadata.
    • Why it works: Valkey is optimized for speed, not for deep storage of very large objects. Offloading large data reduces the memory pressure on Valkey, allowing it to perform its caching role more effectively.
  3. Default Encoding for Small Collections: Lists, sets, and hashes that are small often default to ziplist or listpack encoding, which is memory-efficient. However, as they grow, they might convert to more memory-intensive representations.

    • Diagnosis: Use OBJECT ENCODING <key> for individual keys. If you see hashtable for a small hash or linkedlist for a short list, that’s a sign.
    • Fix: Manually re-encode these structures if they exceed optimal thresholds. Valkey often does this automatically, but sometimes manual intervention or configuration tuning is needed. For example, if a hash grows too large, it might convert from listpack to hashtable. You can re-save it to force it back to listpack if it’s still within size limits.
    • Why it works: ziplist and listpack are compact, single-memory-block encodings for smaller collections. When they grow beyond certain thresholds (configurable via hash-max-ziplist-entries, hash-max-ziplist-value, list-max-ziplist-entries, etc.), Valkey converts them to their more general, but less memory-efficient, hashtable or linkedlist counterparts.
  4. Unbounded TTLs or Stale Data: Keys that are never evicted because they have no TTL or a very long TTL, and are no longer actively used.

    • Diagnosis: Use INFO persistence or INFO memory to check evicted_keys. If evicted_keys is 0 and used_memory is high, you might have a lot of stale data. SCAN and TYPE can help identify keys without TTLs.
    • Fix: Implement a sensible eviction policy (maxmemory-policy). For example, allkeys-lru (evict least recently used keys) or volatile-lru (evict least recently used keys with TTL). Set appropriate TTLs for your cache data.
    • Why it works: An eviction policy combined with TTLs ensures that Valkey automatically reclaims memory by removing stale or less-used data, preventing unbounded memory growth.
  5. High Memory Fragmentation: The mem_fragmentation_ratio in INFO memory indicates how much actual memory Valkey is using compared to the memory it has allocated from the OS. A ratio significantly above 1.0 means fragmentation.

    • Diagnosis: Check mem_fragmentation_ratio in INFO memory. If it’s consistently above 1.5, fragmentation is an issue.
    • Fix: Restart the Valkey instance. This is often the most straightforward way to reclaim fragmented memory. For persistent issues, consider tuning jemalloc (if used) or understanding your workload patterns.
    • Why it works: Memory allocators like jemalloc (Valkey’s default) can leave small, unused gaps between allocated blocks over time. A restart forces a fresh allocation from the OS, consolidating memory.
  6. Large Number of Small Objects: Even with efficient data structures, a massive number of tiny keys can add up due to per-key overhead.

    • Diagnosis: INFO memory will show a very high keyspace_hits and keyspace_misses count, and if bytes_per_key is very low, this is the problem.
    • Fix: Batch operations. Instead of getting/setting individual small keys, use MGET/MSET or pipeline multiple commands. If you are storing many small, related items, consider compressing them and storing them as a single string value in Valkey.
    • Why it works: Batching reduces the network round trips and the per-command overhead. Compression can significantly reduce the storage size of many small items when combined.

The next thing you’ll likely run into after optimizing memory is a need to scale, either by adding more nodes to a cluster or by increasing the memory of existing nodes.

Want structured learning?

Take the full Valkey course →