Valkey’s MULTI/EXEC command pair doesn’t actually guarantee atomicity in the way most people expect from traditional database transactions.
Let’s watch a MULTI/EXEC block in action. Imagine we have a simple key mykey with the value 10.
import redis
r = redis.Redis(decode_responses=True)
# Initial state
r.set('mykey', '10')
print(f"Initial value of mykey: {r.get('mykey')}")
# Start a transaction
pipe = r.pipeline()
# Queue commands
pipe.incr('mykey')
pipe.incr('mykey')
pipe.get('mykey') # This will get the value *after* the increments
# Execute the transaction
results = pipe.execute()
print(f"Transaction results: {results}")
print(f"Final value of mykey: {r.get('mykey')}")
When you run this, you’ll see:
Initial value of mykey: 10
Transaction results: [12, 12, '12']
Final value of mykey: 12
Here, incr('mykey') was called twice, and the value correctly went from 10 to 12. The get('mykey') in the transaction returned the final value, 12. This looks like a solid, atomic transaction. But what happens when things go wrong? That’s where WATCH comes in.
WATCH is Valkey’s optimistic locking mechanism, designed to prevent race conditions during MULTI/EXEC blocks when external modifications might occur. Instead of locking keys upfront (which would be slow and defeat Valkey’s single-threaded nature for command execution), WATCH simply monitors keys. If any of the watched keys are modified by another client before EXEC is called, the entire MULTI/EXEC block is aborted, and EXEC returns None.
Let’s illustrate this. We’ll set up two clients.
Client 1 (Transaction with WATCH):
import redis
import time
r1 = redis.Redis(decode_responses=True)
# Watch the key
r1.watch('mykey')
print("Client 1: Watching mykey")
# Start a transaction
pipe1 = r1.pipeline()
# Queue commands
pipe1.incr('mykey')
pipe1.get('mykey')
# Simulate some work before executing
print("Client 1: Sleeping for 5 seconds...")
time.sleep(5)
# Attempt to execute
print("Client 1: Attempting to execute transaction...")
results1 = pipe1.execute()
print(f"Client 1: Transaction results: {results1}")
print(f"Client 1: Final value of mykey: {r1.get('mykey')}")
Client 2 (Interfering Client):
import redis
import time
r2 = redis.Redis(decode_responses=True)
# Wait a bit for Client 1 to start watching
time.sleep(1)
print("Client 2: Modifying mykey...")
r2.set('mykey', '100') # This modification will trigger the WATCH
print("Client 2: Finished modifying mykey.")
If you run Client 1 first, then Client 2, and then let Client 1 finish its time.sleep(5), you’ll see something like this in Client 1’s output:
Client 1: Watching mykey
Client 1: Sleeping for 5 seconds...
Client 2: Modifying mykey...
Client 2: Finished modifying mykey.
Client 1: Attempting to execute transaction...
Client 1: Transaction results: None
Client 1: Final value of mykey: 100
Notice results1 is None. Because Client 2 modified mykey after Client 1 started WATCHing it but before EXEC was called, Valkey automatically discarded the transaction. The incr and get commands were never executed. The value of mykey remains 100, as set by Client 2.
The core problem MULTI/EXEC solves is ensuring that a sequence of commands are queued and then sent to the server without interruption. Valkey processes commands sequentially. When you use MULTI, Valkey enters a special state. It stops processing commands immediately and instead enqueues them internally. Only when EXEC is called does Valkey execute all the queued commands in one go. This guarantees that no other client commands can interleave between the commands within your MULTI/EXEC block.
WATCH adds a layer of safety for scenarios where the state of the data you’re operating on might change between when you start your transaction and when you commit it. It’s Valkey’s way of saying, "If the data you’re about to modify has been touched by someone else since you started watching, abort this whole thing." This is crucial for maintaining data integrity in concurrent environments.
A common misconception is that MULTI/EXEC provides true ACID atomicity like a relational database. It does not. While the commands within a MULTI/EXEC block are executed atomically (meaning no other client command can run in between them), the transaction itself can fail if WATCH detects modifications. Furthermore, if one of the commands within the block fails due to a programming error (e.g., trying to INCR a string), Valkey will still execute the subsequent commands in the block. This is known as "all-or-nothing" execution but not necessarily "all-or-nothing" success.
The next hurdle you’ll encounter is understanding how to handle the None return value from EXEC when WATCH has been triggered, and how to re-attempt your transaction.