The ZeroMQ poller and reactor are not about waiting for messages; they’re about actively managing multiple connections and their readiness to send or receive without blocking.
Let’s see it in action. Imagine a simple request-reply scenario where a server handles multiple clients.
Server (server.py)
import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.ROUTER)
socket.bind("tcp://*:5555")
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
while True:
socks = dict(poller.poll(100)) # Poll with a 100ms timeout
if socket in socks and socks[socket] == zmq.POLLIN:
# Receive message, which includes the client's identity
identity, request = socket.recv_multipart()
print(f"Received request from {identity.decode()}: {request.decode()}")
# Simulate work
time.sleep(0.5)
# Send reply back to the specific client
reply = b"World"
socket.send_multipart([identity, reply])
print(f"Sent reply to {identity.decode()}")
Client (client.py)
import zmq
import sys
context = zmq.Context()
socket = context.socket(zmq.DEALER)
socket.connect("tcp://localhost:5555")
identity = sys.argv[1].encode()
request = b"Hello"
print(f"Client {identity.decode()} sending '{request.decode()}'")
socket.send_multipart([identity, request])
# Wait for the reply
reply = socket.recv()
print(f"Client {identity.decode()} received reply: {reply.decode()}")
When you run python server.py and then python client.py 1, python client.py 2 in separate terminals, you’ll see the server handling requests from both clients concurrently. It doesn’t get stuck waiting for one client to finish before checking the other. The poller.poll(100) call is the key: it checks all registered sockets for readiness to receive (or send, if registered with zmq.POLLOUT) and returns a dictionary of ready sockets. The server then iterates through this, processing only those sockets that are actually ready, preventing any single slow client from blocking the entire application.
The ZeroMQ poller allows you to monitor multiple ZeroMQ sockets simultaneously for events like incoming messages (POLLIN) or readiness to send (POLLOUT). It’s the foundation of an event-driven architecture where your application reacts to external events rather than actively polling or blocking. The reactor pattern, often built on top of the poller, is a higher-level abstraction that maps incoming events to specific handler functions. You register a socket with the poller, and when an event occurs on that socket, the poller signals your main loop. The reactor then dispatches this signal to the appropriate piece of code that knows how to handle that specific type of message or event from that specific source. This decouples the event detection from the event handling logic.
The ROUTER socket on the server is crucial here. Unlike a REP socket which expects a strict request-reply sequence and will block if you try to send before receiving, ROUTER allows you to receive messages from multiple clients and send replies back to specific client identities. The DEALER socket on the client is the counterpart, allowing it to send messages and receive replies without needing to know the server’s internal structure. The recv_multipart() on the server is what retrieves both the client’s identity frame and the actual message frame. The send_multipart() then uses that same identity frame to route the reply back.
A common mistake is to think you can just recv() on a socket within a loop without checking for readiness. If no message is available, recv() will block indefinitely, defeating the purpose of concurrency. The poller, by returning an empty dictionary or a dictionary with only non-ready sockets when no events occur (within the timeout), allows your loop to continue executing. This means your application can perform other tasks, maintain other connections, or simply yield control back to the operating system while waiting for I/O events.
When using POLLOUT, you’re essentially asking "can I send data on this socket right now without blocking?" This is vital for scenarios where you might have a large amount of data to send or multiple outgoing messages queued. If a socket isn’t ready for POLLOUT, it means its internal send buffer is full, and attempting to send() will block. By registering for POLLOUT and waiting for the event, you ensure that your send() operation will succeed immediately.
The poller.poll(timeout) method is where the magic happens. The timeout parameter (in milliseconds) determines how long the poll call will wait for an event before returning. A timeout of 0 makes poll non-blocking – it checks for events and returns immediately. A timeout of -1 makes poll blocking, waiting indefinitely until an event occurs. The returned socks dictionary maps socket objects to their event masks. You then check which sockets have the zmq.POLLIN flag set to know which ones have incoming data.
You might be tempted to use socket.getsockopt(zmq.EVENTS) to check for events directly. However, this only tells you what has happened historically, not what is currently ready for I/O without blocking. The poller.poll() method is the definitive way to query the readiness of multiple sockets at a given moment in time.
The next step in managing complex event-driven systems is often to integrate a framework that handles the reactor pattern more explicitly, such as pyzmq’s zmq.asyncio module, which allows you to use ZeroMQ sockets within Python’s asyncio event loop, seamlessly blending I/O-bound operations with other asynchronous tasks.