The Paranoid Pirate pattern in ZeroMQ isn’t about ensuring your messages are delivered to everyone reliably; it’s about ensuring you don’t get cheated by a peer that claims to have received a message but actually dropped it, or that you don’t get stuck waiting for an acknowledgment that will never come.
Let’s see it in action. Imagine a simple scenario: a "Pirate" (server) sending data to a "Paranoid" (client). The Paranoid needs to confirm it got the data before the Pirate sends more.
Pirate (Server) Code Snippet:
import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.ROUTER)
socket.bind("tcp://*:5555")
print("Pirate ready...")
request_count = 0
while True:
# Wait for a request (which includes an empty message for identity)
identity, request = socket.recv_multipart()
print(f"Received request from {identity.decode()} for request #{request_count}")
# Send the data
message = f"Data Packet {request_count}".encode()
socket.send_multipart([identity, message])
print(f"Sent '{message.decode()}' to {identity.decode()}")
# Wait for acknowledgment
ack = socket.recv()
if ack.decode() == "ACK":
print(f"Received ACK for request #{request_count}")
request_count += 1
else:
print(f"Received unexpected message: {ack.decode()}")
# Potentially handle error or retransmit
time.sleep(0.1) # Simulate some work
Paranoid (Client) Code Snippet:
import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.DEALER)
socket.connect("tcp://localhost:5555")
print("Paranoid ready...")
for request_num in range(5):
# Send an empty message first to establish identity, then the request
socket.send_multipart([b"", f"Request {request_num}".encode()])
print(f"Sent request {request_num}")
# Wait for data
identity, data = socket.recv_multipart()
print(f"Received data: {data.decode()}")
# Send acknowledgment
socket.send(b"ACK")
print(f"Sent ACK for data.")
time.sleep(0.1) # Simulate some work
In this example, the ROUTER socket on the Pirate side acts as a smart switch, keeping track of which client sent which message. The DEALER socket on the Paranoid side automatically assigns an identity to outgoing messages. The core of the pattern is the explicit ACK sent by the Paranoid and received by the Pirate. Without this ACK, the Pirate would never know if the Paranoid actually got the data.
The Paranoid Pirate pattern is built on the foundation of ZeroMQ’s ROUTER/DEALER socket pair. The ROUTER socket can receive messages from multiple clients and then route replies back to the specific client that sent the request, preserving the client’s identity. The DEALER socket acts as a client to the ROUTER, automatically handling connection management and message routing. The crucial element is the application-level acknowledgment (ACK) that the client sends back. This acknowledgment isn’t handled by ZeroMQ itself; it’s a contract between the two communicating applications. The Pirate (server) must wait for this ACK before sending another message, and the Paranoid (client) must send it once it has successfully processed the received data.
The real magic, and the reason for the "Paranoid" name, lies in how the Pirate handles timeouts. If the Pirate sends data and doesn’t receive an ACK within a certain window, it assumes the message was lost or the client died. It will then re-send the same message. This requires the client to be idempotent – processing the same message multiple times should have the same effect as processing it once. The Pirate also needs a mechanism to detect if the Paranoid has become unresponsive or is sending duplicate acknowledgments, hence the "Paranoid" aspect. A common implementation involves the Pirate sending a heartbeat or a sequence number with each message and expecting a matching heartbeat or acknowledgment. If a timeout occurs, the Pirate might close the connection and try to re-establish it.
The most counterintuitive aspect of the Paranoid Pirate pattern is that it’s not about guaranteeing delivery to every single node in a broadcast or multicast scenario. Instead, it’s a specific mechanism for one-to-one reliable request-reply where the sender needs absolute certainty that the receiver has the data before proceeding. It enforces a strong, two-way handshaking protocol at the application layer, making it robust against network glitches and receiver failures, but at the cost of increased latency and complexity.
The next step beyond this is often implementing a more sophisticated message queuing or fan-out scenario, potentially using the DEALER socket on the server side to distribute work to multiple worker processes, or employing the XSUB/XPUB pattern for publish-subscribe with guaranteed delivery semantics.