ZeroMQ’s real magic is that it doesn’t care what language you’re speaking, as long as you’re both speaking the same ZeroMQ dialect.
Let’s see this in action. Imagine a Python publisher sending a stream of timestamped messages, and a Go subscriber receiving them.
Python Publisher (pub.py):
import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5555")
print("Publisher started on tcp://*:5555")
while True:
message = f"Timestamp: {time.time()}"
print(f"Sending: {message}")
socket.send_string(message)
time.sleep(1)
Go Subscriber (sub.go):
package main
import (
"fmt"
"log"
"time"
"github.com/pebbe/zmq2go"
)
func main() {
context, _ := zmq2go.NewContext()
socket, _ := context.NewSocket(zmq2go.SUB)
socket.Connect("tcp://localhost:5555")
socket.SetSubscribe("") // Subscribe to everything
fmt.Println("Subscriber connected to tcp://localhost:5555")
for {
message, err := socket.Recv(0)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Received: %s\n", message)
time.Sleep(100 * time.Millisecond) // Simulate some processing
}
}
To run this:
- Save the Python code as
pub.pyand the Go code assub.go. - Make sure you have the ZeroMQ library installed on your system and the
zmq2goGo package (go get github.com/pebbe/zmq2go). - Start the Python publisher in one terminal:
python pub.py - Start the Go subscriber in another terminal:
go run sub.go
You’ll see the Go program printing timestamps that the Python program is sending, seamlessly bridging the language gap.
ZeroMQ is fundamentally a library of network sockets that provides a messaging abstraction. Unlike traditional sockets (like socket.io or raw TCP), ZeroMQ doesn’t manage connections for you in the typical client-server sense. Instead, it provides patterns like Publish/Subscribe, Request/Reply, and Push/Pull. You create sockets of specific types, bind or connect them, and then exchange messages. The library handles the underlying network plumbing, message framing, and even reconnection attempts based on the pattern you’ve chosen.
The core idea is that you have endpoints. An endpoint is an address like tcp://127.0.0.1:5555 or ipc:///tmp/my_queue. You bind an endpoint to a socket if you want it to listen for incoming connections (like a server), and you connect a socket to an endpoint if you want it to initiate a connection (like a client). Crucially, ZeroMQ allows for many-to-one and many-to-many connections within a pattern. For instance, in Publish/Subscribe, one publisher can have many subscribers, and one subscriber can receive messages from multiple publishers (though the example above uses a single publisher and subscriber for clarity).
The patterns are what define the communication flow and the socket types.
- PUB/SUB (Publish/Subscribe): One sender (PUB) broadcasts messages to many receivers (SUB). Subscribers can filter messages they want to receive using
setsockopt(ZMQ_SUBSCRIBE). This is one-to-many. - REQ/REP (Request/Reply): A strict, synchronous request-reply cycle. A REQ socket sends a request and must block until it receives a reply. A REP socket receives a request and must send a reply before it can receive another request. This is typically one-to-one.
- PUSH/PULL (Push/Pull): A load-balancing pattern. PUSH sockets send messages to a group of PULL sockets. Messages are distributed across the PULL sockets. This is one-to-many, but with load balancing.
- DEALER/ROUTER: More flexible, asynchronous versions of REQ/REP. DEALERs can send multiple requests without waiting for replies and can receive replies out of order. ROUTERs can send replies to specific clients and can receive messages from multiple DEALERs.
ZeroMQ handles serialization implicitly for basic types. When you send_string in Python, ZeroMQ frames that string and sends it. When zmq2go receives it, it knows how to unframe it into a string. For more complex data, you’ll need to serialize it yourself (e.g., using JSON, Protocol Buffers, MessagePack) before sending and deserialize after receiving. The interop is at the message framing and transport level, not at the language’s internal data representation.
The real power is in the zmq.Context and socket objects. The Context is like the ZeroMQ "environment" for your process. You typically create one context and then create multiple sockets from it. Each socket is then configured with a type (zmq.PUB, zmq.SUB, etc.) and an address (bind or connect). The send and recv operations are the core of the interaction, blocking or non-blocking depending on flags, and transferring bytes between processes or machines.
Most people don’t realize that the ZMQ_SUBSCRIBE option in the SUB socket takes a prefix string. If you subscribe with socket.setsockopt_string(zmq.SUBSCRIBE, "timestamp:"), the subscriber will only receive messages that start with "timestamp:". An empty string "" subscribes to everything. This prefix matching is a fundamental, built-in filtering mechanism that can drastically reduce the amount of data a subscriber needs to process.
The next logical step is to explore the REQ/REP pattern and see how synchronous request-response works across languages, potentially involving C++ for performance-critical components.