SYN cookies are a defense mechanism that prevents a system from being overwhelmed by a TCP SYN flood attack. Instead of allocating resources to each incoming SYN request, the system sends back a SYN-ACK with a specially crafted sequence number (the "SYN cookie") and waits for the client’s ACK. If the ACK arrives, the system can reconstruct the connection state from the cookie, effectively bypassing the need for initial resource allocation.
Let’s see this in action. Imagine a server under a SYN flood. Normally, it would try to create a SYN_RECV state for each bogus SYN packet, consuming memory and eventually exhausting its backlog.
# On the server, observing established connections (or lack thereof)
ss -s
# Output might show a high number of SYN_RECV states before SYN cookies are enabled.
# After enabling SYN cookies, the server is still responsive
# and can establish legitimate connections even under attack.
# A legitimate client can still connect:
telnet <server_ip> <port>
# The server responds with a SYN-ACK.
# The client sends an ACK.
# The connection is established.
The core problem SYN cookies solve is the vulnerability of the TCP handshake’s initial state. When a client initiates a TCP connection, it sends a SYN packet. The server responds with a SYN-ACK and allocates resources (memory, state) to track this half-open connection. A SYN flood attack exploits this by sending a massive volume of SYN packets, often with spoofed source IP addresses. The server dutifully responds to each with a SYN-ACK, but since the source IPs are fake or unreachable, the final ACK never arrives. The server’s connection table fills up with these half-open connections, preventing legitimate users from establishing new connections.
SYN cookies flip this model. When the server receives a SYN packet and its connection backlog is full, instead of dropping the packet or allocating state, it sends back a SYN-ACK packet. The key is that this SYN-ACK doesn’t contain a normal sequence number. Instead, it contains a "SYN cookie," which is a cryptographic hash derived from information about the connection, such as the source IP, source port, destination IP, destination port, and a secret key. Crucially, the server does not store any state about this connection. It just sends the SYN-ACK with the cookie.
When the legitimate client (or an attacker attempting to complete the handshake) receives the SYN-ACK, it responds with an ACK. If this ACK packet arrives, the server can extract the SYN cookie from the sequence number field of the ACK packet. Using the same cryptographic algorithm and secret key, the server can reconstruct the initial connection parameters (IPs, ports, and the original sequence number) from the cookie. If the reconstructed parameters match what the server expects, it knows this ACK corresponds to a valid SYN-ACK it sent, and it can then allocate resources and establish the connection. The beauty is that this state reconstruction only happens after the client has proven it received the SYN-ACK and can send a valid ACK, effectively offloading the state-tracking burden to the client.
To enable SYN cookies on Linux, you typically modify kernel parameters. The relevant parameter is net.ipv4.tcp_syncookies.
# Check the current status of SYN cookies
sysctl net.ipv4.tcp_syncookies
# Expected output: net.ipv4.tcp_syncookies = 0 (disabled) or 1 (enabled)
# Enable SYN cookies immediately
sudo sysctl -w net.ipv4.tcp_syncookies=1
# Make the change persistent across reboots by editing /etc/sysctl.conf
# Add or uncomment the following line:
# net.ipv4.tcp_syncookies = 1
# Then apply the changes:
sudo sysctl -p
By setting net.ipv4.tcp_syncookies to 1, you instruct the kernel to use the SYN cookie mechanism when the system’s TCP connection backlog is nearing its limit. The kernel will then generate SYN cookies for incoming SYN packets and only allocate full connection state upon receiving a valid ACK that can be decoded into a cookie. This prevents the connection table from being filled by malicious SYN packets.
The effectiveness of SYN cookies relies on the client’s ability to correctly reconstruct the sequence number from the cookie and send a valid ACK. If the client is also under duress or if the network path is highly lossy, legitimate connections might still be impacted. However, for typical scenarios, it’s a highly effective, low-overhead defense against SYN floods.
One aspect often overlooked is that SYN cookies are only enabled when the system is under load. If your tcp_syncookies parameter is set to 1, it means the kernel will use cookies when needed, but it doesn’t mean every SYN packet will be processed with cookies all the time. The kernel will first try to handle SYN packets normally. Only when the tcp_max_syn_backlog is nearly full does it start generating cookies. This is a performance optimization; cookies involve more computation than normal state allocation, so you don’t want to use them unnecessarily.
The next hurdle you might face after implementing SYN cookies is understanding how to tune the tcp_max_syn_backlog parameter, which dictates when the kernel starts employing this cookie defense.