SYN cookies are a clever defense against SYN flood attacks, a common denial-of-service tactic where an attacker bombards a server with TCP SYN packets, overwhelming its ability to establish legitimate connections.
Here’s how it looks in action, let’s say we have a web server webserver.example.com listening on port 80.
# On the web server, before SYN cookies are enabled
netstat -s | grep "SYN dropped"
# Output might show a non-zero count if under attack
# On the attacker machine (simulated)
hping3 -S -p 80 --flood webserver.example.com
When webserver.example.com is hit with SYN floods, the kernel’s backlog queue fills up. If SYN cookies are not enabled, the server might start dropping incoming SYN packets, preventing new connections.
# On the web server, after SYN cookies are enabled
sysctl net.ipv4.tcp_syncookies
# Output: net.ipv4.tcp_syncookies = 1
# On the web server, during a SYN flood with SYN cookies enabled
netstat -s | grep "SYN cookies sent"
# Output will show a non-zero count, indicating SYN cookies are being used
netstat -s | grep "SYN dropped"
# Output should ideally remain zero or very low, showing the backlog isn't overwhelmed
With SYN cookies enabled, the server doesn’t immediately allocate resources for an incoming connection. Instead, it crafts a special SYN-ACK packet containing a "cookie" derived from the client’s IP address, the server’s IP address, the client’s port, the server’s port, and a secret key. This cookie is essentially an encoded hash.
When the client responds with an ACK packet (the final step in the TCP handshake), the server recalculates the cookie. If it matches the one it sent, it knows the ACK is legitimate and proceeds to establish the connection, allocating resources only then. This way, the server avoids filling its connection queue with bogus requests.
The key levers you control are the net.ipv4.tcp_syncookies kernel parameter and the net.ipv4.tcp_max_syn_backlog setting. tcp_syncookies should be 1 (enabled) on all servers facing the internet. tcp_max_syn_backlog determines the maximum number of half-open connections the server will queue before it starts dropping SYN packets or resorting to SYN cookies. A higher value can buffer against moderate spikes, but too high can still consume memory.
The most surprising true thing about SYN cookies is that they don’t actually increase the server’s capacity to handle connections; they merely shift the burden of proof of a legitimate connection request to the client, thereby preventing resource exhaustion from spoofed SYN packets.
Hardening TCP involves several layers beyond SYN cookies. One critical aspect is managing the TCP connection backlog. The net.ipv4.tcp_max_syn_backlog parameter defines the maximum queue size for incomplete TCP connections. If this queue overflows due to a SYN flood or a sudden surge of legitimate traffic, the kernel will start dropping incoming SYN packets.
# Check current backlog setting
sysctl net.ipv4.tcp_max_syn_backlog
# Output: net.ipv4.tcp_max_syn_backlog = 1024
# Increase the backlog (example value)
sysctl -w net.ipv4.tcp_max_syn_backlog=2048
Increasing tcp_max_syn_backlog allows the server to hold more half-open connections before dropping them. This is particularly useful for handling legitimate traffic spikes, not just attacks. The default value of 1024 is often too low for busy servers. A value of 2048 or even 4096 might be appropriate, depending on your server’s memory and expected traffic.
Another hardening technique is to tune TCP keepalive settings. While not directly for SYN floods, it helps clean up stale connections that might otherwise consume resources.
# Check TCP keepalive defaults
sysctl net.ipv4.tcp_keepalive_time
sysctl net.ipv4.tcp_keepalive_intvl
sysctl net.ipv4.tcp_keepalive_probes
# Example: Reduce keepalive time to 30 minutes (1800 seconds)
sysctl -w net.ipv4.tcp_keepalive_time=1800
# Example: Reduce interval between probes to 15 seconds
sysctl -w net.ipv4.tcp_keepalive_intvl=15
# Example: Reduce number of probes to 5
sysctl -w net.ipv4.tcp_keepalive_probes=5
These settings tell the kernel how often to send keepalive probes to idle connections and how many probes to send before considering the connection dead. Shorter times mean stale connections are cleaned up faster, freeing up resources. The default tcp_keepalive_time of 7200 seconds (2 hours) is often too long for actively managed servers.
Tuning the TCP TIME_WAIT state is also crucial. The net.ipv4.tcp_tw_reuse and net.ipv4.tcp_tw_recycle parameters can help. tcp_tw_reuse allows new TCP connections to be initiated using the same local port for sockets in TIME_WAIT state, provided the new connection’s timestamp is newer than the one in the TIME_WAIT state. This is generally safe. tcp_tw_recycle, however, is more aggressive and can cause issues with NAT or load balancers because it relies solely on timestamps and can prematurely discard valid connections from clients behind the same IP.
# Enable TIME_WAIT reuse (generally safe)
sysctl -w net.ipv4.tcp_tw_reuse=1
# Enable TIME_WAIT recycling (use with extreme caution, often disabled)
# sysctl -w net.ipv4.tcp_tw_recycle=1
The common mistake is enabling tcp_tw_recycle without understanding its implications, leading to dropped connections for users behind NAT. The tcp_fin_timeout parameter controls how long sockets stay in the FIN-WAIT-2 state. Reducing this can also help release resources faster.
# Reduce FIN-WAIT-2 timeout to 30 seconds
sysctl -w net.ipv4.tcp_fin_timeout=30
The one thing most people don’t know is that the secret key used in SYN cookie generation is derived from the system’s network configuration at boot time and is not directly configurable. If this key were to be compromised or become predictable, the entire SYN cookie mechanism would be undermined, allowing an attacker to forge valid cookies without the client’s ACK.
The next step in TCP security is understanding and mitigating TCP reset (RST) attacks.