TCP File Transfer: Tools and Performance Optimization (2026)

TCP file transfer is surprisingly inefficient for large files because it’s fundamentally designed for interactive, small-packet communication.

Let’s watch pv and netcat do their thing. Imagine a 10GB file on server_a and we want to pull it to client_b.

On server_a:

dd if=/dev/zero of=~/large_file.bin bs=1G count=10
pv ~/large_file.bin | nc -l -p 12345

On client_b:

nc server_a 12345 > ~/received_large_file.bin

You’ll see pv on server_a reporting the transfer speed, and netcat on client_b will just sit there until the file is done. It’s a simple pipe, but it’s also a bottleneck.

The core problem is TCP’s congestion control and flow control mechanisms. When you send data, TCP assumes the network might be congested and backs off. It uses algorithms like Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery. For a single, large, continuous stream like a file transfer, these mechanisms, while crucial for general network health, can severely limit throughput. The sender’s congestion window (cwnd) grows slowly, and even if the receiver has plenty of buffer space, the sender won’t fill it if it thinks the network is unhappy.

Here’s how you can squeeze more performance out of TCP file transfers:

1. Increase TCP Buffer Sizes: The default TCP send and receive buffer sizes are often too small for high-speed, long-latency links. Increasing these allows the sender to "stuff" more data into the network before waiting for acknowledgments.

• Diagnosis: Check current buffer sizes:

  sysctl net.core.rmem_max
  sysctl net.core.wmem_max
  sysctl net.ipv4.tcp_rmem
  sysctl net.ipv4.tcp_wmem
  

  These values are in bytes. tcp_rmem and tcp_wmem are triplets: min, default, max.
• Fix: Increase them system-wide. For example, on Linux:

  sudo sysctl -w net.core.rmem_max=16777216 # 16MB
  sudo sysctl -w net.core.wmem_max=16777216 # 16MB
  sudo sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
  sudo sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
  

  These settings allow for much larger send and receive windows, potentially up to 16MB. This helps keep the pipe full on high-bandwidth, high-latency networks by allowing more unacknowledged data to be in flight.

2. Tune TCP Congestion Control Algorithm: Linux offers several congestion control algorithms. cubic is the default and generally good, but bbr (Bottleneck Bandwidth and Round-trip propagation time) can perform much better on networks with significant packet loss or variable latency.

• Diagnosis: Check current algorithm:

  sysctl net.ipv4.tcp_congestion_control
  

• Fix: Switch to bbr (if available, requires kernel 4.9+):

  sudo sysctl -w net.core.default_qdisc=fq # Required for BBR
  sudo sysctl -w net.ipv4.tcp_congestion_control=bbr
  

  bbr aims to explicitly measure the bottleneck bandwidth and round-trip time, rather than inferring congestion from packet loss. This allows it to maintain a larger congestion window when the network isn’t actually congested, leading to higher throughput.

3. Use rsync with Compression: For repetitive transfers or when bandwidth is constrained, rsync can leverage existing files and its delta-transfer algorithm. Combined with compression, it can dramatically reduce the amount of data sent.

• Diagnosis: Not applicable; this is an alternative tool.
• Fix: Use rsync with -z for compression. For a full file transfer:

  # On the receiving side:
  rsync -avz --progress user@server_a:/path/to/large_file.bin /path/on/local/
  

  The -z flag compresses the data stream during transfer. This is highly effective if the file has redundancy or if the link is slow, as sending compressed data is faster than sending uncompressed data over a low-bandwidth link.

4. Use screen or tmux for Long Transfers: While not a performance optimization, it’s a crucial practical tip. If your SSH session drops, your netcat or rsync transfer will die.

• Diagnosis: Not applicable.
• Fix: Start a screen or tmux session before initiating the transfer.

  screen -S filetransfer
  # Then start your netcat or rsync command inside screen
  # To detach: Ctrl+A, D
  # To reattach: screen -r filetransfer
  

  This detaches the session from your terminal, allowing it to run in the background even if your SSH connection breaks. The transfer continues uninterrupted.

5. Consider mosh for Unreliable Networks: If your network connection is highly unstable (e.g., Wi-Fi with dropouts), mosh (Mobile Shell) can provide a more resilient interactive experience than SSH, which can sometimes help keep long-running TCP transfers alive.

• Diagnosis: Not applicable.
• Fix: Install mosh on both client and server, then connect:

  mosh user@server_a
  # Then start your transfer
  

  mosh uses UDP for its control channel and can maintain a session across IP address changes and intermittent connectivity, making it more robust for long-running tasks over unreliable links.

6. Use Parallel Streams (with caution): For very high-speed networks, a single TCP stream might not saturate the link due to protocol overhead or single-stream limitations. You can sometimes achieve higher throughput by opening multiple parallel TCP connections and transferring different parts of the file simultaneously. Tools like parallel or aria2c can do this.

• Diagnosis: Not applicable.
• Fix: Using aria2c for a single file (it’s designed for multi-source downloads but works here):

  # On the receiving side:
  aria2c -x 8 -s 8 http://server_a/large_file.bin -o received_large_file.bin
  # Note: This requires serving the file via HTTP, e.g., using 'python -m http.server' on server_a
  

  By using multiple connections (-x 8 and -s 8 for 8 connections), you can often saturate higher-bandwidth links. Each connection gets its own TCP congestion window, and the aggregate throughput can exceed what a single stream can achieve. This is also what tools like wget -P with multiple URLs or download managers do.

The next error you’ll hit is likely a disk I/O bottleneck if your storage can’t keep up with the network speed you’ve achieved.