Storage IOPS and throughput are the two primary metrics for understanding how fast your storage can handle data requests.
Let’s see this in action. Imagine you’re running a database. A common workload involves many small, random reads and writes. This is where IOPS, or Input/Output Operations Per Second, shines. If your database is constantly waiting for data to be fetched from disk, you’re likely IOPS-bound.
# Example: Monitoring IOPS on a Linux system using iostat
# This command shows I/O statistics for all devices (-x for extended) every 5 seconds (-d 5)
iostat -xd 5
The output might look something like this:
Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz await svctm %util
sda 0.50 2.10 4.00 16.80 0.00 0.10 0.00 4.55 8.20 15.50 0.05 13.10 4.50 1.17
nvme0n1 150.75 75.30 603.00 301.20 1.20 0.50 0.79 0.66 1.50 2.20 1.50 1.80 3.20 71.25
Here, nvme0n1 is showing 150.75 r/s (reads per second) and 75.30 w/s (writes per second). The total IOPS for this device is roughly 226. The await column shows the average time in milliseconds (ms) for I/O requests to be served. High await times often indicate a bottleneck.
Throughput, on the other hand, measures the amount of data transferred per unit of time, typically in megabytes per second (MB/s) or gigabytes per second (GB/s). This is crucial for workloads that involve large, sequential data transfers, like video streaming, large file backups, or data warehousing queries that scan massive datasets.
# Example: Monitoring throughput on a Linux system using iostat
# Same iostat command as above, focusing on the rkB/s and wkB/s columns
iostat -xd 5
In the previous iostat output, nvme0n1 has 603.00 rkB/s (read kilobytes per second) and 301.20 wkB/s (write kilobytes per second). This means it’s moving about 904.2 KB/s total, or approximately 0.9 MB/s. If your application needs to move gigabytes of data quickly, this device would be a throughput bottleneck.
When provisioning storage, you need to consider both. A high-IOPS drive might be slow for large sequential reads, and a high-throughput drive might struggle with thousands of small, random operations. The key is to match the storage characteristics to your application’s dominant workload. For a transactional database, you’d prioritize high IOPS and low latency. For a video editing workstation, you’d focus on high throughput.
The svctm (service time) and %util columns are also critical. svctm is the time the storage device actually spent working on I/O requests. If svctm is significantly higher than await, it means the device is spending a lot of time processing each request, indicating it’s saturated. %util shows the percentage of time the device was busy. Consistently high %util (near 100%) is a strong indicator of a bottleneck.
The distinction between IOPS and throughput becomes even more nuanced when you consider block size. A single IOPS operation can transfer a small block (e.g., 4KB) or a large block (e.g., 1MB). Thus, you can have high IOPS with low throughput, or low IOPS with high throughput. For instance, a 7200 RPM hard drive might achieve 100 IOPS by transferring 4KB blocks, resulting in ~0.4 MB/s throughput. The same drive might achieve 100 MB/s throughput by transferring 1MB blocks, but only at 100 IOPS. Modern SSDs and NVMe drives can achieve hundreds of thousands of IOPS and multiple gigabytes per second of throughput, often simultaneously.
The performance of your storage subsystem is rarely just about the raw numbers of the drive itself. It’s also heavily influenced by the storage controller, the bus interface (SATA, SAS, NVMe), the operating system’s I/O scheduler, and the application’s access patterns. Understanding these interactions is key to diagnosing and resolving performance issues. For example, an NVMe drive capable of 500,000 IOPS might only deliver 50,000 IOPS if the PCIe lanes are saturated by other devices, or if the CPU is too busy to feed it data fast enough.
The next step after understanding IOPS and throughput is to look at latency, which is the time it takes for a single I/O operation to complete from start to finish.