34

I understand what IOPS and throughput are. Throughput measures data flow as MB/s and IOPS says how many I/O operations are happening per second.

What I don't understand is why many storage services just show the IOPS they provide. I really can't see any scenario where I would prefer to know the IOPS instead of the throughput.

Why do IOPS matter? Why does AWS mainly shows its storage provisions in IOPS? Where are IOPS more relevant than throughput (MB/s)?

EDIT:

Some people are looking into this question as if I asked what random access is and how it impacts performance or how HDD and SSD work... although I think this information is useful for people new to storage behavior, a lot of focus is being applied to this and it isn't the goal of the question, the question is about "What new piece of information do I get when I see an IOPS number, that I wouldn't get seeing a throughput (MB/s) number?"

mFeinstein
  • 503
  • 2
  • 5
  • 10
  • 1
    [Interesting article](http://recoverymonkey.org/2012/07/26/an-explanation-of-iops-and-latency/) – Tim May 24 '17 at 01:04
  • 3
    If you want to move large data, you care about throughput. If you need to r/w lots of small data you need more IOPS. e.g.1 If there is a single operation that can read MB of data from the device then you only need 1 operation to get a high throughput. e.g.2 If you need to read dozens of file attributes you are not looking at a large amount of data each time but need to do a lot of operations to fetch small bits of data. Throughput would be low but you would need lots of operations. – TafT May 24 '17 at 08:47

8 Answers8

58

This is because sequential throughput is not how most I/O activity occurs.

Random reads/write operations are more representative of normal system activity, and that's usually bound by IOPS.

Streaming porn from one of my servers to our customers (or uploading to our CDN) is more sequential in nature and you'll see the impact of throughput there.

But maintaining the database that catalogs the porn and tracks user activity through the site is going to be random in nature, and limited by the number of small I/O operations/second that the underlying storage is capable of.

I may need 2,000 IOPS to be able to run the databases at peak usage, but may only see 30MB/s throughput at the disk level because of the type of activity. The disks are capable of 1200MB/s, but the IOPS are the limitation in the environment.

This is a way of describing the capacity potential of a storage system. An SSD may have the ability to do 80,000 IOPS and 600MB/s throughput. You can get that throughput with 6 regular 10k SAS disks, but the would only yield around 2,000 IOPS.

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • Could you give me an example where IOPS would give me an insight on my system's performance where MB/s won't be useful? – mFeinstein May 22 '17 at 22:09
  • @mFeinstein See porn example above. – ewwhite May 22 '17 at 22:23
  • 34
    +1 for porn example lol – mFeinstein May 22 '17 at 22:56
  • So is it safe to say that throughput and IOPS complement each other? A low throughput could be either high IOPS or an idle system, so the IOPS information comes to "explain" the throughput levels? – mFeinstein May 22 '17 at 22:58
  • See above edits. – ewwhite May 22 '17 at 23:40
  • 2
    Also, an operating system is likely doing a bunch of little random accesses. Seq throughput won't help. That's a reason to run the OS on an SSD, at least in PCs. – sudo May 23 '17 at 01:46
  • 3
    I often see fully utilized disks that do ~2MB/sec. That's because it's 100% random IO. Sometimes, incredible perf gains are possibly by laying out data sequentially on disk (e.g. removing fragmentation, indexing in databases). – boot4life May 23 '17 at 15:58
  • Yeah, but just to be clear, the question is not about random access...I am pretty aware of random access and etc, my point was "if my throughput is low, isn't it enough to show me the whole picture? What new information will IOPS give me that will change the way I look into the problem?" – mFeinstein May 24 '17 at 00:43
  • @mFeinstein See: https://www.reddit.com/r/storage/comments/6cyw3s/why_do_iops_matter/dhz3s0v/ – ewwhite May 24 '17 at 13:17
36

Throughput

Throughput is useful when you're doing things like copying files. When you're doing almost anything else it's random reads and writes across the disk that will limit you.

IOPS

IOPS typically specify the size of each data packet. For example, AWS gp2 can do 10,000 IOPS with a 16KiB payload size. That multiplies out to 160MiB/sec. However, it's probably unlikely that you'll use the full payload size all the time, so actual throughput will probably be lower. NB KiB is 1024 bytes, KB is 1000 bytes.

Because IOPS specify a packet size that does give you total throughput as well. Whereas high throughput doesn't mean you have high IOPS.

Scenarios

Consider these scenarios:

  • Booting your PC. Consider the difference between an SSD and a spinning disk in your computer, which is something many people have first hand experience with. With a spinning disk the boot time can be a minute, whereas with an SSD this can come down to 10 - 15 seconds. This is because higher IOPS leads to lower latency when information is requested. The throughput of the spinning disk is quite good, 150MB/sec, though the SSD is likely higher this isn't why it's faster - it's the lower latency to return information.
  • Running an OS update. It's going all over the disk, adding and patching files. If you had low IOPS it would be slow, regardless of throughput.
  • Running a database, for example selecting a small amount of data from a large database. It will read from the index, read from a number of files, then return a result. Again it's going all over the disk to gather the information.
  • Playing a game on your PC. It likely loads a large number of textures from all over the disk. In this case IOPS and throughput are likely required.

LTO Tape

Consider for a moment a tape backup system. LTO6 can do 400MB/sec, but (I'm guessing here) probably can't even do one random IOP, it could be as low as seconds per IOP. On the other hand it can probably do a whole lot of sequential IOPS, if an IOPS is defined as reading or writing a parcel of data to tape.

If you tried to boot an OS off tape it would take a long time, if it worked at all. This is why IOPS is often more helpful than throughput.

To understand a storage device you probably want to know if it's random or sequential IOPS, and the IO size. From that you can derive throughput.

AWS

Note that AWS does publish both IOPS and throughput figures for all its storage types, on this page. General purpose SSD (gp2) can do 10,000 16KiB IOPS, which gives a maximum of 160MB/sec. Provisioned IOPS (io1) is 20,000 16KiB IOPS, which gives a maximum of 320MB/sec.

Note that with gp2 volumes you get 3 IOPS per GB provisioned, so to get 10,000 IOPS you need a 3.33TB volume. I don't recall if io1 volumes have a similar limitation (it's been a while since I did the associate exams where that kind of thing is tested), but I suspect they do, and if so it's probably 60IOPS per GB.

Conclusion

High sequential throughput is useful, and in some cases is the limiting factor to performance, but high IOPS is likely to be more important in most cases. You do still of course need reasonable throughput regardless of IOPS.

Perelx
  • 103
  • 4
Tim
  • 30,383
  • 6
  • 47
  • 77
  • I get that IOPS measures random access performance, but it doesn't actually shows how fast you are doing things...you might be doing 10000 IOPS, but this could be something slow or fast, the only way to know is knowing how many MB/s the operation is consuming. – mFeinstein May 22 '17 at 23:48
  • IOPS typically specify the data payload size. AWS says 16KiB. So 10,000 IOPS at 16KiB/s gives you 160MB/sec. – Tim May 23 '17 at 01:18
  • oh that makes more sense, I guess that information is usually hidden – mFeinstein May 23 '17 at 01:50
  • I'm not sure it's hidden, it may just be assumed that the reader understands that. For AWS it is documented on the EBS page. – Tim May 23 '17 at 02:05
  • hidden in a sense that is not obvious when you are reading the numbers, you have to go and search for it somewhere...like when you buy a new SSD and they just say the IOPS on the SSD and not the block size they used for the tests. – mFeinstein May 23 '17 at 02:20
  • And usually the tests are conducted at some unreasonable payload size... 4kb or something. So the IOPS numbers are usually inflated too. – ewwhite May 23 '17 at 10:45
  • 2
    10000 IOPS at 16KB will not translate to 20000 IOPS at 8KB, though (maybe ~11000). This means one needs to know both IOPS and throughput to assess a drive/workload. – boot4life May 23 '17 at 16:00
  • @boot4life I think you're right about that. It's a maximum of that throughput, but in practice it's probably less. – Tim May 23 '17 at 18:47
  • @mFeinstein updated based on your updated question – Tim May 24 '17 at 01:03
  • 4
    Just to be pedantic, it's still 1 IOPS, not 1 IOP. The s isn't a plural – Matthew Steeples May 24 '17 at 19:34
  • @Tim you said "high throughput doesn't mean you have high IOPS" how is this possible, won't every sequential block be one IO operation, leading to high IOPS in high throughput even on sequential reads? – mFeinstein May 25 '17 at 13:53
  • @mFeinstein read the paragraph about LTO tape again. It has high sequential IOPS but low random IOPS. So you're mostly right there. – Tim May 25 '17 at 18:35
  • "KiB is 1024 bits, KB is 1000 bits." No, KiB is 8192 bits. Also 10000 * 16 KiB is not 160 MiB. – Ben Voigt May 25 '17 at 19:32
  • Thanks for the correction @BenVoigt, it was incorrect as I had bits instead of bytes. However according to [Wikipedia](https://en.wikipedia.org/wiki/Kibibyte) and [this answer](https://ux.stackexchange.com/questions/13815/files-size-units-kib-vs-kb-vs-kb) a KiB is 1024 bytes, not 8192 as you suggested. If you have a reference that backs up 8192 I'd like to read it. – Tim May 25 '17 at 20:14
  • @Tim: 8192 bits and 1024 bytes are the same, 1 KiB equals both. Google agrees: https://www.google.com/search?q=1+KiB+in+bits – Ben Voigt May 25 '17 at 20:36
  • @BenVoigt thanks, I really need to have a coffee before I start writing things that require attention to detail. I had corrected my answer to be right but I misread your comment bits/bytes. – Tim May 25 '17 at 20:40
  • @Tim since you left your text on hight throughput and low IOPS, could you point me a case where this is true? I understand on tape storage it won't be, as I was rigtht, as you pointed out, but this left space to conclude there are others where this applies? – mFeinstein May 25 '17 at 23:12
  • 1
    I can't think of any others. Most things that are high IOPS are reasonably high throughput, but in most cases are useful because of the IOPS not the throughput. Another example could be a relational database, though that's not a storage device it's software. I'm not sure what else you want out of this question, I think the concept has been thoroughly explained to you. Anything with a high seek time or latency probably has low IOPS, but throughput can be decoupled and be high in some cases. – Tim May 26 '17 at 01:27
  • @Tim _I know, right?_ – ewwhite May 27 '17 at 01:44
  • While very broad on the subject I get weird feeling that You have mistaken IOPS as Input/Output Packet Size rather than I/O per second. Also as @MatthewSteeples has pointed out there is no IOP, perhaps Input /Output Operation (IOO) – bocian85 Sep 20 '17 at 12:02
  • I take IOPS as operations per second, but each operation likely has a fixed packet size. Each device could have a different packet size. – Tim Sep 20 '17 at 18:00
6

While ewwhite's answer is completely correct, I wanted to provide some more concrete numbers just to help put why the difference matters in perspective.

As ewwhite already correctly stated, most non-streaming applications primarily perform non-sequential disk operations, which is why IOPS matter in addition to theoretical peak throughput.

When a coworker and I first installed SSDs in our development systems to replace the HDDs we'd previously been using, we ran some performance measurements on them that really highlighted why this matters:

SATA HDD Results:

Sequential Read Throughput: ~100 MB/s
Non-Sequential Read Throughput (2k blocks, IIRC): ~1 MB/s

PCIe-attached SSD Results:

Sequential Read Throughput: ~700 MB/s
Non-sequential Read Throughput (2k blocks, IIRC): ~125 MB/s

As you can clearly see from the example, just listing a max throughput for each device would give an extremely inaccurate picture of how they compare. The SSD is only about 6-7x as fast as the HDD when reading large files sequentially, but it's over 100x as fast when reading small chunks of data from different parts of the disk. Of course, with HDDs, this limitation is largely due to the fact that HDDs must physically move the r/w head to the desired track and then wait for the desired data to spin under the head, while SSDs have no physical parts to move.

Our compile times improved much more dramatically than a simple comparison of the maximum throughputs would have suggested. Builds that previously took over 30 minutes now finished in about a minute, since the disk I/O during a large build consists of reading and writing lots of separate source files which aren't individually very large and may be scattered physically all over the disk.

By providing both throughput and IOPS numbers, you can get a much better idea of how a given workload will perform on a given storage device. If you're just streaming large amounts of data that isn't fragmented, you'll get pretty close to the maximum throughput. However, if you're doing a lot of small reads and/or writes that are not stored sequentially on the disk, you'll be limited by IOPS.

Community
  • 1
reirab
  • 162
  • 1
  • 8
3

To perform an IO operation the drive(s) must go through a series of operations. For a mechanical hard drive they need to.

  1. Seek to the right track and select the right head.
  2. Wait for the platter to rotate to the right position.
  3. Actually transfer the data.

The time taken for 3 depends on the size of the block of data, but the time taken for 1 and 2 is independent of the size of the request.

The headline throughput and IOPs figures represent extreme cases. The headline throghput figures represent the case where each operation involves a large block of data, so the drive spends most of it's time actually moving data.

The headline IOPs figure represent the case where the blocks of data are very small so the majority of time is spent seeking the heads and waiting for the platters to rotate.

For many workloads the blocks are sufficiently small that the number of blocks to be transferred is far more important than the size of the blocks.

Peter Green
  • 4,056
  • 10
  • 29
2

There are two types of bottleneck that you can experience on IO volumes (or IO in general in fact).

Actual performance is indeed measured to include a component based on the volume of data moved, scaled by the available bandwith or similar, unitcost * size, but there is also an overhead associated with requests, that is constant, be that disk, network, or numerous other things.

unitcost * size + overhead. the equation of a line.

If the unitcost is large, or the size is large, then it makes sense to charge based around these volumes, such as mobile phone networks, on the other hand sometimes the overheads are far more critical.

You can do a simple experiment of this yourself, create a directory with a few 1GB files (or whatever is practical, something that large enough it takes several seconds to read/write it), and then create a folder with a million 100 byte files (note, thats 0.1GB of data), and then see what happens to your throughput when you start trying to move all this stuff say between different partitions/disks - you'll get performance throttled by throughput for the large files, and throttled by the number of files for the smaller stuff.

I would assume amazon are aware of both charging models and have simply found one better represents their infrastructure's capabilities.

There is a limit on the size of an IOP that is broadly related to the ammount the store can transfer in a "cycle" anyway, thus large requests still end up costing you multiple IOPS.

There's a nice piece here from amazon themselves about IOPS and costing, and 'savings' they pass on through optimisations

I/O Characteristics and Monitoring

Not read it all but it looks interesting, if you're curious about this area.

Iain Price
  • 71
  • 2
2

Answering your question

"What new piece of information do I get when I see an IOPS number, that I wouldn't get seeing a throughput (MB/s) number?"

directly, it is how many IO operations of specified queue depth and file size can storage do per second. You can calculate throughput at given conditions using following formula:

IOPS * file size = Throughput

Storage tests may generate different number of IOPS depending on the file size and queue depth. At queue depth = 1 or 2, controller won't take advantage of caching, while at queue depth 32, 256, 512 number rise several times and doesn't change much. At file size 128KB IOPS count could be lower next to 4KB files, but throughtput - higher.

The best way to evaluate performance of a storage is to seek IOPS and throughput tests at several different block size and queue depth.

Eugene
  • 287
  • 1
  • 11
  • I believe you might be confusing IOPS with throughput a bit... Throughput is not a synonymous of continuous access, but the total MB/s the storage was able to process at a given time.... So when you say the HDD and the SSD would have the same throughput, it is for continuous access... As there is throughput for random access as well... Just a lot less for HDDs in general because of the seeking time. – mFeinstein May 24 '17 at 18:03
  • So you should include in your answer that you are referring to continuous access at the beginning and to random access at the end, as IOPS isn't synonymous to random access as well... Just when it makes more sense to use IOPS as a measurement – mFeinstein May 24 '17 at 18:06
  • @mFeinstein I've edited the answer, have a look. – Eugene May 25 '17 at 11:26
1

Generally speaking, IOPS is more difficult to get than throughput. If you have lots of IOPS, you will have enough throughput most of the time.

With classic hard drives, the number of axes is your limiting factor, since the head must be physically moved on each drive: and it's terribly slow. SSDs have far better IOPS capacity.

If you have just one user, copying one big file to the network, you may have only a dozen of seeks to get the data, and the rest will be only streaming from the disk.

However, if you are hitting a database, or have lots of concurrent users, you will have to access different parts of your storage at the same time, with the IOPS skyrocketing.

Just updating 10 rows in parallel on a relational database might end in generating hundreds of IOs: reading the indexes, reading the data, appending the logfile, updating the indexes and the data. Most operating systems and databases try very hard to limit the number of IOs by caching and delaying/grouping the IOs when possible.

Xavier Nicollet
  • 600
  • 3
  • 10
1

I will be answering my own question as well because I think most answers went a lot off topic and the answer could be a lot simpler:

If you look at your storage devices throughput only, you might miss what's going on... If there is low throughput (low MB/s) you might have a slow device OR having a lot of random access in a HDD or some other device that doesn't handles random access nicely.

By looking into the IOPS and knowing the chunk size of each I/O operation you can know how many access the storage device is able to handle and what is the throughput of these IOPS (chunk size * IOPS).

So looking at high IOPS you can conclude your storage device is handling a lot of random access, even if this comes with low throughput....or maybe you are looking into low IOPS which have the same low throughput which means your device is just idle.

So by looking the IOPS we can get an insight of what the throughput actually means, they both complement each other.

mFeinstein
  • 503
  • 2
  • 5
  • 10
  • IOPS = Inputs/Outputs Per Second, it is not about plural, and trailing S should not be omitted. :) – Eugene May 25 '17 at 13:41
  • 1
    It's not about plural, I have seen some people referring to IOP as a short for "I/O OPeration" as it sounds like... But yeah, this might lead to confusion, so I will replace it, thanks – mFeinstein May 25 '17 at 13:45