bs
, the buffer size, means the size of a single read() call done by dd.
(For example, both bs=1M count=1
and bs=1k count=1k
will result in a 1 MiB file, but the first version will do it in a single step, while the second will do it in 1024 small chunks.)
Regular files can be read at nearly any buffer size (as long as that buffer fits in RAM), but devices and "virtual" files often work very close to the individual calls and have some arbitrary restriction of how much data they'll produce per read() call.
For /dev/urandom
, this limit is defined in urandom_read() in drivers/char/random.c:
#define ENTROPY_SHIFT 3
static ssize_t
urandom_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
{
nbytes = min_t(size_t, nbytes, INT_MAX >> (ENTROPY_SHIFT + 3));
...
}
This means that every time the function is called, it will clamp the requested size to 33554431 bytes.
By default, unlike most other tools, dd will not retry after receiving less data than requested – you get the 32 MiB and that's it. (To make it retry automatically, as in Kamil's answer, you'll need to specify iflag=fullblock
.)
Note also that "the size of a single read()" means that the whole buffer must fit in memory at once, so massive block sizes also correspond to massive memory usage by dd.
And it's all pointless because you usually won't gain any performance when going above ~16–32 MiB blocks – syscalls aren't the slow part here, the random number generator is.
So for simplicity, just use head -c 1G /dev/urandom > output
.
3IMHO I don't think there are many valid use cases for
dd
at all. I'd usehead
,cat
orrsync
in its place almost always. And your question if one of the reasons why the alternatives are usually safer. – Bakuriu – 2018-12-28T17:13:05.860@Bakuriu - also, if you just want to produce a file full of zeroes (or rather you do not care about what is inside it) use truncate. It is much faster. – Konrad Gajewski – 2018-12-29T09:09:00.817
@KonradGajewski FYI truncate tries to make a sparse file (if that matters) – Xen2050 – 2018-12-29T20:33:07.013
5
@Bakuriu
– Kaz – 2018-12-31T01:09:06.823head
cannot do this task without the-c
option that isn't in POSIX. I don't know any version ofcat
which can solve this.rsync
is a completely nonstandard utility. That is neither here nr there; skimming through its man page, I don't see how it can solve this problem, either.Technically,
/dev/urandom
isn't in POSIX either... – user1686 – 2018-12-31T09:47:29.200@Xen2050 it might. Thx. – Konrad Gajewski – 2018-12-31T11:21:12.287
It's the
bs=1G
part that makes it use a lot of memory. You only didn't notice it with /dev/urandom, because of the reasons already explained. – user1686 – 2018-12-31T18:53:53.867@grawity that's exactly what I meant in my edit. – Trismegistos – 2018-12-31T19:03:47.310
@Kaz Those were only a couple of example. There is no
dd
task that cannot be done better with an other tool. Also,rsync
isn't POSIX but if you aren't using it you are a moron. Using only POSIX tool transferring files reliably and efficiently between systems cannot be done. – Bakuriu – 2019-01-02T08:39:34.673@Bakuriu Those were a couple of examples indeed: of tools that cannot do most of the things
dd
does. I believe there aredd
tasks that cannot be done more portably than withdd
. The command argument language ofdd
is pretty excellent: very easy to remember, and the semantics is transparent. – Kaz – 2019-01-02T21:23:48.313Related: What’s the difference between dd’s command options bs=1024 count=1 and bs=1 count=1024?
– Scott – 2019-01-23T06:03:23.883