How to append data in a file by dd?

Question

I want to append new data in a file stored in SSD.

dd if=/dev/shm/test of=/data/sdb/test bs=1G oflag=append

But df -h shows the dd command always overwrite the test file, instead appends new data in the test file. I also tried

dd if=/dev/shm/test of=/data/sdb/test bs=1G conv=notrunc

It does not work, either.

Answer 1

What about:

 dd if=/dev/shm/test bs=1G >>/data/sdb/test

Answer 2

dd if=/dev/shm/test of=/data/sdb/test bs=1G oflag=append conv=notrunc 

That is what I think you should have used.

REF : https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=373736

Answer 3

In Linux kernel 4.1 FALLOC_FL_INSERT_RANGE option was added. From fallocate(2) man page:

Specifying the FALLOC_FL_INSERT_RANGE flag (available since Linux 4.1) in mode increases the file space by inserting a hole within the file size without overwriting any existing data. The hole will start at offset and continue for len bytes. When inserting the hole inside file, the contents of the file starting at offset will be shifted upward (i.e., to a higher file offset) by len bytes. Inserting a hole inside a file increases the file size by len bytes.

And recently this option support was added to util-linux:

   -i, --insert-range
          Insert a hole of length bytes from offset, shifting existing
          data.

So when util-linux version 2.30 will be released and your linux distro will update to this version we will be able to increase file size in a flash by running:

fallocate -i -l 1G -o 128M /path/to/file

where 128M is the current file size.

Answer 4

There is an easier way to append a sparse hole to a file.

truncate is much faster than dd. To grow the file with 10 bytes use:

 truncate -s +10 file.txt 

answer found in: https://serverfault.com/a/343726/70242