11

When I'm running an rsync backup job which involves copying over large files, the machine running the backup (both Mac desktops and Linux servers) grinds to a halt and the load average goes through the roof.

I've tried:

  • niceing the rsync process (doesn't help - the bottleneck is the disk)
  • On Linux, reniceing a kjournald (helps, but seems like a hack and doesn't work on the Mac)
  • Using the --bwlimit rsync flag (helps, but it means all the transfers are slow - even when they don't need to be)

So, is there any way I can "nice" rsync's IO so the machines are useable while the backup is running?

PS: I'm aware of the dangers of rsync on the Mac… But I've used BackupBouncer to verify my backups, though, and they seem OK.

David Wolever
  • 2,237
  • 3
  • 23
  • 27

2 Answers2

7

It looks like the setpriority API on Mac OS X is supposed to be able to alter IO scheduling (see http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/setpriority.2.html). I don't have any MacOS handy so I can't test that nice actually changes the IO priority.

On the Linux side, ionice is what you're looking for.

Kyle Brandt
  • 82,107
  • 71
  • 302
  • 444
Evan Anderson
  • 141,071
  • 19
  • 191
  • 328
  • 1
    Ah, thanks. Here's what the `setpriority` manpage says: "When setting a thread into background state the scheduling priority is set to lowest value, disk and network IO are throttled." -- which seems to indicate that disk/network IO are only throttled when something is niced down to 20 (the "background state"). – David Wolever Jan 06 '10 at 17:21
  • 2
    Some quick testing also seems to confirm this - when the backup script was only niced to 10 (ie, `nice backup`), I noticed a slowdown... But when it's niced down to 20 (ie, `nice -n 20 backup`), I don't notice any slowdown. – David Wolever Jan 06 '10 at 17:22
  • (and by "noticed a slowdown" I mean "noticed that interactive programs running on the computer didn't respond as quickly as they normally do") – David Wolever Jan 06 '10 at 17:23
  • 1
    It's a little disappointing that you only get throttling at a priority of 20, though. – Evan Anderson Jan 06 '10 at 18:09
  • +1, Nice answer. Always wondered about twiddling IO. – Avery Payne Jan 06 '10 at 23:12
  • The background state is only set when you do a setpriority with PRIO_DARWIN_BG (0x1000), which you cannot set with the CLI nice. – lhunath Dec 02 '13 at 12:35
  • Is there one for network IO? Or does this work for network IO aswell? – CMCDragonkai Jul 21 '14 at 08:49
  • `renice -n 20 -p PID` doesn't seem to result in much better performance for me, as in... if I SIGSTOP the process using disk IO my other requests are *much* faster. – Michael Apr 24 '19 at 03:31
4

On linux you can use ionice http://linux.die.net/man/1/ionice

Amandasaurus
  • 30,211
  • 62
  • 184
  • 246