Make rsync use less bandwidth?

46

11

I use rsync to backup and synchronize network shares and my computer. I have this figured out.

The problem is, when using rsync to transfer files, it uses all the bandwidth it can. I want to cap it's speed, so that I can use my connection for other things.

Specifically, right now I am listening to Last.FM, but it keeps cutting out, as rsync is saturating local connection.

It sounds unlikely, but in this case my LAN really is as fast as my internet (10 megabit for both, yay university!). Either way, I have no extra bandwidth while rsync is running. Any ideas?

Oh, btw, specifically I am running Ubuntu 9.04.

Mike Cooper

Posted 2009-10-20T22:48:13.457

Reputation: 2 036

Answers

67

Use the --bwlimit=KBPS option to limit I/O bandwidth, KBytes per second

Also refer to the man page.

DaveParillo

Posted 2009-10-20T22:48:13.457

Reputation: 13 402

8after the fact, but wanted to point out that this option limits the average bandwidth. the first file is sent full blast and subsequent files are throttled to attempt to get down to the specified bandwidth value. to truly limit bandwidth, you would want to look into something like trickle – joshtronic – 2011-09-19T01:42:14.313

Perfect. I don't know why I missed that in the man page the first time around. – Mike Cooper – 2009-10-20T23:03:17.153

1@joshtronic I wondered if you noticed supervacuo's answer? I have now tested this and can vouch for the fact that "the first file is sent full blast" is not true except for the smallest files. – None – 2013-04-02T19:14:01.267

20

(This would ideally be a reply to joshtronic's comment)

--bwlimit=XX in fact has the opposite problem; the transfer is indeed a moving average — as Rsync Basics helpfully explains:

Due to the nature of rsync transfers, blocks of data are sent, then if rsync determines the transfer was too fast, it will wait before sending the next data block. The result is an average transfer rate equaling the specified limit.

It's not clear whether the average is taken across files, but in any case it is not true that

the first file is sent full blast and subsequent files are throttled to attempt to get down to the specified bandwidth value

In fact the first file will be throttled as long as it is large enough for the averaging to kick in (which means all but the smallest files).

You're right that trickle would be a better solution, but what I understand from the explanatory paper ("Trickle: A Userland Bandwidth Shaper for Unix-like Systems") is that trickle also works by delaying I/O based on a moving transfer average. I guess the hope in recommending it is that it uses a higher-frequency measurement to apply the average. I haven't been able to find any data online that confirms this to be the case (although the paper above does refer to rsync's code as "simple", suggesting the authors of trickle think theirs does a better job).

supervacuo

Posted 2009-10-20T22:48:13.457

Reputation: 531