0
The objective is to use rsync to synchronize a daily single .gz backup file. The command is:
rsync -auvx /backup/source/*.gz /save/dest/
The files are on two separate systems; the source is mounted via NFS. The file is about 10GB. Rsync is spawning 181 threads and the sync is taking over 5 hours to complete. (A simple copy runs in under an hour.) The multiple threads cause significant system overhead.
Is there a way to force rsync to not spawn multiple threads?
....
dansawyer
Posted 2018-11-29T17:47:16.837
Reputation: 1
How many files are sitting in
/backup/source? How many in/save/dest/? Yourrsynccommand has to read even files that are already synched to make sure they are synched. Are your "last modified" dates accurate on both sides? – David Schwartz – 2018-11-29T17:51:44.047Only the one large daily file is being copied. – dansawyer – 2018-11-29T18:28:02.840
Clarification: There are about 10 dailies at a time in the source path and 9 in the destination. The update is run daily via crontab and only 1 file in the source has changed, the newly created daily backup. – dansawyer – 2018-11-29T20:02:14.147
The problem is likely all the work being done to decide that only that one file needs to be copied. Test with empty directories (but for the file you want to transfer) to see if this theory is correct. – David Schwartz – 2018-11-29T20:57:38.917