12

Trying to update files on a server, with the rsync command:

rsync -ravq -e "ssh -o ConnectTimeout=2 -o ServerAliveInterval=2 -ServerAliveCountMax=2" --delete ./local_dir user@$SERVER:/dest_dir

corrupt packet errors keep getting thrown, specifically:

rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (11337 bytes received so far) [sender]
rsync error: unexplained error (code 255) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/io.c(605) [sender=3.0.9]

This is probably related to a ssh timeout, as it seems to happen with large(r) files. Also, I keep getting timeouts using WinSCP. This is happening only to me; several of the people I work with that use this server do not have the same problem.

Using rsync from a Cygwin terminal in Windows 7, against a Centos 6.3 server.

I am not sure what other information might be useful or how to obtain it. I will update the question or add comments as per any suggestions.

How should I solve this?

Thanks very much!

aculich
  • 3,520
  • 1
  • 25
  • 33
Juan Carlos Coto
  • 677
  • 2
  • 6
  • 13

5 Answers5

15

I'm not sure what might cause the corrupt packet problem that drops your connection, but you might find rsync's --partial or --partial-dir option helpful when transferring large files so that when you restart the transfer it will continue where the transfer left off instead of having to start over transferring the whole file again:

--partial-dir=.rsync-partial

So you can modify your original command like this:

rsync -rav --progress --partial -e "ssh -o ConnectTimeout=2 -o ServerAliveInterval=2 -ServerAliveCountMax=2" --delete ./local_dir user@$SERVER:/dest_dir

or

rsync -rav --progress --partial-dir=.rsync-partial -e "ssh -o ConnectTimeout=2 -o ServerAliveInterval=2 -ServerAliveCountMax=2" --delete ./local_dir user@$SERVER:/dest_dir

Note that for this example I removed the -q (--quiet) option and added the --progress option in the first example and --partial-dir=.rsync-partial in the second example.

The difference between --partial and --partial-dir=.rsync-partial is that the later creates a directory that keeps the partial files separate from the fully-transferred files if that is important to you on the receiving (server) side.

The rsync manpage will explain this in further detail, though I'll also point out an important security note from the manpage:

IMPORTANT: the --partial-dir should not be writable by other users or it is a security risk. E.g. AVOID "/tmp".

aculich
  • 3,520
  • 1
  • 25
  • 33
1

I just ran into the same problem. The underlying problem in my case was that the machine I was transferring to had ran out of disk space.

Gussoh
  • 111
  • 1
1

The corruption suggests a bad NIC or NIC driver in your machine; had this once on my wife's Windows box: had to repeatedly try to get rsync installed. Once that succeeded, I could repeatedly invoke rsync to transfer and fix the other software needed, which most notably included updated NIC drivers and a checksum tool.

Phil P
  • 3,040
  • 1
  • 15
  • 19
0

If ulimit value is 0 or a little number, it should be giving that error. Try to increase ulimit value (such as 9999999999) and try again. you can write a little shell script (a.sh) and use something like this:

#!/bin/sh
ulimit 99999999999

rsync -avrz --perms --delete --chmod=u+rwx,g+rx,o+x /dir1/ /dir2/
Bart De Vos
  • 17,761
  • 6
  • 62
  • 81
C.O.A.
  • 1
  • 1
  • 2
    That's interesting. I haven't been working on this for a while; however, if you'd like to explain why it works, I think it'd be really useful for whoever reads into this question. Thanks! – Juan Carlos Coto May 07 '15 at 21:24
-1

just loop until it transfers finally 

while ! sshpass -p 'xxxx' rsync --partial --append-verify --progress -a -e 'ssh -p 22' /source/ remoteuser@1.1.1.1:/dest/; do sleep 5;done
Toolkit
  • 149
  • 1
  • 2