1

I have been having issues with various rsync operations over SSH, on a particular machine on our network, failing and giving errors like:

rsync: connection unexpectedly closed (45482 bytes received so far)  [generator]

I found a similar post (rsync - unexplained error) for which one of the answers suggested running the SSH command used by rsync in verbose mode and redirecting the output, in my case as follows:

rsync -av -e 'bash -x -c "ssh -p 22 -vvvv $0 $@ 2>/tmp/rsync-ssh.stderr | tee /tmp/rsync.stdout"' --rsync-path='sudo rsync' "backup_user@10.0.0.2:/media/remote_volume/" "/media/local_volume"

This rsync command seems to run fine for a while but eventually fails with the error:

tee: standard output: Resource temporarily unavailable

I am guessing it is likely failing for a similar reason as is described in the answer to the following question: Why is this tee losing stdout? However, it doesn't seem clear to me how I should change the SSH command in my case to remidy the issue.

Does anyone know way of making SSH play nice with tee in this scenario?

PicoutputCls
  • 243
  • 4
  • 15

1 Answers1

0

Having done a bit more reading on the solution discussed in the answer to the question "Why is this tee losing stdout?", it seems like the crucial change is replacing the pipe with redirection to a subshell containing the command that would otherwise be being piped into. (Thus, replacing: command1 | command2 with: command1 > >( command2 ))

The reason for this is that by redirecting to a subshell, rather than directly piping into the next command, one is forcing the subshell to handle any EAGAIN error codes returned. As discussed in this FreeBSD message board message: bin/164947: tee looses data when writing to non-blocking file descriptors (also linked in the answer to the other question) tee and other system binaries can be quite bad at handling retries themselves.

For the rsync command above, the following alterations seem to fix my original issue:

rsync -av -e 'bash -x -c "ssh -p 22 -vvvv $0 $@ 2>/tmp/rsync-ssh.stderr > >( tee /tmp/rsync.stdout )"' --rsync-path='sudo rsync' "backup_user@10.0.0.2:/media/remote_volume/" "/media/local_volume"
PicoutputCls
  • 243
  • 4
  • 15