Duplicity backup to Box stalling

Question

We have been using duplicity to backup our server to a box.com account for some time. The commands being run are:

duplicity --full-if-older-than 1M -v4 --exclude *A number of files* / webdavs://*box email*@dav.box.com/dav/*Path to backup folder*  --no-encryption --log-file *Todays date*.log

Followed by:

duplicity remove-older-than 6M --force webdavs://*box email*@dav.box.com/dav/*Path to backup folder* --no-encryption --log-file *Todays date*.log

The password is set in an environmental variable.

Up until recentrly the log used to look something like this:

WARNING 1
. Attempt 1 failed. SSLError: The read operation timed out

NOTICE 1
. Local and Remote metadata are synchronized, no sync needed.

NOTICE 1
. Last full backup date: Fri Aug 26 01:00:01 2016

NOTICE 1
. --------------[ Backup Statistics ]--------------
. StartTime 1472338925.59 (Sun Aug 28 01:02:05 2016)
. EndTime 1472339008.64 (Sun Aug 28 01:03:28 2016)
. ElapsedTime 83.05 (1 minute 23.05 seconds)
. SourceFiles 63777
. SourceFileSize 61857468557 (57.6 GB)
. NewFiles 24
. NewFileSize 69559829 (66.3 MB)
. DeletedFiles 6
. ChangedFiles 23
. ChangedFileSize 7702407 (7.35 MB)
. ChangedDeltaSize 0 (0 bytes)
. DeltaEntries 53
. RawDeltaSize 92935471 (88.6 MB)
. TotalDestinationSizeChange 92982066 (88.7 MB)
. Errors 0
. -------------------------------------------------
. 

WARNING 1
. Attempt 1 failed. SSLError: The read operation timed out

NOTICE 1
. Local and Remote metadata are synchronized, no sync needed.

NOTICE 1
. Last full backup date: none

NOTICE 1
. No old backup sets found, nothing deleted.

The first read opperation allways timed out but after that everything got through.

Latelly however the log started looking like this instead:

WARNING 1
. Attempt 1 failed. SSLError: The read operation timed out

WARNING 1
. Attempt 2 failed. SSLError: The read operation timed out

WARNING 1
. Attempt 3 failed. SSLError: The read operation timed out

WARNING 1
. Attempt 4 failed. SSLError: The read operation timed out

ERROR 50
. Giving up after 5 attempts. SSLError: The read operation timed out

WARNING 1
. Attempt 1 failed. SSLError: The read operation timed out

NOTICE 1
. Local and Remote metadata are synchronized, no sync needed.

NOTICE 1
. Last full backup date: none

NOTICE 1
. No old backup sets found, nothing deleted.

This is the result when the cronjob tries to run it. I've been experimenting with it manually and observed the following things:

• I'm able to extract a list of files from the box account by running duplicity list-current-files This indicates that the login credentials are correct and that the connection to the server is established.

• If i run the backup on a smaller folder and ask duplicity for the progress it outputs:

  Local and Remote metadata are synchronized, no sync needed Last full backup left a partial set, restarting. Last full backup date: Fri Sep 30 09:12:01 2016 RESTART: The first volume failed to upload before termination. Restart is impossible...starting backup from beginning. Local and Remote metadata are synchronized, no sync needed. Last full backup date: none No signatures found, switching to full backup. 0.0KB 00:00:03 [0.0KB/s] [> ] 0% ETA 3sec 0.0KB 00:00:06 [0.0KB/s] [> ] 0% ETA 6sec 0.0KB 00:00:09 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:12 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:15 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:18 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:21 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:24 [0.0B/s] [> ] 0% ETA Stalled! 0.0KB 00:00:27 [0.0B/s] [> ] 0% ETA Stalled!

• The behaviour is the same regardless of if I run on the account used for the backups, which is a Business account, or if i run on a newly setup free account.

So duplicity seems to have stalled but I get no error message as on why it has stalled or what I could do to fix this. Im also unsure on if this actually relates to the SSL error and ERROR 50 or if we have some kind of "double fault". We are running duplicity 0.7.10. We also recently updated our server to Ubuntu Server 16.04.1 LTS but the update does not seem to coincide exactly with the problems starting. Has anyone else encountered this or something similar? Any tips on how to solve it?

I tried using FTP instead of WebDav to connect to box but i get the same result, the connection is established but it stalls completely.

This might be a clue to the stalling: How to go through this duplicity stalled connection? It does not however seem to solve the SSL Error: The read operation timed out

The connection is established but then it stalls completely.
Has anyone else encountered this or something similar?
Any tips on how to solve it?

Answer 1

I solved it now by switching to ftp instead of webdav. This seemed to solve the SSLError. I don't know if the problem is on the duplicity or box end.

The other problem, duplicity reporting that the connection stalled, seems to be a bug, see: How to go through this duplicity stalled connection?

All in all the backups are now running as normal!