9

I'm wondering which file transfer protocols available on Linux support server-side copying (and doesn't internally download and reupload files). The simplest use case I have is for copying large files (such as VMs) from one directory to another on the same share.

I know Samba 4.1.0 recently implemented support for FSCTL_SRV_COPYCHUNK in the SMB2 protocol, but as far as I know, no client-side tool/file manager supports this yet (link). I've also looked at AFP, but the only two Linux tools for mounting AFP shares are gvfs-afp and afpfs-ng and neither are too great. afpfs-ng is unmaintained and doesn't cooperate well with netatalk (segfaults), while gvfs-afp has very low throughput compared to afpfs-ng when it works. Update: it looks like the next version of NFS might support this too.

Are there any other protocols that support server-side copying?

Thanks in advance!

Andrew Gunnerson
  • 193
  • 1
  • 4
  • 1
    Do you need this rarely or often? Used by users of operators? If rarely or operators, I'd just ssh to the machine – miniBill Jan 12 '14 at 14:13

2 Answers2

6

As mentioned by Ross Lagerwall, Samba 4.1+ SMB2 server-side copy support can be utilized by the Linux CIFS kernel client (cifs.ko) via the CIFS_IOC_COPYCHUNK_FILE ioctl.

The Cloner utility shipped with fstests can be used to issue the ioctl on cifs.ko SMB2 mount points. Work on smbclient support is also in progress.

One benefit of the Samba server-side copy implementation is that it can be used atop a Btrfs filesystem to perform near-instantaneous copy / deduplication of file data, as demonstrated in this performance comparison.

nico.ruti
  • 103
  • 4
David Disseldorp
  • 61
  • 1
  • 1
5

As you noted, AFP supports server side copies. The latest unreleased version of gvfs contains a number of improvements to gvfsd-afp to make it more reliable and faster. In my tests, it is able to get about 70MB/s for a large transfer on 1GBE and is able to duplicate a directory of 10000 files in 11 seconds which is not too bad. If server side copies are important and you don't want to be logged into a terminal, this is what I'd recommend to use.

We also plan on adding server side copy support for gvfsd-smb soon, but who knows when "soon" may be...

(Disclaimer, I help maintain gvfs.)

Ross Lagerwall
  • 66
  • 1
  • 3
  • Many thanks! I'll compile gvfs from git and give it a try. By the way, do you know if Samba 4 supports server-side copy yet or does it require a Windows server? – Andrew Gunnerson Aug 10 '14 at 18:43
  • I just compiled gvfs from git master (`232d6d76029dc1cbc0c76c2459f9db26b7717d28`) and AFP is working beautifully now. Server-side copies work and the throughput is on par with gvfs-smb and sftp (over 802.11ac). Thanks again! – Andrew Gunnerson Aug 10 '14 at 19:47
  • Awesome :-) Yeah, the Samba 4.1 _server_ does support server-side copy afaik. The Samba client tools don't. I think support has been added to the cifs kernel module to do a server-side copy but a special ioctl needs to be used to make use of it which most programs don't make use of. – Ross Lagerwall Aug 12 '14 at 20:17