Is there an FTP or FTP-like client which allows "delta uploads" - i.e. only uploading the parts of a file which have been changed?

9

2

I frequently make small changes to fairly large text files via FTP (CSS files for web development) and because my internet connection is terribly slow, it's a very painful process.

Even if I change only one character of the file, I still have to sit and wait for considerable time while the entire file gets re-uploaded to the server.

Is there some way I can edit remotely-hosted files on my local computer and when I save the file, only upload the parts of the file which changed?

Edit: I think I've been unclear about how I'm using my FTP client at the moment. Basically, I open a remote file over SFTP using WinSCP. WinSCP saves it to a tmp folder, launches Sublime Text and I start editing it. When I hit save, WinSCP re-uploads the file automatically. I'm never actually saving the file permanently on my computer - everything's done via WinSCP's temp folder. That's why I'm looking for a solution which can be used in this "hit and run" type manner.


Previous Discussion

I previously posted this question on ServerFault where using something like git was suggested, however as I explained:

With git or a similar versioning tool I'd have to make a commit each time I save the file, right? Even if that could be done automatically (i.e. if I had a macro which made Ctrl-S in Sublime Text save file and execute a git push at the same time), I might make 50 - 100 tiny edits of a CSS file per hour; surely I'd end up with a spammed-out commit log? Also the git solution is inflexible as it relies on whatever file I'm editing being part of a git repo.

The only solution I can think of so far is some kind of hacked-up process using rsync but even then, it's very fiddly:

  1. I download the file(s) I need to my computer (via FTP/rsync/whatever).
  2. I open a file in (e.g.) Sublime Text and start making edits.
  3. A background process is monitoring the folder for file modifications. When I save a file, that process instantly triggers an rsync operation from my computer to the server, transferring the changed parts of the file.

However, I'd need to manually set up or specify the remote file or directory paths or else the local rsync script won't know where to rsync the files so this method is still pretty inflexible. This method wouldn't let me instantly browse to a random file, open it and save it like an FTP client does.

There's also no visible feedback like an upload progress bar with this method.

WackGet

Posted 2016-10-12T14:05:01.107

Reputation: 366

7Using git really is the right answer. What you're looking for here is a workaround for poor practice. What if you do something wrong and break things? Git gives you rollbacks. – TRiG – 2016-10-12T14:26:24.957

1I disagree. What if you simply want to change a single line of a large text file which isn't part of a repo? What if your organisation doesn't allow or support git? My use-case is that I'm making hundreds of tiny changes to files which are remotely hosted and which must be edited "live" and "in place". – WackGet – 2016-10-12T14:30:38.163

I don't have time to test on a large file right now, so posting this as a comment, not an answer, but look into Beyond Compare - it's a cross-platform GUI diff file that can edit over FTP/SFTP and you can choose which lines to sync across. Probably still not ideal, but workable. They have a free trial too. – Nick – 2016-10-13T17:44:03.783

Answers

7

Vim has a built-in plugin called netrw, which allows Vim to edit remote files. Rsync is one of the protocols it can use:

:e rsync://user@hostname:port/path
:w rsync://user@hostname:port/path

Of course, if you have SSH access to the remote machine, and Vim is installed on it, it may be simpler to just use Vim remotely. Nano, Emacs, and other editors with command line interfaces are also options in that case. Depending on just how bad your connection is, though, this might not be a pleasant option.


That all said,

making hundreds of tiny changes to files which are remotely hosted and which must be edited "live" and "in place"

without a reliable rollback system such as what git can provide is you and your organization just begging for agony when you make the inevitable mistakes which may very well go unnoticed until you've forgotten exactly what you did.

8bittree

Posted 2016-10-12T14:05:01.107

Reputation: 2 595

SSH seems to be the obvious answer if the server is running any version of Linux. If it's not already running on the server, I'd be pressing the server admin to set that up. – Charles Burge – 2016-10-12T17:31:08.890

Thank you both; I do have full SSH access and can configure whatever's necessary on the server. I can't use Vim remotely because the latency is often very high. Also I'd like to be able to use desktop editors like Sublime. I think I've been unclear about how I'm using my FTP client at the moment -- basically, I open a remote file using WinSCP, start editing it, and when I hit save, WinSCP re-uploads the file automatically. I'm not actually downloading it to any permanent directory on my computer - everything's done via WinSCP's temp folder. I'll edit my question and clarify. – WackGet – 2016-10-12T18:25:52.873

@WackGet Vim does work on the desktop, too, it's called GVim. Cream is a particular configuration of Vim that's supposed to be relatively easy for those more used to editors like Sublime, Notepad, or Textmate. What you're doing with WinSCP is essentially how netrw works. It downloads to a temporary location, waits for you to save, then writes it back to the remote server, using whichever protocol you specified.

– 8bittree – 2016-10-12T18:33:34.380

@8bittree Thanks, this will certainly come in useful. I won't mark this one as the answer to my original question because it limits you to using the Vim-based plugin. If someone could implement netrw-like delta transfers into an FTP or FTP-like client, then that would be perfect. Currently I haven't found this yet but I'm hopeful it exists. – WackGet – 2016-10-12T22:40:12.013

1

Beyond Compare is a diff tool that can compare and merge folders and files; FTP servers are supported too and can be handled like a local folder. It is proprietary, a lifetime license for one install costs 30 USD for the standard edition (with FTP) or 60 USD for the professional edition (with SFTP). It runs on Windows, Mac OS and Linux. Personally I develop my website on my home computer and upload the changes to my Web server when I am happy. I go through the changes by hand with Beyond Compare but this can be automated too. You could try it and see if it uploads only complete files or chunks of changes; I am not sure about that.

Stéphane Tréboux

Posted 2016-10-12T14:05:01.107

Reputation: 223

Beyond compare is awesome - perfect if you don't have SSH access, but you still need to make sure files are synced on a remote server (FTP over SSL is supported) – Sam Heuck – 2017-02-26T22:31:21.643