6

Sometimes when I'm uploading a large number of small files via FTP, some of the files' contents will be replaced with FTP control messages. For example, after uploading a website I'll notice that an image isn't displaying. When I examine the image on the server, I find that its contents have been replaced with something like

Response:   125 Data connection already open; Transfer starting.
Response:   226 Transfer complete.
Status:     Directory listing successful

It's very difficult to reproduce. Any corrupted files that I notice, I can usually just retransfer them. If I retransfer the whole site, maybe different files will be corrupted, maybe none at all.

This problem has been plaguing me for several years, and in that time I've replaced my computer (twice), my router, my cable modem, and I've moved halfway accross the country. I haven't had the problem recently, but that's probably only because I've learned ways of avoiding it, like zipping everything up before transferring it. I use FileZilla, by the way.

I discribed the problem to one of my web hosts once. They'd never heard of anything like it, didn't have a clue. I would love to know what's been going on. As it is, I'm very leery about using FTP for anything more than a handful of files.

Metaphile
  • 161
  • 1
  • 1
  • 3
  • 1
    FTP is not a reliable protocol, but I'd suggest trying another client since you don't mention trying that. Or another server. The file being replaced sounds like a nasty bug. – gparent May 08 '12 at 16:24
  • Do you have the problem across multiple web hosts? – Jeremy May 08 '12 at 16:38
  • @Jeremy I'm honestly not sure because the problem is so sporadic and difficult to detect, but I _think_ I've had the problem with at least two different hosts: a small company in Ohio and HostGator. – Metaphile May 08 '12 at 16:59
  • Can you be more specific about the "large number"? Is it greater than, say, 32k or 64k? – Gerald Combs May 08 '12 at 17:10
  • Are you sure your transferring files in binary mode and not ASCII? – jftuga May 08 '12 at 17:14
  • @GeraldCombs I would guess between several hundred and several thousand files. I don't think I've ever transferred as many as 32,000 files in one go. – Metaphile May 08 '12 at 17:28
  • I've tried transferring files in auto-detect and force binary modes. And I've tried enabling/disabling simultaneous transfers. I _really_ thought disabling simultaneous transfers would do the trick but ... nope. – Metaphile May 08 '12 at 17:31
  • 1
    As you've asked for theories, rather than technical explanations, I'm going to go with the possibility that aliens have intercepted the transmission and modified it as a first step to breaking our communications systems. – John Gardeniers May 08 '12 at 23:51

5 Answers5

10

Very simply: the universe (and/or Murphy) is punishing you for continuing to use a thoroughly obsolete and insecure protocol like FTP.

When you connect to your web server using FTP, your username and password are sent unencrypted, in the clear, for everyone and Big Brother to read. So are the contents of all the files that you upload or download using FTP. For security reasons alone, I would strongly suggest switching to SFTP, which is a built-in feature of SSH. Any web hosting provider worth your business supports SSH and SFTP.

Any FTP problems that you may be experiencing on the server side almost certainly will not apply when using SFTP, because SFTP connections are handled by a completely separate daemon/service on the server.

Unlike FTPS, FTPES, etc., SSH/SFTP implementations are very consistently interoperable. You are unlikely to experience strange and unpredictable problems when transferring files via SFTP.

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
  • You're so right. :D – Metaphile May 08 '12 at 17:47
  • I completely agree with your answer, but I think it is a little misleading to start off by blaming "unencrypted FTP", as the issue isn't the lack of encryption, but the fact that FTP as a protocol is very flaky. No downvote or anything, but it might be worth revising as many people don't read beyond the first line :) – Jeremy May 08 '12 at 21:57
  • @Jeremy Updated for clarity. – Skyhawk May 08 '12 at 22:06
1

My money is on a FTP server bug (your host's problem). Make sure you are using the latest version of FileZilla, and configure it to log all your FTP sessions so that if you discover this corruption, you can provide the host with a log of the session. It might help them troubleshoot.

If you have shell access to your host, you could use SCP instead.

Jeremy
  • 938
  • 2
  • 7
  • 18
1

just to be on safe side, I'd make sure transfer set to "binary" sometimes it transfer files in ASCII mode and that's where you usually get so to speak corruption, so try that if that doesn't solve your problem then I'd look into replacing/updating your server & client and if that doesn't help either then look on interface if you're getting errors there...

alexus
  • 12,342
  • 27
  • 115
  • 173
1

for me this was because of the transfer mode set to ASCII in filezilla, i set Transfer -> Transfer type -> Auto, (to ensure it uses binary mode when required)

this solved my issue. Hope this helps you guys as well.

user202834
  • 11
  • 2
0

Potential root of your issue (and many other similar issues with corrupted files):

invalid chunks due to occurrence of duplicated data in file (eg. lots of zeros)

and/or something else:

- network configuration related (MTU size?!)

- outdated server software (few years ago in Debian was bug related to this issue)

- not using binary mode

Workarounds:

  1. As mentioned by others, switch to SFTP - encryption will minimalize occurance of duplicated packets/chunks

  2. Enable compression in your client - as in option 1, compression will do similar thing

FileZilla ticket (6 years old): https://trac.filezilla-project.org/ticket/4672

meloniq
  • 1
  • 1