5
I'm experiencing very difficult to diagnose problem, which causes file corruption on files I'm uploading. Corruption happens to many files which contain continuous hexadecimal pattern (like thousands of "FF FF FF").
Please have a look at the comparison of the same file in Hex view. The file I was trying to send is on the left and the same file after sending it via FTP is on the right:
As you can see, the file on the right has "00 00 00" instead of "FF FF FF".
In attempt to diagnose the cause I did the following:
- I made sure all my hardware has the latest firmware
- I tested my PCs against latest viruses
- I've been restarting router and modem several times
- I reset the router to factory settings
- Enabled and disabled firewall
- I tried various FTP clients
- I tried sending in Binary and ASCII modes, with and without MODE Z compression
- I tried various FTP servers
- I tried all of my machines (Win PC, Win laptop, Mac laptop even my android smartphone)
- I own NAS, so I asked my friends to download the files from it
- I excluded switch from my network
- I replaced the router
- I replaced the modem
- I replaced all Ethernet cables
None of the above helped apart resetting the modem. Before I reset the modem the corruption ratio was 99% percent – literally every time I uploaded 100 files 98-99 of them were corrupted. When it came to the step "restart the modem", the ratio dropped to 10%, so I thought it must be the modem and bought a new one, but after installing the new modem ratio of corrupted files remained at 10%.
I also created a hotspot on my mobile and connected my network to it, I could successfully send files without any corruption.
When I'm sending files which don't have the continuous hex pattern, the problem doesn't exist.
I'm on VDSL (BT Infinity), the Internet provider won't help, I think I exhausted all the options.
I'd appreciate any advise as I'm desperate for help.
zee
Posted 2015-07-10T10:57:43.090
Reputation: 51
1Interesting issue... perhaps the FF pattern is treated as EOF, though in BIN that is not supposed to happen. Have you tried sending the files with Zip or other compression (compressing the file, not transmission)? Compression tries to eliminate repeated blocks by sending the block and a count of repetitions. Hope this workaround helps. – DrMoishe Pippik – 2015-07-10T16:39:34.997
1Hi, thank you for the suggestion, yes I tried, and sending compressed files is ok - I'm using it now as a workaround, but this is causing me a lot of inconvenience, so I'm desperate to find a reason and fix it. – zee – 2015-07-11T13:57:22.847
You need to look into the exact protocol used in the transmission, down to frame level. Groups of 'FF's can be interpreted as something special in some cases. I find it quite unusual for a file to contain that many 'FF'/255s. What exactly is that ? – Overmind – 2017-06-08T09:13:05.920
Hi @Overmind, thank you for your comment.
Believe it or not, I still can't solve the issue. BT (my ISP) won't help me. It takes weeks to get this issue through to 2nd technical level, where after weeks of investigations, they come up with nothing and drop the case.
These are TIFF files where FF FF FF FF is a white pixel with 100% opacity, so any image with white background will have loads of them.
I'm pretty sure that this is a line fault, as since the original post, it got worse and worse. Now even regular jpegs (so files without FF's very often are going out as corrupt files). – zee – 2017-06-10T05:53:58.133
Binary mode suppose to x-fer as-is, without regard to content. Best thing I can think of at this moment is for you to use a lower MTU size. That has a chance of fixing this. Also, can you try SFTP/FTPS instead ? How about Stream mode ? I'll put those into an answer. – Overmind – 2017-06-12T08:48:13.273