Getting around the FAT32 4GB file size limit

Question

I recently purchased a 32GB USB 3 stick that was formatted FAT32. I plugged it into my computer and attempted to copy a film onto it, the file was over 4GB however and it would not let me copy the file across because of the 4GB file size limit imposed by FAT32.

After some googling I found that I could format my USB stick to have an exFAT format which would mean I could put files onto the stick greater than 4GB in size and the drive would work on both my Mac and my PC.

The problem with this solution is that my PS3 cannot detect the USB stick when it is formatted with exFAT.

I would like to know if there is a way I can have my USB stick formatted so it can have files greater than 4 GB and work on my PC, Mac and PS3.

Answer 1

Unfortunately, there is no way to copy a >4GB file to a FAT32 file system. And a quick google says your PS3 will only recognize FAT32 file systems.

Your only option is to use smaller files. Maybe chop them into pieces before moving them or compress them.

I would try a networked solution to file sharing.

Answer 2

Natively, you cannot store files larger than 4 GB on a FAT file system. The 4 GB barrier is a hard limit of FAT: the file system uses a 32-bit field to store the file size in bytes, and 2^32 bytes = 4 GiB (actually, the real limit is 4 GiB minus one byte, or 4 294 967 295 bytes, because you can have files of zero length). So you cannot copy a file that is larger than 4 GiB to any plain-FAT volume. exFAT solves this by using a 64-bit field to store the file size but that doesn't really help you as it requires a reformat of the partition.

However, if you split the file into multiple files and recombine them later, that will allow you to transfer all of the data, just not as a single file (so you'll likely need to recombine the file before it is useful). For example, on Linux you can do something similar to:

$ truncate -s 6G my6gbfile
$ split --bytes=2GB --numeric-suffixes my6gbfile my6gbfile.part
$ ls
my6gbfile         my6gbfile.part00  my6gbfile.part01
my6gbfile.part02  my6gbfile.part03
$

Here, I use truncate to create a sparse file 6 GiB in size. (Just substitute your own.) Then, I split them into segments approximately 2 GB in size each; the last segment is smaller, but that does not present a problem in any situation I can come up with. You can also, instead of --bytes=2GB, use --number=4 if you wish to split the file into four equal-size chunks; the size of each chunk in that case would be 1 610 612 736 bytes or about 1.6 GiB.

To combine them, just use cat (concatenate):

$ cat my6gbfile.part* > my6gbfile.recombined

Confirm that the two are identical:

$ md5sum --binary my6gbfile my6gbfile.recombined
58cf638a733f919007b4287cf5396d0c *my6gbfile
58cf638a733f919007b4287cf5396d0c *my6gbfile.recombined
$

This can be used with any maximum file size limitation.

Many file archivers also support splitting the file into multi-part archive files; earlier this was used to fit large archives onto floppy disks, but these days it can just as well be used to overcome maximum file size limitations like these. File archivers also usually support a "store" or "no compression" mode which can be used if you know the contents of the file cannot be usefully further losslessly compressed, as is often the case with already compressed archives, movies, music and so on; when using such a mode, the compressed file simply acts as a container giving you the file-splitting ability, and the actual data is simply copied into the archive file, saving on processing time.

Answer 3

Expanding on Michael's idea, many compression utilities/formats support a "store" mode, where they don't actually do any compression. Most of those same utilities also support splitting into multiple archives. Combine the two, and you can split a file without wasting a bunch of time compressing it, especially if it's non-compressible data. I've used this technique myself to overcome the exact problem you're having.

One big advantage to doing it this way is that the compression format acts as a wrapper, keeping you from accidentally doing anything with only one part of the file. It also tends to be simpler for non-technical users. (Not everyone knows how to cat files, but almost everyone can open a zip.) It's also very obvious that it's a multipart file, since the file is formatted as such. Loose files may not look like a multipart file, especially if they lose their filenames somehow.

Of course, if you actually want to be able to work on the separate files, this doesn't work as well. This may be important if you don't have any "scratch space" to write the final file to. In that case, you should just split the file.

Here's an example of splitting a file using zip on Linux:

zip -0 -s 3g out.zip foobar

# "-0" sets the compression level to 0, or store
# "-s 3g" sets the split size to 3 GB
# Add "-r" if "foobar" is a directory
# The output will be "out.zip", and "out.z01", "out.z02", and so on...

If you're more of a GUI person, my goto has always been 7-Zip.

Answer 4

Another option not stated would be to use partitions. A USB flash drive is most often treated by the OS as a hard drive. Resize the FAT32 partition and make an exFAT (or other supporting filesystem) partition that is large enough to hold the file.

If you ever need to access the large file in place on the USB drive, this is probably the best solution. If all you need to do is transfer the file, and don't mind having to copy it to the hard drive to use it, the splitting solution is probably better.

This won't work if your USB drive is setup as a "super floppy," but this is increasingly uncommon. You can convert a "super floppy" into a hard drive format by using a partioning tool such as fdisk or gparted. But it will probably involve copying the files off, converting, copying them back, and then making the drive bootable again.

Answer 5

As answered by others splitting the file and joining works. But the easiest solution is to use ext 2/3/4 file system for your usb drive. It the native filesystem for linux. In windows use ext2fsd for reading the data. It also support write mode. Just install the free app on windows access file, no splitting , no joining.

Answer 6

Have you tried HFS+? It is the filesystem used on Macs. Don't laugh. It seems to work on the Xbox, so it's worth a try.

Get Past The 4GB File Limit On An External Hard Drive For The Xbox360

It might be implemented because it is also used on (some) iPods that are widely popular, so even M$ Xbox360 seems to read it. With appropriate software on PC, IF it works on the PS3, you'd get get a universal solution.

Answer 7

This works – follow my steps:

• Download WinRAR

• Right click to the file that you want to copy and select "Add to archive"

• On the bottom-left side of the popup box, write down split to volume size number, for example if your file is 4.63 GB you can write down 3 GB in the box, and click OK.

• Wait for it to finish

• Now copy all the files to USB

• Now select the first file "XXX.part1", right click and click "Extract here"

• It should extract your file.

Answer 8

This works for me

1. If it Iso or Compressed File you should just copy paste the thing inside it (I mean like, your iso file sized 5Gb, when you mount it, theres 6 file inside, you just need to copy the six file inside it, not the iso file)

If it rar, Descompress it then copy it

Rule for this Methode, Every file inside is below 4gb

2. If it above 4gb but its not an iso file and not a compressed file, Compress it into some part. Rule for this methode: Every part is below 4gb

3.Convert Fat32 system into Ntfs

Answer 9

In case this hasn't been said/discovered yet - using Filezilla to FTP the large files to the external HD also works even if the HD is Fat32.

Answer 10

You could use FFMPEG to split the file into parts that are playable, without losing data.

Example: 8GB video is 1 hour long, so I would split in into 3 (It could be 2, be to be on the safe side use 3); So I'd use the command

ffmpeg -i original-video.mp4 -ss 00:00:00 -t 00:20:00 -codec copy Part01.mp4
ffmpeg -i original-video.mp4 -ss 00:20:00 -t 00:20:00 -codec copy Part02.mp4
ffmpeg -i original-video.mp4 -ss 00:40:00 -t 00:20:00 -codec copy Part03.mp4

ffmpeg terms: -i : input file -ss : start time -t : length -codec : codec to use copy : 1:1 copy of the source

Here you can still use it with your PS3, and on a computer you can combine the mp4's with ffmpeg into one file

Answer 11

I needed to move a 15.5gb virtual environment. I removed the partition on the usb key (32GB) and made a new one that was extn4 (linux), mounted it as 32GBkey moved the file. Booted the machine that needed the file using knoppix live (linux disc), mounted the key, moved the file.

Then umounted the key, changed the partition back to vfat.

Answer 12

I have found that using Adobe Bridge (if it is available to you) will bypass the 4GB limit on our servers. This can be done simply by "drag and drop" from the OS launcher into the bridge client window. Works for large (15gb+) video files where I work.

Answer 13

FAT32 has a hard limit for how every file is listed (32bit field for file size). However, applications that bypass this limit use more fields. In theory, this can bypass the hard limit, but requires placement of the next fields immediately following the first, and linking the end of the first to the beginning of the next as if it were part of the file (it reads that end sector, and gets another file bitfield message from the second field) which is usually handled by PC operating systems alright, but not always.

Other programs move files in chunks of 600mb to 2gb and use a flag bit, then use multiple bitfields for the chunks, though all is seen as a single file; such is the case with adobe software, especially with video files. It segments the files, and creates a bitfield for each segment for video\audio that is read by the os as a single file. These files are simply wrappers, a folder of sorts that wraps around a set of images, or samples that are quickly cached into memory for playback. The DVD standard VOB is another great example that shows where this comes from and why. Its cut into chunks of quickly cached files for playback on DVD players. They use the old UDF1 or joliet format, which borrows from HFS, but uses mostly FAT capable lists. The file size limit is roughly 2gb, mainly for caching speeds of DVD player set-top-boxes. Most often it is in 1\10th to 1\5th of that limit, and multiplexed audio and video chunks are cached together very quickly, for playback.

The answer to this problem isn't complex, but it isn't one size fits all. For the specific question, use a splitter program; WinRar, 7zip are great, and have a gui. It's a simple answer and works well. The OS (operating system) will set the file to use multiple fields, and multiple chunks, but it will be seen as one file after the merge. With many compressed video files (almost every format is a compressed format by one algorithm or another), the read speeds for FAT are fast enough to playback on most smart TV's, or minimal OS Console systems (PS3\4, XBOX, etc). Other files like ISOs etc, not as capable unless in DVD format for video, which is a heavily cached format anyway. For game ISOs, check ntfs, and a few other formats your computer can build on the USB drive, and then try it with no split actions.

Answer 14

As the OP said 'for a PS3', the answer has to relate solely to FAT32, as this is all you get with PS3. As for Linux users, the answer can be found in EX4 partition formatting, and of course, NTFS with Windoze. To move files from EX4 to NTFS via Samba is easy. If moving a file requires 'jumping through hoops', you really should consider 'am I formatting my partitions properly to handle the files I need to handle on a regular basis?' That said, it's also nice to have more than one computer around that can process what you need to, how you need to, too. (This, coming from someone that just formatted a USB 32gb dongle in FAT32, only to find 10 minutes later that I needed to store a 15gb .img file on it, and had to move everything off it, change the partition format, and move it all back onto it).