32

(This is a follow-up to "What is the partition id / filesystem type for UDF?")

I know two ways to format a hard drive as UDF:

  • Windows Vista or later: "format x: /fs:UDF" (don't use /q ! )
  • Linux: "mkudffs --media-type=hd --blocksize=512 /dev/sdx"

The problem is that the 'other' OS does not recognize the disk as formatted at all: it simply refuses to mount it, no matter what commands I try.

How can I format a hard drive as UDF so that both Windows and Linux will be able to use it?

EDIT: updated the commands, now the result should work in either OS.

Community
  • 1
skolima
  • 1,253
  • 3
  • 16
  • 28
  • 1
    The command you give works great on Linux. I formatted on Karmic, and can read and write under both Windows 7 and Karmic. – Matt Joiner Dec 29 '09 at 13:44
  • 5
    Do not forget to zero the MBR first (dd if=/dev/zero of=/dev/sdx bs=512 count=1), to avoid confusion with any leftover partition table (since UDF does not use the first sector). – CesarB Jan 02 '10 at 02:59
  • 3
    Related question: http://superuser.com/questions/39942/using-udf-on-a-usb-flash-drive – CesarB Jan 02 '10 at 03:03
  • 1
    Thanks Skolima, this is great. Bye bye FAT32. I only need to be careful to stick to UDF version 2.01 and to 512 block size and I have the perfect hard disk partition shared across Linux and Windows. – MarcH Dec 29 '11 at 23:46
  • for some strange reason you also need to create the partition from Windows. Not necessarily format it there but create it. – MarcH Dec 30 '11 at 01:05
  • Interesting technical analysis http://sipa.ulyssis.org/2010/02/filesystems-for-portable-disks/ – MarcH May 15 '12 at 09:30
  • What UDF version will that Windows formatted partition have? Is there a way to choose? – DanMan Sep 08 '13 at 12:14
  • Don't use UDF. I would have chosen either FAT32 or NTFS file system, which is visible from both linux and windows. – djangofan Aug 20 '09 at 22:13
  • How is UDF different in this regard? Is it not visible from Linux or Windows? – Gerald Combs Aug 20 '09 at 22:29
  • FAT32 has a 4GB file size limit and does not store POSIX permissions. NTFS does not store POSIX permissions. I do need those features, and only UDF provides them. Besides, it is faster than NTFS. – skolima Aug 24 '09 at 17:01
  • -1 because this is not an answer to the question. – dolmen Jun 02 '14 at 15:41
  • @skolima - What? That is like saying that you choose to raise Camels to generate milk without even considering Goats or Cows. There are way better file systems out there than UDF. – djangofan Jun 02 '14 at 15:54
  • NTFS is proprietary, and non-windows support is only reverse-engineered, not robust and standards-backed. Which are the better file systems that are compatible across OSes? – Milind R Aug 29 '14 at 20:18
  • Also, UDF is log-structured (similar to F2FS: it doesn't over-write inplace the data. It writes a new copy and eventually garbage collects the older copy on rewritable media like USB sticks and packed-writing DVDs). (That's a bit similar to Copy-on-Write filesystems like BTRFS, ZFS, etc.) That makes it much more resiliant to corruption than FAT32 and exFAT. It also makes it much nicer to the wear levelling of flash media. – DrYak Mar 22 '16 at 19:50
  • 1
    @DrYak UDF is log-structured only when being used with PseudoOverwrite or VAT partitions, AFAIU. – Milind R Jul 09 '17 at 06:35
  • UDF was designed for optical media, perhaps that's the problem? – Chopper3 Aug 17 '09 at 16:32
  • For initial revisions (before 1.5) this is true. However, later versions (those available in Linux and Windows Vista) are also meant to be used with random-rewrite media like hard drives. – skolima Aug 17 '09 at 17:08
  • "UDF is a truly universal file system. It can be used on all kinds of optical media, including read only [...], write once [...], rewritable [...], and of course block device (hard drives)." From ["Wenguang's Introduction to Universal Disk Format"](https://sites.google.com/site/udfintro/) – fbmd Jan 20 '14 at 09:30
  • https://web.archive.org/web/20151103171649/http://sipa.ulyssis.org/2010/02/filesystems-for-portable-disks/ , https://www.google.com/search?q=pieter+wuille+udf – MarcH Dec 14 '21 at 19:20

3 Answers3

17

It turns out that using the /q switch on Windows was the culprit: it enables 'quick format', i.e. the formatting process continues in background with every write made to the disk. Once it finishes, the drive is handled by Linux just fine.

skolima
  • 1,253
  • 3
  • 16
  • 28
  • On the other hand, formatting with Linux' mkudffs seems to sucessfully complete in seconds. – MarcH Dec 29 '11 at 23:41
  • I actually had to do a quick format in Win7 after creating the file system with mkudffs in Linux. Without that, Windows just kept wanting to format the drive. It ended up creating a 5MB empty space at the end of the drive, for whatever reason. But at least it works in both OS now. – DanMan Nov 03 '12 at 23:41
  • In reply to myself: those 5MB might be the meta data block in recent UDF versions. – DanMan Sep 08 '13 at 11:44
  • 2
    Totally wrong! Quick formatting is NOT "continuing format in background". It is just initialization of the file system without formatting tracks. – Anixx Sep 19 '16 at 12:43
7

When formatting UDF, one thing that can't be underscored enough is how important it is to use the proper block size when formatting UDF. As explained here, using a configured block size that doesn't match the physical sector size likely means that some operating systems will not be able to detect the drive as valid UDF.

Inspired by the above-linked answer (and my other research/testing), I wrote a script to automate the process of formatting in UDF--using the properly detected sector size. See format-udf on GitHub. Notable features:

  • Formats a block drive (hard drive or Flash drive) in Universal Disk Format (UDF)
    • UDF revision 2.01 used for maximal compatibility
    • First 4096 sectors are zeroed out to erase any existing MBR (necessary for proper UDF detection)
  • Resulting file system can be read/written across multiple operating system families (Windows, OS X, and Linux)
  • Runs on any OS having a Bash environment

Because of the last point, this script I wrote cannot be used on Windows. However, the script will run on OS X and Linux. After doing so, Windows should be able to magically detect the newly formatted UDF drive.

To directly answer the questions posted, format-udf will:

  • choose the appropriate tool for formatting based on operating system and environment (mkudffs on Linux)
  • automatically detect and populate all parameters necessary for formatting (including blocksize)
  • maximize OS compatibility (see GitHub page for compatibility chart)
  • yield the maximum feature set (and minimal limitations) that the asker is looking for
j0nam1el
  • 201
  • 2
  • 2
5

How did you reconcile your discovery in the previous question (that the UDF filesystem should be created on the whole disk, not a partition) with that Windows command ("format x: /fs:UDF")? In my attempts, Windows only gives drive letters to partitions.

As far as formatting for compatibility, I think the key is in the block size. Since most hard drives and USB flash sticks have a block size of 512 bytes, I've had the most compatibility when I create the FS that block size. I think format.com is using that block size, and mkudffs has a command switch for changing the block size. I could only get OS X and Windows to mount the filesystem when I used 512 byte blocks. Older versions of Linux assumed a block size of 2048, but you can always mount with "-o bs=512".

The whole disk vs partition issue still causes compatibility problems. Windows won't mount when I format the whole disk, and OS X doesn't look beyond the partition's type number when determining its filesystem, forcing me to mount it manually. Linux didn't care, as long as I gave it the appropriate device name (sda vs sda1).

In summary, the most compatible setup I've found is a singe partition of type 06(FAT16), formatted with UDF at block size 512. Works automatically on Windows, and a small bit of manual intervention on Linux and OS X.

Simon
  • 173
  • 3
  • 3
    I've tried another experiment. This time I used a USB flash stick instead of a hard drive. Formatted with "mkudffs --media-type=hd --blocksize=512 /dev/sda", works without hassle on Linux, OS X, and Vista. Too bad Windows seems to allow partitionless formats on USB sticks but not hard drives. – Simon Oct 09 '09 at 18:22
  • You definitely want to keep the same block size to avoid prematurely wearing the flash memory, see: http://lwn.net/Articles/428584/ – MarcH Dec 29 '11 at 01:22