exFAT (Extended File Allocation Table) vs UDF (Universal Disk Format) - File System for Flash Drives
Post date: Apr 3, 2018 6:03:56 AM
Simple tests exFAT vs UDF on Flash Drive, performance, compatibility, just for fun.
Test data set & results
Large file: 1 x 1 GB
Small files: 10k x 1 KB = 10 MB
Time Unit: minutes:seconds.second_decimals
Write:
Large file: 2:24.90
Small files: 1:07.81
Read:
Large file: 0:41.06
Small files: 0:14:25
Delete:
Large file: 0:00.90
Small files: 0:22.27
Write:
Large file: 2:20.77
Small files: 0:18.96
Read:
Large file: 2:11.32
Small files: 0:19.10
Delete:
Large file: 0:00.36
Small files: 0:08.44
Notes
After every test all IO buffers and caches were flushed (and it's included in the time) and cleared (no data re use from cache)
All test steps were repeated three times, and results were consistent enough and the values shown are the median values.
Some of the results are so confusing, I've gotta recheck... Let's verify...
Just to be sure, I recreated all the tests using another, but 100% similar storage device.
After all, it's clear that the file systems behave very differently in different situations. Yet, the absolutely most interesting part was the large file read performance. How it can differ so much with same storage media? And repeated tests confirmed that. Amazing and confusing at the same time.
Oh one more remark, the exFAT device was using 4K cluster size, which is disadvantage for exFAT small files handling. UDF uses 512 byte "blocks" on the used device. So this can explain why the small files read is bit slower on exFAT.
The flash drives used for testing are drives which are generally quite slow, but provide good random access performance. Results could be different with some of the Flash Drives, which are fast, but provide devastatingly poor random access performance, especially when writing.
Conclusion
There's no silver bullet or simple answer. Both solutions do work, and provide different performance characteristics. Yet probably which ever you're using, you're not using it inherently for performance reasons, but for compatibility. Anyway, if exFAT works for you, use it. As far as I can see, there's no clear benefits for using UDF.
Reasons why I bothered even to check out UDF, were the fact that exFAT free space bitmap gets often corrupted (lost disk space) and you can't get it fixed on Linux without reformat. Yet I also managed to corrupt UDF on unclean dismout tests quite quickly, ending up with file which couldn't be read or deleted. Just as with exFAT there's no way to fix this, except to format the device again.
Other more or less interesting stuff? I created the UDF systems on Linux using mkudffs and when I run chkdsk on Windows 10, the results were curious:
With partition table, yet Windows removed partition table from USB drive when chkdsk was run ... This is amazing...
Also all files written with Linux were read-only files when being used with Windows, but it was enough to remove the read-only flag.
Tech stuff for nerds
On this device, I did create MBR and partition with UDF. Windows removed the partition table. What if I would have had multiple partitions on the device? I didn't test that, but it could be ahem, quite bad end result. Anyway, in both of cases the list of things getting corrected is quite scary. It's horrible to see, how there can be so different views how UDF should be implemented and operated.
The type of the file system is UDF.Logical Volume Integrity Descriptor at block 273 is corrupt.Creating new Logical Volume Integrity Descriptor at block 15654592.Reserve Volume Descriptor Sequence is corrupt or unreadable.Volume MBRPRT is UDF version 2.01. CHKDSK is verifying ICBs ...ICB verification completed.CHKDSK is looking for orphan ICBs ...Search for orphans completed.CHKDSK is verifying ICB links ...ICB link verification completed.CHKDSK is verifying link counts and parent entries ...Link count and parent entry verification completed.CHKDSK is checking system files.CHKDSK is checking the directory tree for cycles.CHKDSK is identifying lost files.Lost file identification completed.CHKDSK is verifying object size for ICBs with alternate data streams ...ICB object size verification completed.Correcting errors in Space Bitmap Descriptor at block 0.Correcting directory count to 1903 in Logical Volume Integrity Descriptor.Correcting file count to 19789 in Logical Volume Integrity Descriptor.Correcting partition size for partition 0 to 15654317 in Logical VolumeIntegrity Descriptor.Correcting partition free space for partition 0 to 12144363 in Logical VolumeIntegrity Descriptor.Creating Reserve Volume Descriptor Sequence at sector 15654841.Correcting errors in Unallocated Space Descriptor in the VolumeDescriptor Sequence.Correcting errors in the Volume Descriptor Sequence.Correcting errors in Anchor Volume Descriptor Pointer. Windows has made corrections to the file system.No further action is required. 7827158 KB total disk space. 1750573 KB in 19789 files. 2474 KB in 1903 directories. 0 KB in 1 stream directories. 1929 KB in use by the system. 6072181 KB available on disk. 512 bytes in each allocation unit. 15654317 total allocation units on disk. 12144363 allocation units available on disk.
With this device, I created pure UDF file system directly on the block device without partition table as UDF is supposed to be created.
The type of the file system is UDF.Logical Volume Integrity Descriptor at block 273 is corrupt.Creating new Logical Volume Integrity Descriptor at block 15646400.Reserve Volume Descriptor Sequence is corrupt or unreadable.Volume UDFPURE is UDF version 2.01. CHKDSK is verifying ICBs ...ICB verification completed.CHKDSK is looking for orphan ICBs ...Search for orphans completed.CHKDSK is verifying ICB links ...ICB link verification completed.CHKDSK is verifying link counts and parent entries ...Link count and parent entry verification completed.CHKDSK is checking system files.CHKDSK is checking the directory tree for cycles.CHKDSK is identifying lost files.Lost file identification completed.CHKDSK is verifying object size for ICBs with alternate data streams ...ICB object size verification completed.Correcting errors in Space Bitmap Descriptor at block 0.Correcting directory count to 1903 in Logical Volume Integrity Descriptor.Correcting file count to 19789 in Logical Volume Integrity Descriptor.Correcting partition size for partition 0 to 15646125 in Logical VolumeIntegrity Descriptor.Correcting partition free space for partition 0 to 12136683 in Logical VolumeIntegrity Descriptor.Creating Reserve Volume Descriptor Sequence at sector 15646649.Correcting errors in Unallocated Space Descriptor in the VolumeDescriptor Sequence.Correcting errors in the Volume Descriptor Sequence.Correcting errors in Anchor Volume Descriptor Pointer. Windows has made corrections to the file system.No further action is required. 7823062 KB total disk space. 1750573 KB in 19789 files. 2218 KB in 1903 directories. 0 KB in 1 stream directories. 1928 KB in use by the system. 6068341 KB available on disk. 512 bytes in each allocation unit. 15646125 total allocation units on disk. 12136683 allocation units available on disk.But if the file system is created using Windows format /FS:udf then the output is different:
The type of the file system is UDF.Volume BlueGld is UDF version 2.01.CHKDSK is verifying ICBs ...ICB verification completed.CHKDSK is looking for orphan ICBs ...Search for orphans completed.CHKDSK is verifying ICB links ...ICB link verification completed.CHKDSK is verifying link counts and parent entries ...Link count and parent entry verification completed.CHKDSK is checking system files.CHKDSK is checking the directory tree for cycles.CHKDSK is identifying lost files.Lost file identification completed.CHKDSK is verifying object size for ICBs with alternate data streams ...ICB object size verification completed.Windows has scanned the file system and found no problems.No further action is required. 7827136 KB total disk space. 0 KB in 1 directories. 1911 KB in use by the system. 7825224 KB available on disk. 512 bytes in each allocation unit. 15654272 total allocation units on disk. 15650448 allocation units available on disk.So it seems that Windos and Linux got different views of UDF 2.01. - That's good start for many problems.
kw: USB, flash, performance, benchmark, test, testing, reading, writing, Microsoft Windows, Linux, exFAT, UDF, file system, fs, flash drive, drives, exFAT vs UDF, UDF vs exFAT, comparison.