Solved Best filesystem for USB sticks

[theLegionWithin]

Why should MSFT have put in support for the various file systems (and which one, exactly, do you mean as we have many options.)?

my take on that would be that if it's popular & would allow them to capture a certain percentage of the market, they'll do it - an example would be PDF document handling; now, PDF was developed by Adobe (love them or hate them for it), and initially a separate, non-microsoft app was required to open that filetype. eventually, PDF handling was baked into their web browser - it took a while to "work" and it's not ideal (especially if you hate the edge browser), but it's an ok solution. so, if competing filesystem types become commonplace (ie: 10% of the consumer market), then microsoft will likely bake that functionality into winOS.

 

[KGIII]

if it's popular & would allow them to capture a certain percentage of the market, they'll do it -

I'd not speculate as those days are likely to be a long way off, but they certainly could if it was something they could justify.

Only a scant few of us (comparatively speaking) are Linux users. The percentage of that who cares about Windows working with Linux file systems is likely to be vanishingly small.

We also don't all use the same file system. Also, there are a lot of file systems out there (not just for Linux).

List of file systems - Wikipedia

en.wikipedia.org

As mentioned in my previous post, they don't even support Apple's file system - which is significantly more popular than Linux. In that case, they'd only have to support one. It'd even cover the millions and millions of iOS devices out there.

 

[Sherri is a Cat]

...freedom to do what we want on our equipment.... M$ treating your equipment like it is theirs....for them it is all about how much money they can squeeze out of each of us.

After what I witnessed the summer of 2015, I knew I would eventually move to Linux. I got an app to put a leash on the incessant invitations and ongoing attempts to trick me into downloading an OS I didn't want.

Last year I finally got fed up. Long after support for my OS officially ended I continued to find little unwanted gifts downloaded onto my machine without my permission or knowledge. Every single one of them triggered some sort of malfunction in hardware. Finding and deleting them fixed the problem every single time.

One morning I flipped my lid. That night I took Linux for a test run (free of the latest hardware interference). The next morning I did a DOD wipe on my hard drive and reformatted it for Linux. I haven't looked back and I haven't regretted one single bit.

 

[wizardfromoz]

USB stick capacity has not been mentioned, but for any stick larger than 32 GB, ex-FAT should be used, unless you have specialist tools.

Cheers

Wiz

 

[Deleted member 204243]

USB stick capacity has not been mentioned, but for any stick larger than 32 GB, ex-FAT should be used, unless you have specialist tools.

I've never heard about this. Can you give (point to) more information?

 

[wizardfromoz]

Try Googling eg

fat32 usb stick size limit

first and see how you go.

Microsoft set it at 32 GB years ago, but I believe last year they were talking about agreeing to remove it.

Cheers

Nite.

 

[Deleted member 204243]

Microsoft set it at 32GB in their tool. It was always possible to use independent programs and format sticks up to 1TB in size. Also, Linux was never affected by it. The main limitation of FAT32 is the maximum file size (4GB).

So, no - exFat in not the only option for sticks bigger than 32GB. You can use FAT32, NTFS, ext4 or exFat.

 

[CaffeineAddict]

Microsoft set it at 32GB in their tool. It was always possible to use independent programs and format sticks up to 1TB in size. Also, Linux was never affected by it. The main limitation of FAT32 is the maximum file size (4GB).

So, no - exFat in not the only option for sticks bigger than 32GB. You can use FAT32, NTFS, ext4 or exFat.

Why do you bother with this?

The only time you really need FAT32 or ExFAT is when flashing BIOS because flash requires these filesystems.
For file storage just use ext4 for Linux and NTFS for Windows.

Problem solved, end of story.

If you happen to fail formatting 64GB USB to FAT32, just use ExFat and problem solved.

 

[Deleted member 204243]

Why do you bother with this?

Me? I don't. I just corrected a statement that was obviously wrong.

If you happen to fail formatting 64GB USB to FAT32, just use ExFat and problem solved.

I've never failed to format any stick to FAT32. 64GB, 128GB, 256GB... I was just complaining about the speed of transfer (it mainly affected transfer od small files). Your answer sounds like "Do you have a problem with a Windows program? Just install Linux."

And, no - I don't use all my sticks with Windows/Linux machines only.

 

[CaffeineAddict]

I was just complaining about the speed of transfer (it mainly affected transfer od small files). Your answer sounds like "Do you have a problem with a Windows program? Just install Linux."

What I meant, if you transfer files from Linux to Windows then format USB as NTFS on Linux, put files on it and transfer to Windows.

You said transfer speed is slow, is it slow on Linux machine or Windows machine?

 

[The Duck]

I had a very bad one today. It took 5 min. to transfer 400MB of data to the stick with FAT32.

This may be of interest to you.

Speed Up your Mint!

Easy tips, tweaks and tricks for Linux Mint and Ubuntu, both for beginners and for advanced users. Complete starter's guide with simple how-to's.

easylinuxtipsproject.blogspot.com

 

[dos2unix]

ntfs is slower than exFat because it does journaling, however the journaling in ntfs isn't 100% compatible with Linux. ntfs isn't POSIX compatible. (neither is exFat) ntfs allow caracters in filenames that make them difficult to work with in Linux (in some cases impossible) like : | ? * " < >
the "real" ntfs is closed source proprietary, it has been "mostly" reverse engineered to work with linux. fat32 is open source. exFat specification is still held by microsoft, the code is open source.

I tend to stay away from ntfs on Linux. It usually causes problems for me.

 

[Fanboi]

Recently I needed to move some files from one laptop to another. Firstly, I used a stick that had NTFS on it and it failed to unmount/mount. It was my mistake. I know that NTFS is not the best choice. I formatted it to FAT32, but the transfer was extremely slow. So, I formatted it to ext4 and finally everything works as it should, but, of course, I won't be able to use these sticks with Windows machines, my audio system or TV. ExFat is another file system from Microsoft. Do you have the same problems with FAT32 and NTFS or it's just a problem with my USB sicks?

For backups of backups, I use ext4, otherwise for sharing I use exFat because I know anything (i.e. other people's devices) will read it. I never have issues with slowdowns, except for specific use cases, which, alas brings me to this:

No. They were photos and video files.

If you have a lot of of files, especially of non-trivial size, especially of varying sizes (i.e. photos and videos) copying will slow down (with a lot of other variables at play such as caching, your drives' internal firmware, etc., blah, blah).

My advice (within this context):
Format: exFat
Copying: use rsync to copy (it has some optimizations over standard copying) and/or first archive (i.e. no compression) your photos and videos before copying them.

My alternative advice (depending on the full context):
You could also just use a cloud drive. Mega give you 20GB on a free account (last I checked, I'm on the 400GB paid plan -- which is well worth it). Honestly this is a good option because it also means your photos and videos are permenantly backed up and can be sync'd -- i.e. no copying needed.

Side Notes:
1. You could try WSL2 and use an ext4 drive for sharing and call it a day... Just another thought.
2. Since you're just transferring files, if the laptops are on the same network, just run a portable FTP server on the Linux one and connect via your local network.

Hope that helps

~~~
​

Really off-topic:
Remembered I bookmarked this a while back. Since it's of minor relevance here RE the whole Wondows doesn't support Linux FS thing: https://github.com/maharmstone/btrfs -- looks promising.

 

[bob466]

Not something I've ever worried about...plug them in and they work as Linux sees them all.

The Distro formats the Drive...Ventoy formats the Flash Drive...the only time I format is when I setup an External HDD for Timeshift which must be EXT4.

 

[Deleted member 204243]

You said transfer speed is slow, is it slow on Linux machine or Windows machine?

Linux. I also have an audio equipement and old Bluray and DVD players that I use with USB sticks.

The problem is solved now, but I always like to know why things are the way they are and if others have similar issues to the one I'm experiencing or that the fault is on my side only.

 

[bob466]

I remember back in the dark days of windoze...FAT32 meant you couldn't copy a file or folder larger than 4GB to a Drive formatted with FAT32 but I've never had this problem in Linux.

I've copied folders much larger than 4GB to a Drive formatted to FAT32 because Linux doesn't have restrictions like windoze...but which is faster FAT32 or EXT4 ?

I took one 2TB External HDD...one 32GB Flash Drive...both FAT32...all USB 3 and the USB port on my Tower is USB 3.

I transferred a 12GB folder from the External HDD to the Flash Drive...it took 24 minutes 32 seconds.

I then formatted the Flash Drive to EXT4...it took 19 minutes 40 seconds...so EXT4 is faster but only just.

It's worth noting...formatting the Flash Drive to EXT4 took up 1.6GB of space...whereas FAT32 took up next to nothing...so take your pick.

 

[MikeWalsh]

...all USB 3 and the USB port on my Tower is USB 3.

@bob466 :-

Heh. Oh, if ONLY it were that "simple", mate.

USB 1.0 & USB 2.0 were relatively straight-forward. Apart from the upgrade to "Superspeed" during the early part of USB 2.0's lifespan, everything was fairly predictable.

• USB 1.0 - max transfer rate was 12 MB/sec
• USB 2.0 - we all knew max transfer rate was pegged at 480 MB/sec. No ifs or buts. Rarely achieved in real-life, but theoretically possible, of course....

And THEN USB 3.0 arrived. With the exception of "Gen 1" - the early release - which gave a theoretical max transfer rate of 5GB/sec, everything rapidly went to hell in a handbasket!

USB 3.0 - Wikipedia

en.wikipedia.org

There was a slew of rapidly updated specifications; several in fact.....over the course of only about 4 months.

• USB 3.1 - also known as "USB 3.1 Gen 2" and "Gen 2x1" - upped that theoretical 'max' to 10GB/sec
• USB 3.2 - When this arrived, things got REAL complicated.....fast. We had modes such as "1x1", "1x2", "2x1", and "2x2" (??!?)....some of these replacing former labels for older implementations, along with denoting whether half-duplex OR full-duplex, single OR dual transfer 'lane' protocols were in use and/or supported. Top & bottom, the max possible transfer rate doubled again, this time to 20GB/sec.

And then "USB 4.0" arrived.....bringing with it theoretical data transfer rates of up to 40GB/sec, and the soon-to-be ubiquitous USB-C connector, along with Thunderbolt support.....IF the manufacturer chose to implement it! Under certain rare combinations of highly-specific conditions, I believe figures of 80GB/sec have been bandied about....

And all that through a plug-in connector.

Goes without saying; all ports/interfaces/cables and chipsets, etc, need to "match" to achieve these insanely high speeds. But as for trying to figure out what you have, and what you might achieve......forget it. Only someone with a post-grad MEng doctorate could possibly make sense of that lot!!

IMHO, the USB Implementer's Forum need their arses kicking. HARD.


Mike.

 

[bob466]

@bob466 :-

Heh. Oh, if ONLY it were that "simple", mate.

It is that simple because it boils down to what you use.

I decided to test my 1TB portable SSD...
The same 2TB portable External HDD formatted to FAT32...then copy the same 12GB folder to my 1TB portable External SSD also formatted to FAT32 from the same USB 3 computer ports.

How long did it take...7 minutes 32 seconds.

It just goes to show...Flash Drives are still slow and the format doesn't matter with the exception of my TV because it only reads FAT32...such is life.