Solved wrestool wasting all disk I/O

[CaffeineAddict]

I finally figured out why copying files to external HDD is painfully slow...

There is a directory on my external HDD which contains only Windows executables, 425 GB in total, cca. 200 *.exe files.
As soon as this directory is open in Dolphin a ton of wrestool instances start running examining and extracting all those executables. (noticed them by running htop)
Reason why it does so appears to be to calculate *.exe file sizes because I noticed sizes of sub directories in Dolphin are not immediately calculated but get slowly updated in UI.

Problem with those wrestool instances is that they consume all disk I/O so copying anything to HDD is either completely halted or painfully slow.
I've run killall wrestool and that made copying files lightning fast because disk I/O was released and taken over by copy operation.

Question is, how to prevent wrestool from automatically running and consuming disk I/O when there are *.exe files on filesystem?

See: man wrestool

 

[f33dm3bits]

I finally figured out why copying files to external HDD is painfully slow...
Question is, how to prevent wrestool from automatically running and consuming disk I/O when there are *.exe files on filesystem?

If I remember correctly certain filesystems are able to deal better with certain types of file structures better, as in some filesystem perform better when you have a lot of small files on than other filesystems and and it's possible to optimize the filesystem if you know what you are doing, that is done with fsck with certain options.

BTRFS, ZFS, XFS, EXT4: What Difference & Which File System to Use?

SalvageData explains the difference between BTRFS, ZFS, XFS, and EXT4 so you can decide what's the best file system for your business. Read the full guide here!

www.salvagedata.com

 

[f33dm3bits]

Then I forgot to ask what, filesystem do you have on the partition of that disk?

 

[CaffeineAddict]

If I remember correctly certain filesystems are able to deal better with certain types of file structures better, as in some filesystem perform better when you have a lot of small files on than other filesystems and and it's possible to optimize the filesystem if you know what you are doing, that is done with fsck with certain options.

This is something to study about to choose best FS for storage, I'll read about it to see if it may be useful to change FS but I highly doubt it would impact wrestool.

I see you've edited your post so I've read the reddit post, but the guy is talking about "3.1GB of ~40 million files" and "generating about 20,000 images per second"
I don't store a lot of small files and nowhere near these crazy numbers:

user@msi:~$ df -ih /dev/sdc1
Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/sdc1         59M  4.6K   59M    1% /media/user/External

user@msi:~$ df -h /dev/sdc1
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc1       916G  661G  209G  77% /media/user/External

Then I forgot to ask what, filesystem do you have on the partition of that disk?

It's ext4, reformatting the disk to another FS and copying over data would take a whole day

 

[f33dm3bits]

It's ext4, reformatting the disk to another FS and copying over data would take a whole day

I have another option but that would require you to buy a new disk since you use it as external storage. What I do for myself now days for external storage is buy a fast M.2 nvme ssd and then I buy a casing for it and then I put the M.2 nvme ssd in there, it's way faster than any normal external storage.

ORICO M.2 NVMe SSD Enclosure
dockteck M.2 NVMe SATA SSD Enclosure

 

[CaffeineAddict]

I have another option but that would require you to buy a new disk since you use it as external storage. What I do for myself now days for external storage is buy a fast M.2 nvme ssd and then I buy a casing for it and then I put the M.2 nvme ssd in there, it's way faster than any normal external storage.

Amazon.com

Amazon.com

Interesting approach and not too expensive, this is my 2nd external HDD for 50€, previous one died and this one is some 5 months old.
Next time I'll consider this instead, although my mobo has USB 3.1 gen 1 port that's also on the back of my case so I normally use USB 2.0 for external HDD which is in front because don't have to move the whole desk to connect it.

It may be scheduled on next mobo purchase, but for now not anytime soon.

I still hope there is software solution to disable wrestool somehow, this would solve the problem and copy\paste to disk is working fine otherwise as long as there is no directory with exe files accessed.

 

[f33dm3bits]

I still hope there is software solution to disable wrestool somehow, this would solve the problem and copy\paste to disk is working fine otherwise as long as there is no directory with exe files accessed.

What file-manager do you use and what DE are you using?

 

[CaffeineAddict]

What file-manager do you use and what DE are you using?

Dolphin, KDE

 

[f33dm3bits]

Dolphin, KDE

From what I have seen and can remember of Dolphin is that it's quite configurable, I may be wrong, so I would look for settings somewhere that disables previewing or calculating files. I don't have anything with KDE installed in a vm right now, but I'm sure you can find that yourself. Other solution you could try is if wrestool is a binary on the system rename it to disabled or rename and try symlinking it to /dev/null. Or write a cronjob that runs all the time that runs "killall wrestool".

 

[CaffeineAddict]

You're my hero!
I disabled the option from screenshot below and that stops wrestool from running for exe files.

This is solved, thanks!

 

[f33dm3bits]

You're my hero!
I disabled the option from screenshot below and that stops wrestool from running for exe files.

Glad that worked! I would still recommend for the future when you replace your external storage to use an M.2 nvme ssd in an enclosure.

 

[CaffeineAddict]

I would still recommend for the future when you replace your external storage to use an M.2 nvme ssd in an enclosure.

I was considering SSD instead of HDD, but given how rarely I use it and only for backup I ended up saving some bucks.
Maybe I will in the future, it's not that big price difference, but it will have to go together with new mobo with USB 3.0 in front of tower.

I really pray that my CPU or mobo dies to have a reason to buy a new build, this one is some 5-6 yrs old and still functioning, so far I've upgraded only SSD, PSU and GPU, as long as mobo and CPU works I'll be only upgrading, or unless games start demanding stronger CPU, no need otherwise.

 

[f33dm3bits]

I really pray that my CPU or mobo dies to have a reason to buy a new build, this one is some 5-6 yrs old and still functioning, so far I've upgraded only SSD, PSU and GPU, as long as mobo and CPU works I'll be only upgrading, or unless games start demanding stronger CPU, no need otherwise.

Buy yourself an early Christmas present, as an excuus to buy something new

 

[CaffeineAddict]

Buy yourself an early Christmas present, as an excuus to buy something new

Even better, I'll ask at home if santa will come if I'm good lol