.gitignore | ||
99-usb-sync.rules | ||
LICENSE | ||
PKGBUILD | ||
README.md | ||
udev-usb-sync | ||
udev-usb-sync.conf | ||
udev-usb-sync.install |
Linux filesystem cache
Linux agressively caches files in order to improve overall performance.
When copying large amount of files to an USB storage this often results in some wait time until the device can be safely removed.
How long you have to wait depends on your system and the quality of the USB storage device.
Numerous issues regarding this has resulted in various suggestions involving sysctl tweaks and trim.
Examples:
- https://forum.manjaro.org/t/unbelievably-slow-file-transfer-speeds/123222
- https://forum.manjaro.org/t/decrease-dirty-bytes-for-more-reliable-usb-transfer/120798
- https://forum.manjaro.org/t/slow-usb-transfers/146863
- https://forum.manjaro.org/t/usb-transfer-speed-5mb-sec-dropping/146777
- https://forum.manjaro.org/t/extremely-slow-file-transfer-speeds-from-usb-3-0-to-external-hdd/39339
- https://forum.manjaro.org/t/transfer-speed-slows-down-to-a-crawl-after-10gb-transfered/43453
- https://forum.manjaro.org/t/usb-transfer-data-drops-drastically-and-maximum-number-of-clients-are-reached/93278
- https://forum.manjaro.org/t/file-copy-on-a-usb-2-slows-down-to-a-halt/36316
- https://forum.manjaro.org/t/dolphin-large-file-transfer-to-usb-stick-indicates-completed-after-caching-is-done-transfer-still-ongoing/132634
udev rule to disable write-cache
In another place one user threw in an udev rule which would disable write-cache for devices when they were added and while it was just an idea - it triggered my curiosity.
I dug into the intricacies of udev and found a method to only target USB storage devices.
- Documentation for https://wiki.archlinux.org/title/udev
The rule has gotten a major overhaul and now consist of the rule, a config file and a script
https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/99-usb-sync.rules
https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/udev-usb-sync.conf
https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/udev-usb-sync
How it works
The rule activates when udev detects
- add or change
- kernel event for disk devices sd[a-z]
- only if the device environment ID_USB_TYPE=='disk'
- run
- hdparm -W 0 /dev/%k (disable write cache if supported)
- udev-usb-sync %k
- applies defaults
- read config and apply user values
- if use_tweaks=0 the script exits
- if use_tweaks=1 the applies the values (default or config)
- set a max_bytes value based on device speed
How to setup manually
Create a file in /etc/udev/rules.d/99-usb-sync.rules and paste the rule into it. Create a file in /etc/udev-usb-sync/udev-usb-sync.conf and paste the default values. Create a file in /usr/bin/udev-usb-sync and paste the script content.
Install hdparm package.
sudo pacman -Syu hdparm
Reload udev
sudo udevadm control --reload
Then plug an usb device - open in your file manager - copy a huge amount of files to the device - when the copy is done - click eject in the file manager - note how quick the device is ejected.
For those preferring the package manager, I have created a PKGBUILD which will pull the hdparm dependency upon installation.
pamac build udev-usb-sync
Another fine utility script provided by @cscs fine-tunes a number of system parameters with the option to input your own values when the script is run
https://gitlab.com/cscs/maxperfwiz
https://gitlab.manjaro.org/fhdk/udev-usb-sync
- Cross posted - https://root.nix.dk/en/system-configuration/disable-write-cache-for-usb-devices
- Related topic with tests - Slow USB transfers - #12 by linux-aarhus