diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ce8d4d7 --- /dev/null +++ b/.gitignore @@ -0,0 +1,15 @@ +# ---> ArchLinuxPackages +*.tar +*.tar.* +*.jar +*.exe +*.msi +*.zip +*.tgz +*.log +*.log.* +*.sig + +pkg/ +src/ + diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..2071b23 --- /dev/null +++ b/LICENSE @@ -0,0 +1,9 @@ +MIT License + +Copyright (c) + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..dca3cb9 --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +## Linux filesystem cache + +Linux agressively caches files in order to improve overall performance. + +* https://linuxatemyram.com + +When copying large amount of files to an USB storage this often results in some wait time until the device can be safely removed. + +How long you have to wait depends on your system and the quality of the USB storage device. + +Numerous issues regarding this has resulted in various suggestions involving sysctl tweaks and trim. + +Examples: + +* https://forum.manjaro.org/t/unbelievably-slow-file-transfer-speeds/123222 +* https://forum.manjaro.org/t/decrease-dirty-bytes-for-more-reliable-usb-transfer/120798 +* https://forum.manjaro.org/t/slow-usb-transfers/146863 +* https://forum.manjaro.org/t/usb-transfer-speed-5mb-sec-dropping/146777 +* https://forum.manjaro.org/t/extremely-slow-file-transfer-speeds-from-usb-3-0-to-external-hdd/39339 +* https://forum.manjaro.org/t/transfer-speed-slows-down-to-a-crawl-after-10gb-transfered/43453 +* https://forum.manjaro.org/t/usb-transfer-data-drops-drastically-and-maximum-number-of-clients-are-reached/93278 +* https://forum.manjaro.org/t/file-copy-on-a-usb-2-slows-down-to-a-halt/36316 +* https://forum.manjaro.org/t/dolphin-large-file-transfer-to-usb-stick-indicates-completed-after-caching-is-done-transfer-still-ongoing/132634 + +## udev rule to disable write-cache + +In [another place][2] one user threw in an udev rule which would disable write-cache for devices when they were added and while it was just an idea - it triggered my curiosity. + +I dug into the intricacies of udev and found a method to only target USB storage devices. + +* Documentation for https://wiki.archlinux.org/title/udev + +The rule has gotten a major overhaul and now consist of the rule, a config file and a script + +https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/99-usb-sync.rules + +https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/udev-usb-sync.conf + +https://gitlab.manjaro.org/fhdk/udev-usb-sync/-/blob/master/udev-usb-sync + +## How it works +The rule activates when udev detects + +* add or change +* kernel event for disk devices **sd[a-z]** +* only if the device environment **ID_USB_TYPE=='disk'** +* run + - **hdparm -W 0 /dev/%k** (disable write cache if supported) + - **udev-usb-sync %k** + - applies defaults + - read config and apply user values + - if use_tweaks=0 the script exits + - if use_tweaks=1 the applies the values (default or config) + - set a max_bytes value based on device speed + +## How to setup manually +Create a file in **/etc/udev/rules.d/99-usb-sync.rules** and paste the rule into it. +Create a file in **/etc/udev-usb-sync/udev-usb-sync.conf** and paste the default values. +Create a file in **/usr/bin/udev-usb-sync** and paste the script content. + +Install **hdparm** package. + + sudo pacman -Syu hdparm + +Reload udev + + sudo udevadm control --reload + +Then plug an usb device - open in your file manager - copy a huge amount of files to the device - when the copy is done - click eject in the file manager - note how quick the device is ejected. + +For those preferring the package manager, I have created a [PKGBUILD][1] which will pull the **hdparm** dependency upon installation. + + pamac build udev-usb-sync + +Another fine utility script provided by @cscs fine-tunes a number of system parameters with the option to input your own values when the script is run + +https://gitlab.com/cscs/maxperfwiz + +https://gitlab.manjaro.org/fhdk/udev-usb-sync + +* Cross posted - https://root.nix.dk/en/system-configuration/disable-write-cache-for-usb-devices +* Related topic with tests - [Slow USB transfers - #12 by linux-aarhus](https://forum.manjaro.org/t/slow-usb-transfers/146863/12) + +[1]: https://aur.archlinux.org/packages/udev-usb-sync +[2]: https://forum.endeavouros.com/t/how-come-everything-is-written-when-i-decide-to-unmount-a-usbdrive/37178/10 \ No newline at end of file