If I mount my USB drive, and I write something to it and take it out without unmounting then the files aren’t there unless if I actually unmount then it actually writes to the USB drive.
Is there a way to write to the USB drive directly as soon as when I performed an action (such as copying, removing saving etc)?
This is called caching and is intended to improve performance, especially on slow devices.
I’m sure it’s tunable, but of course you can take two approaches to ensure files are written.
Why is unmounting safely unsatisfactory for you ?
Its actually slower in my case as I am sitting there waiting for files to actually get copied when I want to unmount the device.
Yes how would I do this?
It’s not going to be faster. There is no magic to make files write faster, they generally go as fast as the device can support.
If you want faster, you buy like a portable SSD or something. Or perhaps use a bluetooth device that you can move without being tethered to a physical connection.
A quick google (which you might want to learn about) reveals this is the answer to the tunability of write-caching. With the device being whatever your device is.
sudo hdparm -W 0 /dev/blockdevice
I’m also interested in a way around this apparent insolence especially from GNOME and derrivatives. I have to deal with it on Manjaro MATE and in the past, it was a flaw with Solus MATE as well. Dolphin and Thunar could also do it.
In my experience it’s copying many me(bi|ga)bytes of data to an USB v3.0 device on a computer that doesn’t have “Super Speed” drive. Request unplug and the thing sits there for a l-o-o-o-ng time after “do not unplug” notification goes away, and before it gets around to saying “Device may be unplugged”. This has always annoyed the heck out of me. It’s just rude and inelegant. It would be nice to have a way to force it to write to the disk right at the time it’s copying, in this situation. What if I have a power outage? I do have my computer connected to an UPS but it’s unreliable because power generation is problematic where I live.
This makes me afraid to really do anything profound on EndeavourOS such as compiling a “large” project like I had to do a couple of times on Slackware 32-bit.
I don’t think it would be any different on a computer that does have the “Super Speed” drives, it’s only faster with the progress which makes more annoying the long wait to get permission to unplug.
Using the sync command could help a lot, as @dbarronoss already mentioned.
And unmounting is also very sensible.
Generally speaking, the device isn’t sitting there doing nothing until you try to unmount it. It is usually writing the data from it’s cache to non-volatile storage the entire time. There is nothing you can do to make that process faster. Skipping the cache wouldn’t make a material difference in actual write times. It would just tell you it is done much less soon.
The real problem is that many of these USB drives have exceptionally slow storage in them. Even if the device is usb 3.1 that doesn’t mean the storage in it is any faster than the old usb 2.0 drive sitting next to it. That is why you have to look at performance tests that far exceed the cache to see the actual performance of the device. The way to make it faster, is to buy faster devices.
Even running sync isn’t going to make it faster. That is just doing the same thing that is happening when you try to unmount it.
What I mean is I don’t want to be sitting there waiting for the USB drive to be writting when I want to unmount it. To me that is slower than having directly to write to the USB drive with slower times.
how would I do this to any USB drive I connect to without having to run this command?
That is what I want
I agree with everyone who has cautioned you not to disable write caching. However, should you really want to do so, here’s the article to read:
https://wiki.archlinux.org/title/hdparm
In particular, read these sections:
https://wiki.archlinux.org/title/hdparm#Write_cache
https://wiki.archlinux.org/title/hdparm#Persistent_configuration_using_udev_rule
Warning: What I write below has not been tested by me, and I may be completely wrong. Experiment at your own risk.
Assuming that you want to disable write caching only for non-rotational drives, my guess is that you would do the following…
Create this file:
/etc/udev/rules.d/69-hdparm.rules
Then add this code to the file:
ACTION=="add|change", KERNEL=="sd[a-z]", ATTRS{queue/rotational}=="0", RUN+="/usr/bin/hdparm -W 0 /dev/%k"
Assuming that you want to disable write caching for all drives, even rotational drives—which would be an even worse idea, in my opinion—my guess is that you would add this code instead:
ACTION=="add|change", KERNEL=="sd[a-z]", RUN+="/usr/bin/hdparm -W 0 /dev/%k"
Is this for both internal and external drives though? or is this for just external drives (connected via USB connection such as a USB drive)?
As written, it’s for both internal and external drives, I think. However, the KERNEL parameter seems to be a regular expression, which you should be able to modify easily to exclude your internal drives.
The expected behavior of the OS was explained by @dalto to me and therfore I must abide to it. Allow it to waste time if it has to. What could I do? I don’t want to lose any data.
Windows went into antics enough times for me. It convinced me that it could disconnect disks “out of the blue” while going into its Defender scans, wasteful on a computer which never had Internet access. But one day it was my fault trying in vain to get a connection from an external USB hub that was supposed to be powered. Powie bye-bye directory table. It’s a good thing I backed up a short time before and the contents weren’t super important. 
In some use cases, the most important concern would indeed be the protection of data. The article I cited above makes that point:
While write cache is a free performance boost, it also comes with a downside. Enabling write cache increases the risk of data loss when the power is suddenly lost.
One of the greatest attributes of Linux is that it provides a way to configure the system to serve the needs of almost any use case. If disabling the write cache makes sense in your use case, then you should certainly do it.
This kind of topic is usually created by users who have not yet developed an understanding of their linux system and the inner workings.
The topic has been frequently discussed on Manjaro forum - with various sysctl tweaks as a result.
SUBSYSTEMS=="usb", SUBSYSTEM=="block", ENV{ID_FS_USAGE}=="filesystem", ENV{UDISKS_MOUNT_OPTIONS_DEFAULTS}+="sync", ENV{UDISKS_MOUNT_OPTIONS_ALLOW}+="sync"– https://forum.manjaro.org/t/decrease-dirty-bytes-for-more-reliable-usb-transfer/120798/4
In the beginning of EndeavourOS as it was with Manjaro - it was Linux superusers and experts - and when the distribution gets traction - and news attention - it attracts all kinds of users including users who knows next to nothing of Linux, users who makes the common mistake to substitute Linux for Windows.
Linux != Windows
The writing starts immediately but as USB device storage is slower than memory - data is cached to memory using write cache.
All kinds of USB devices exist - devices slow as snail and others like a cheetah.
The suggestion to use an udev rule to disable write cache drew my attention.
To me the idea is a 
moment and I know how to improve this rule to only target USB devices
ACTION=="add|change", KERNEL=="sd[a-z]", RUN+="/usr/bin/hdparm -W 0 /dev/%k"
One should realize that not all USB devices support write-caching and in such case - the result of disabling write-caching will likely affect system especially when it comes to large transfers from one usb to another.
So - this will never be a one size fits all - but when you know what you are doing - the input you gave me here is noteworthy.
To come up with this I had to get an idea of what happened and udevadm is a great tool
To watch what happens when a device is inserted
udevadm monitor
To get info on the device - where y is the device just added
udevadm info /dev/sdy
Testing a handful usb devices I realized the ID_USB_TYPE=disk would be an ideal pointer to ensure only usb devices was targeted.
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/hdparm -W 0 /dev/%k"
This addition will likely affect sdcards as well - suffice they support write-caching - as card readers - including internal - is usually attached to the usb bus.
There is a couple of usable E:
E: ID_USB_TYPE=disk
E: ID_USB_DRIVER=usb-storage
SDcard
$ udevadm info /dev/sdf
P: /devices/pci0000:00/0000:00:03.1/0000:02:00.0/0000:03:08.0/0000:04:00.3/usb8/8-4/8-4:1.0/host11/target11:0:0/11:0:0:1/block/sdf
M: sdf
U: block
T: disk
D: b 8:80
N: sdf
L: 0
S: disk/by-id/usb-Generic-_USB3.0_CRW-SD_201412031053-0:1
S: disk/by-path/pci-0000:04:00.3-usb-0:4:1.0-scsi-0:0:0:1
S: disk/by-diskseq/13
Q: 13
E: DEVPATH=/devices/pci0000:00/0000:00:03.1/0000:02:00.0/0000:03:08.0/0000:04:00.3/usb8/8-4/8-4:1.0/host11/target11:0:0/11:0:0:1/block/sdf
E: DEVNAME=/dev/sdf
E: DEVTYPE=disk
E: DISKSEQ=13
E: MAJOR=8
E: MINOR=80
E: SUBSYSTEM=block
E: USEC_INITIALIZED=4122115
E: ID_BUS=usb
E: ID_MODEL=USB3.0_CRW-SD
E: ID_MODEL_ENC=USB3.0\x20CRW-SD\x20\x20\x20
E: ID_MODEL_ID=0321
E: ID_SERIAL=Generic-_USB3.0_CRW-SD_201412031053-0:1
E: ID_SERIAL_SHORT=201412031053
E: ID_VENDOR=Generic-
E: ID_VENDOR_ENC=Generic-
E: ID_VENDOR_ID=0bda
E: ID_REVISION=1.00
E: ID_TYPE=disk
E: ID_INSTANCE=0:1
E: ID_USB_MODEL=USB3.0_CRW-SD
E: ID_USB_MODEL_ENC=USB3.0\x20CRW-SD\x20\x20\x20
E: ID_USB_MODEL_ID=0321
E: ID_USB_SERIAL=Generic-_USB3.0_CRW-SD_201412031053-0:1
E: ID_USB_SERIAL_SHORT=201412031053
E: ID_USB_VENDOR=Generic-
E: ID_USB_VENDOR_ENC=Generic-
E: ID_USB_VENDOR_ID=0bda
E: ID_USB_REVISION=1.00
E: ID_USB_TYPE=disk
E: ID_USB_INSTANCE=0:1
E: ID_USB_INTERFACES=:080650:
E: ID_USB_INTERFACE_NUM=00
E: ID_USB_DRIVER=usb-storage
E: ID_PATH=pci-0000:04:00.3-usb-0:4:1.0-scsi-0:0:0:1
E: ID_PATH_TAG=pci-0000_04_00_3-usb-0_4_1_0-scsi-0_0_0_1
E: ID_PART_TABLE_UUID=dc14d963
E: ID_PART_TABLE_TYPE=dos
E: DEVLINKS=/dev/disk/by-id/usb-Generic-_USB3.0_CRW-SD_201412031053-0:1 /dev/disk/by-path/pci-0000:04:00.3-usb-0:4:1.0-scsi-0:0:0:1 /dev/dis>
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:
Do you happen to be using GNOME/ Nautilus?
Nautilus is VERY slow with this end step for making sure you don’t end up with bad data written. I’m not sure of the what or why but Dolphin on KDE doesn’t generally do this as much. I have a couple USB drives that always do it but those are just REALLY slow (10-15mb/s writes)
You might try a different file manager if you’re using Nautilus as its frequently sluggish.