What is absolute fastest recursive copy method on Linux?
For something like copy of whole 4 Tb HDD, mixed files
It depends on a lot of factors including how both drives are connected, if the disk is encrypted, the performance of both drives , what the underlying filesystems are, how large the files are and how full the partitions are.
I almost always use rsync to copy/move files between devices because it is fast, easy to resume if it fails in the middle and, most importantly, if it is capable of copying almost anything completely and accurately.
On the other hand, if you are cloning an entire partition, it might be faster to dd it, especially if you have lots of small files.
1 Like
Hmm…Well let’s say something like
- Both HDD drives connected via SATA 3 same motherboard
- Both not ecrypted
- ext4 is the
- Mixed files from 200 Gb per file, to small text files / pictures etc
I’m not feeling safe with dd, if you catch my drift 
But yeah…It’s powerful no doubt.
Let’s stick to rsync, i’ve heard good things but have never used it.
Can you please give me an example command of how would i move whole drive recursively to another one (preferably with some feedback like % or ETA, if it doesn’t hurt performance)?
P.S. Oh, and also i wonder how rsync goes about symlinks, i’d like them to be just mirrored as same symlinks (not followed / copied as original files, like some tools do)
In general use for backups I use rsync -avhW source destination
This produces a good copy preserving the things I care about. You might want to replace v with P to show progress.
The Arch wiki recommends this for file system cloning
rsync -qaHAXS SOURCE_DIR DESTINATION_DIR
That should copy almost anything exactly. It will even copy things like device files and hard links. This is probably more like what you want. Replacing q with P or --progress should give you some feedback.
7 Likes
There is always Grsync…oooops, that’s a GUI 
2 Likes
Thx a lot!
I really appreciate it, since man rsync is a real mess, and time is of the essence 
One more question, what if i want exclude couple of dirs from source?
Let’s say ./temp and ./VM
P.S. Other suggestions are still welcome, but god forbid NO GUI!!!11 IT’S BLOAT!!! 
2 Likes
Use --exclude. It accepts a pattern and can be added to the command line more than once.
1 Like
If you have tldr installed - it has a short form suggestion or 2…
BTW - one of the neat features of rsync is that you can give a ‘trial run’ to make sure you have the command right before launching the final version of the command - and that applies even to the GUI version grsync too 
4 Likes
Sorry for the /OT/
@dalto, do you know if dd supports btrfs. I tried to clone my system partition the other day and I ended up with a an empty partition on the target disk.
I have never tried it. I would think if you clone a whole disk, it definitely should work. However, I am not sure where btrfs stores it’s metadata. If you only clone a single partition and that excludes the metadata that might be a problem. Keep in mind, that is just speculation. It should be pretty easy to find that out though.
1 Like
This was what I did so it might be that the metadata is stored somewhere else. Thanks for the explanation! I will have to do some research about it to see if I find something.
Wow, that’s actually pretty cool 
2 Likes
Since you’ve listed it as one of factors - what changes when disk is encrypted?
With a method like rsync it won’t make much difference outside of the encryption overhead.
With some disk/partition cloning methods it can make a difference because it often means there is no way to optimize for free space in a partition.
1 Like
i use gparted copy and paste one partition from one drive to another. only thing is after the copy you’ll want to change the uuid on the new drive of the just copied partition so as not to confuse your system…
1 Like
Sounds more like disk cloning, than copy
1 Like
easy peasy
I might not remember it correctly but I think copy-paste partitions in Gparted would copy the UUID of the source to the target as well.
Hence the directive to change it after you finish copying.
2 Likes
Looks like I misread your post!
1 Like