Rsync to NAS

General system Applications
1

I’m trying to backup some folders to my NAS, but I have some doubts about rsync usage.

1- I have two methods of doing it, first is using a SMB mount, thus using /mnt/folder in the rsync command.
The other method is using the NAS credentials to login, user@nas:/folder

Either way of doing it, the user in my NAS is not the same that I have in my local system, would any of these methods cause any problems that a chown -R couldn’t fix later once I restore these folders ?

2- I want to sync folders exactly as they are, removing any files in the destination that doesn’t exist in the source folder, would this be a good option for it ?

rsync -rva --delete local_folder /mnt/destination_folder or user@nas:/volume1/destination_folder be enough, or any other options that I could include in this rsync command ?

I’m using:
r as recursive to copy folders too.
v for verbose.
a to keep metadata.
–delete to erase files in the destination folder that doesn’t exist in the local folder.

Any help is much appreciated, started with rsync today.

2

What kind of NAS do you have? Just curious.

3

Synology DS218+
I’m trying to backup firefox and thunderbird profiles folder.

Edit: I’m mounting these folders through fstab using credentials for a user that only exists in the NAS.

4

I enabled the rsync daemon on my synology. I sync files with this:

rsync -avhW /directory/to/sync/ host:/volume1/destination
3 Likes
5

Thanks for sharing, Dalto.
hmmm, let me try that command to compare, I just did a test and the files are getting there with the wrong permissions, but the owner seems to be correct.

Command used:

rsync -rva --delete /home/myuser/Documentos2/ /mnt/myuser/Backup_Data/EOS_GNOME/EOS_BACKUP/Documentos2/

Permission of the original file in my system:
-rw-r–r-- 1 myuser myuser 73 jul 14 15:52 switch_study.txt

Permission of the file after the rsync, in the NAS:
-rwxr-xr-x 1 myuser myuser 73 jul 14 15:52 switch_study.txt

Edit:
Testing your method now, one sec

1 Like
6

With your method, I got exactly the same output regarding the permissions of the files.
But strange, it copied folders inside Documentos2 without the use of -r option ?

This -W option, I didn’t get it what it means, according to man rsync:
‐‐whole‐file, ‐W copy files whole (w/o delta‐xfer algorithm)

7

Might be a silly question but do you have the rsync service and rsync account enabled on the NAS?

8

There is no silly question, you are trying to help me and I’m pretty thankful for this.

Yes, I do, it is working but it is messing with the permissions of the files.

Now I’m going to try to restore, doing the inverse, to check if the permissions will be fixed, perhaps it works without the need to fix them later.

1 Like
9

Yeah, unfortunately I’ve never ran it from Linux machine>NAS
I’ve only ever run it either from two Synology NAS machines or to a windows machine. Would be cool to see if you get it working, I’d love to learn how to do this.

1 Like
10

Just restored that folder, erased all contents in my system then restored from my NAS by just inverting the command.

It copied the files but permissions are exactly as it is in the NAS, thus changing from the original.

It is changing from 644 to 755, one sec, brb.

1 Like
11

If you want to preserve permissions I would try using an nfs mount.

1 Like
12

I’ll try it.
Unfortunately I will only be able to test tomorrow…

13

Generally speaking I use rsync -avSHX this preserves ownership and permissions.

2 Likes
14

Thanks for sharing nadb.

I thought that I wouldn’t have enough time to test this today, but I was wrong… :slight_smile:
NFS3 is working, and now, permissions don’t change anymore.

So, it is NFS mount with rsync to send/receive.

1- Created an user in the NAS, myuser, with the same name of the user I have in my system (EnOS)
2- Created a group with the same name as my user; myuser:myuser in the NAS.
3- In the NAS, added myuser to myuser group.
4- Enabled NFS3 and rsync in the NAS.
5- Created a shared folder in the NAS and gave NFS permission to EnOS IP address to it, also gave permission to myuser:myuser
6- Created an entry in EnOS fstab like this:

192.168.255.252:/volume1/EOS_BACKUP /mnt/EOS_BACKUP  nfs  _netdev,noauto,x-systemd.automount,x-systemd.mount-timeout=30,timeo=14,x-systemd.idle-timeout=20min,vers=3 0 0

Tested and permissions are working perfectly now.

But I enabled a few options that I’m not sure if are necessary, I have some testings to do…

I don’t think I need to give permission to that shared folder in the NAS to myuser:myuser, perhaps just the NFS permission is enough ? I’ll test.

I enabled this option in the NAS, but perhaps is not necessary ?
Allow users to access mounted subfolders： Checking this option allows NFS clients to access mounted subfolders.

Observations so far:

rsync from EnOS to the NAS, files there in the NAS get this strange gid:
-rw-r--r-- 1 1026 65538 73 jul 14 19:41 switch_study.txt

myuser@192.168.255.252:/$ cat /etc/passwd | grep 1026
myuser:x:1026:100::/var/services/homes/myuser:/bin/sh

But I don’t think this is a problem because during my testing, I erased my local folder and ran a rsync to get the files from the NAS to EnOS (just inverted the paths in the rsync command), and boom, permissions were correct.

By the way, followed these guides:
https://kb.synology.com/en-br/DSM/help/DSM/AdminCenter/file_share_privilege_nfs?version=7
https://wiki.archlinux.org/title/NFS#top-page

1 Like
15

So, you are using these options that I’m not using.

‐‐sparse, ‐S turn sequences of nulls into sparse blocks
‐‐xattrs, ‐X preserve extended attributes
‐‐hard‐links, ‐H preserve hard links

Trying to understand rsync, too many options for my head right now, I need coffee :joy:

16

Yeah I generally use this on systems when I need to create a temporary directory to copy stuff to. Essentially when adding for instance external storage/nfs and needing to mount it on lets say /opt/ihavestuffhere well the stuff I have there needs to go back exactly the way it is pulled out. Hence the options. This rsyncs the stuff over to a new temporary directory. I nuke the stuff in the existing directory. Mount the share, and then rsync the stuff back. In general though options are just good practice especially when dealing with file systems with extended attributes or for instance SELinux enabled systems which makes extensive use of such for labeling the files. Also the hardlinks option makes sure that they exist in the copy and if need be you will bring them back over to the originating system…instead of wondering why something might be broken after needing to restore it. Like I said just a good habit.

1 Like
17

I agree, I’m trying to upgrade my reflexive response from -aHv to something that covers more options so I don’t have to go reading the manual every time something a little different comes along. Rsync manual is interesting, but way to complex to want to visit trivially.

@mcury that 65534 group is probably nobody, and may be getting remapped by NFS since you don’t have a matching group on the NAS? Always easier when the uids/gids match over NFS, not not necessarily easy to setup on something like Synology.

Have you checked if is a nfs3 or nfs4 mount in /proc/mounts? IIRC they have different ways of dealing with id mismatch, and something on my todo list to learn more about for my own environment, but I’m more focussed on think like libvirt kvm etc system user/groups and I would like them to match, which supposedly I can to with nfs4 if I poke it just the right way.

2 Likes
18

What do you mean with “not the same”? username or UID? rsync is using the UID. What is the UID of the files in src and dest?

Do you use rsync as a user or as root? I always use rsync as root to be sure that rsync can do all the chmod/chown on the files.

-r is not needed if you use -a. -a is equivalent to -rlptgoD
I always use -aAxXhH

--archive, -a            archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
--acls, -A               preserve ACLs (implies --perms)
--one-file-system, -x    don't cross filesystem boundaries
--xattrs, -X             preserve extended attributes
--human-readable, -h     output numbers in a human-readable format
--hard-links, -H         preserve hard links

And I always use the syntax
sudo rsync -aAxXhH src-folder/ root@destination:/dest-folder/`
This makes sure that rsync is run natively on the destination machine. I remember a couple of issues with NFS file permissions resp. ownership in the past.

Again, I use root to do the rsync because I backup files form different users. This might be too much for you, but I would recommend that you do use root at least for the trouble shooting process.

1 Like
19

rsync over ssh removes the hassle of the NFS mount if you aren’t using NFS already, and removes NFS uid/gid interpretation from the situation too.

I use rsnapshot, which uses rsync over ssh, for a series of backups nightly, from multiple different machines, and it has worked well for years.

2 Likes
20

According to documentation linked above, the option Map all users to admin from Squash is necessary, so I’m suspecting that this UID/GID is somehow related to it.

I forced NFS 3 mount through fstab, and NAS NFS version is set to 3.
It seems that NFS4 is not compatible with sec=sys and only NFS3 is.

Either way of doing it I meant, using rsync directly to the mounted folder in my EnOS, or using rsync username@NAS_IP:/path_to_folder.

Mounted folder is using SMB credentials, and username@NAS_IP is using SSH credentials for my user there in the NAS. UID and GID are not the same in both locations, I suppose I need to configure id_mapping in /etc/idmapd.conf ?

hmm, I’ll try to use root next time, and thanks for -r insight.

hmm, something new to try, first time I hear about rsnapshot, I’ll try all the options suggested here in this post and will post my findings.