How to arbitrarily map user/group ownerships in rsync

16

4

I need to rsync a directory to a remote server so that all files belonging to user X and group Y on the source (local) machine are mapped to user W and group Z on the destination (remote) machine. If possible by using ssh as the transport, but if I need to use the rsync daemon it's fine as well.

Is there a way to do that? I'm looking for a way to establish an arbitrary user/group map, such as

local user X => remote user W
local group Y => remote group Z
... and as many of these as needed.

This should be a pretty common use case, isn't it? E.g. I have files on my local computer where my username is X, and I need to upload them to a web server where they need to belong to a given user which doesn't have either the same name or the same UID as my user on my personal computer.

I can't find that on rsync's man page...

LINUX on both local and remote machine (Ubuntu local, centOS remote)

Command I tried: rsync -avz /path/to/local root@myhost.com:/path/to/remote

matteo

Posted 2013-11-07T17:41:34.093

Reputation: 3 092

1Please always include your OS. Solutions very often depend on the Operating System being used. Are you using Windows, Linux, Unix, OSX, BSD? Which version? – terdon – 2013-11-07T17:42:41.250

Answers

20

Rsync version 3.1.0 introduced the --usermap and --groupmap options precisely for that purpose. See the man page.

Michel Schinz

Posted 2013-11-07T17:41:34.093

Reputation: 316

This is good to know. There's definitely a use case scenario for this. For example, I've got two Icinga servers and I need to do a one-way sync of the config files from master to slave. But icinga on the master is a different uid than icinga on the slave. So, good to know rsync can handle this. – Michael Martinez – 2014-11-07T17:35:04.743

Version notes: Ubuntu v16.04 has v3.1.1, CentOS 7 has v3.1.2, but CentOS 6 has v3.0.6. This works great for newer distros. – jimp – 2018-12-04T22:03:54.960

4

Last version (at least 3.1.1) of rsync allows you to specify the "remote ownership":

--usermap=tom:www-data

Changes tom ownership to www-data (aka PHP/Nginx). If you are using Mac as the client, use brew to upgrade to the last version. And on your server, download archives sources, then "make" it!

Thomas Decaux

Posted 2013-11-07T17:41:34.093

Reputation: 140

for me it works as --chown=tom:www-data – Federico Galli – 2018-05-21T14:18:11.747

0

If you want to change ownership of files to arbitrary users, you'll first need to be root on the destination box.

I don't think there is such a feature integrated by rsync, but you can achive it by running a find after doing your rsync.

Maybe, a command like this will do the trick : For example, translate from UID 1000 => 505 and UID 1001 => 700 :

find /your/rsynced/path -user 1000 -exec chown 505 {} \;
find /your/rsynced/path -user 1001 -exec chown 700 {} \;

If you have many users, you may consider using a loop with a mapping, in your predilection language.

Have fun.

Adrien M.

Posted 2013-11-07T17:41:34.093

Reputation: 199

But this will work only once. When you run rsync 2nd time to sync changes, it will change the uids back... – Marki555 – 2014-10-17T13:28:23.163

Ok so there just isn't a way to have rsync do that. Astonishing, it seemed to me such an obvious need. – matteo – 2013-11-07T19:35:19.567

1@matteo careful if you use this. If there is a user with id 1000 or 1001 on the remote machine there may be files that are legitimately owned by them in the remote locations and chowning them could cause problems. – terdon – 2013-11-07T19:42:59.057

@terdon how would there be a user without a name on the remote machine? (because if the ID is shown instead of the name when files are listed, it means the user doesn't have a name, right?) – matteo – 2013-11-08T14:14:13.503

1@matteo no, every user has a user id as well as a name. In most Debian based systems (and probably others) the first, default user is user 1000. Try running id on your machine to see. Therefore, if you use the IDs like this, you are assuming that there is no user with the same ID on the remote machine and that is likely not true. – terdon – 2013-11-08T15:49:10.243

in my case there's no user with UID 1000 on the server, at least it is not listed in /etc/passwd. Which was what I guessed because when I list files with "ls -la", files belonging to 1000 (which were the result of rsync -avz from my local machine where user 1000 is my default user "teo") show up with "1000" as the owner. If there was a user with that id, and its name was, say "Someone", then the owner of the files would be shown as "Someone" in the output of ls, right? Anyway that's a useful caveat to take into account, thanks. – matteo – 2013-11-08T19:22:57.333

-1

I am not sure I understand, to connect over ssh you need to provide a username. That username will be user W on the remote machine who belongs to group Z. Therefore, everything will be transferred exactly as you want it to be:

rsync /path/to/local userX@remote.com/path/to/remote

EDIT in answer to the OP's comment.

If you want to do this user mapping and not lose permissions settings, don't use -a. First, run the rsync with mywww@myhost.com to get the right username. Then, instead of -a which will cause you to preserve ownership, specify the options manually. From man rsync:

    -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
    -r, --recursive             recurse into directories
    -l, --links                 copy symlinks as symlinks
    -p, --perms                 preserve permissions
    -t, --times                 preserve modification times
    -g, --group                 preserve group
    -o, --owner                 preserve owner (super-user only)
    -D                          same as --devices --specials

So, -a activates all of the above options but you don't want to preserve the group or owner. This command should do what you want:

rsync -rlptDvz /path/to/local mywww@myhost.com:/path/to/remote

Since you are now logging in as mywww and no longer preserving owner/group information, the copies made by rsync will belong to the mywww user.

terdon

Posted 2013-11-07T17:41:34.093

Reputation: 45 216

No, I do rsync -avz /path/to/local root@myhost.com:/path/to/remote. My question is, how do I get files belonging to user "x" on my local machine to be remapped to user "w" on the server? On my computer, my files belong to "teo". On the server there is not even a user called "teo", I need the user to be the one that apache uses, say "mywww"; and the same for the group. By default, rsync will try to preserve the owner as "teo", and since there's no such user, it will preserve the UID, resulting in files belonging to user "1000" which is a nonexisting user on the server – matteo – 2013-11-07T18:01:35.727

Btw, at the same time I need to preserve file permissions which it already does as per the -a option. I mean, any solution that will make me loose the ability to preserve file permissions (i.e. a 777 file remain a 777 and a 600 remain a 600) won't do the trick – matteo – 2013-11-07T18:04:09.070

@matteo that's because -a enables the preserve owner and group options. See my updated answer. – terdon – 2013-11-07T18:18:10.320

That would only work provided that I wanted ALL files to belong to mywww. AND I first need to change the owner of, or delete, files already present on the server (many of which have wrong owner) because otherwise the mywww user won't have the permissions to overwrite and chown them. I was looking for a solution that would me let connect as root, and just MAP arbitrary owners to arbitrary owners. So that files on the server would be overwritten, and both owner and permissions changed, but mapping given users/groups on the source to given users/groups on the target – matteo – 2013-11-07T19:31:53.100

say for example I want teo->mywww and root->root – matteo – 2013-11-07T19:32:32.697

1@matteo please explain what you actually need when you ask questions. That way we don't waste time providing incomplete answers. Next time, remember to i) include the actual commands you used (rsync does not preserve anything by default, your use of -a did that) ii) clearly explain all constraints that you have, you just mentioned a single user until your last comment and iii) mention your OS. Anyway, the way to do this with rsync would be to copy one set of files as mywww@remote and the next set as root@remote. – terdon – 2013-11-07T19:41:47.847

I think my question was pretty clear; it was intentionally a general (yet clear, I think) question: map arbitrary userS and groupS to arbitrary userS and groupS, plural (see where I say "...as many as needed") so I intentionally omitted the specific use case precisely to prevent people from wasting their time trying to suggest workarounds that would address one specific use case but wouldn't answer the question. Sorry for omitting the OS and the commands used, anyway, and for being inaccurate in my comments when not mentioning options being used. – matteo – 2013-11-08T14:26:10.220

1@matteo what was really not clear was that you were using -a which preserves ownership. The examples you gave suggested you only needed to do this for one user. Anyway, I would recommend you use rsync, just make a list of the different files you want to copy for each user and run multiple rsyncs connecting to the remote machine with the relevant username. That will be simpler and safer. – terdon – 2013-11-08T16:00:47.363

That would be an interesting solution if I could get the lists of files automatically. That is, take all files and directory recursively under a given path filtered by owner, build a file list, pass that list to rsync. However I think that would still not allow to overwrite files on the server that already exist with the "wrong" owner (or whose owner has been changed later on the local machine). – matteo – 2013-11-08T19:29:39.467