rsync 'cannot delete non-empty directory' errors, even with --force option

Question

When running this command:

$ sudo rsync -r --delete --force --checksum --exclude=uploads /data/prep/* /data/app/

I'm getting the following output:

cannot delete non-empty directory: html/js/ckeditor/_source/plugins/uicolor/yui
cannot delete non-empty directory: html/js/ckeditor/_source/plugins/uicolor/yui
cannot delete non-empty directory: html/js/ckeditor/_source/plugins/uicolor
cannot delete non-empty directory: html/js/ckeditor/_source/plugins/uicolor
cannot delete non-empty directory: html/js/ckeditor/_source/plugins
cannot delete non-empty directory: html/js/ckeditor/_source/plugins
cannot delete non-empty directory: html/js/ckeditor/_source
cannot delete non-empty directory: html/js/ckeditor/_samples
cannot delete non-empty directory: html/js/ckeditor/plugins/uicolor/yui
cannot delete non-empty directory: html/js/ckeditor/plugins/uicolor/yui
cannot delete non-empty directory: html/js/ckeditor/plugins/uicolor

From reading the man rsync it was my impression that the --force option would tell rsync to delete these non-empty directories, which is the desired result.

Ref:

--force                 force deletion of dirs even if not empty

How can I modify the command to delete the non-empty directories?

I'm using rsync version 3.0.8, on Gentoo Base System release 2.0.3, in case that's relevant.

Update: Added sudo to the command to make it clear that this is not a file permissions issue.

What filesystem is the destination? Is the filesystem in need of repair? — Marcus Downing, Feb 21 '14 at 09:40
The file system is `ext3`. It's possible the filesystem could be in need or repair. I'll `fsck` at the next opportunity and update with results. — tommarshall, Feb 21 '14 at 10:24
I've just `fsck`'d the system and this issue is still present. — tommarshall, Feb 21 '14 at 11:30

Martin Höger · Accepted Answer · 2018-08-28T12:32:36.007

49

Did you try to add --delete-excluded?

If you delete a directory in your excluded folders on the "remote" side, rsync --delete will not delete the excluded folder on your "local" site.

edited Aug 28 '18 at 12:32

answered Mar 18 '14 at 13:08

Martin Höger

614
5
5

There was a folder called `uploads` in the `html/js/ckeditor/_source/plugins/uicolor/yui`, `html/js/ckeditor/_samples` and `html/js/ckeditor/plugins/uicolor/yui` directories. Thanks for your help. – tommarshall Jul 15 '14 at 08:46
1

if using `--delete-excluded` but you still want to exclude certain files/directories from deletion, you can put this files/directory in `--filter 'protect some_dir/'`, for example: `rsync -ac --delete --delete-excluded --exclude '*.html' --filter 'protect .git/' . /target/destination/` – alexandre1985 Oct 15 '19 at 13:20
`rsync ... ... ` – JesusIniesta Feb 18 '20 at 16:24
By "remote" I guess you actually mean "sending side", "local" means "receiving side", correct? You could be rsyncing _from_ local _to_ remote you know. – Greendrake Jun 11 '21 at 06:18
--delete-excluded is a dangerous command. The deletions will be permanent. – miguel Aug 16 '21 at 20:10

score 8 · Answer 2 · answered May 10 '14 at 11:21

Here are possible sources of this problem:

(1) This error can be the result of the -b (--backup) option. This option will create a backup of each deleted file, by appending a tilde (~) on its file name. (This confused me, as the file-name is clearly a backup, but the directory name is not, as you can't see the tilde.)

To check if this is the same case, read your target directory on the deepest level, and check if there is any tilde (~) ending file. Note that these tilde-appended-files-names are then invisible in some common file browsing system, so you may not see them.

To solve this case, prefer the --backup-dir=DIR option, for instance --backup-dir=.rsync_bak.

(2) The --exclude option can have the same results. Which is possibly happening in your case. The pattern system is powerful, but may be misleading. For instance if you write --exclude='*~', this will skip all tilde ending files, resulting exactly like in case (1) above.

from rsync man page:

if the pattern starts with a / then it is anchored to a particular spot in the hierarchy of files, otherwise it is matched against the end of the pathname

Si if you write --exclude=uploads, this will exclude all files named "updloads", at any level of your files tree.

Check if there is a file named "uploads" inside your unable-to-delete directories.

Solution would be to change "--exclude=uploads" to "--exclude=uploads/"

score 2 · Answer 3 · answered Feb 18 '19 at 10:07

2

Use rules in filter files instead of --exclude. These allow you to mark excludes as "persihable", which will let you delete non-empty directories which contain excluded files.

See this answer for details.

answered Feb 18 '19 at 10:07

mivk

3,457
1
34
29

score 0 · Answer 4 · answered Jan 04 '18 at 19:30

0

In my setup ((source is Ubuntu format type ext4 to target Western Digital type fuseblk) it works with :

rsync -a -v --progress --modify-window=1 -c -b -i -s -m --del -vv --ignore-errors --chmod=ugo=rwx --delete --delete-excluded  --exclude='*~'  --exclude='.*' --backup-dir=.rsync_bak /home/test /media/user/usbHDD

answered Jan 04 '18 at 19:30

InLaw

101
1

3

Which part of this actually solves the problem? – Michael Hampton Jan 04 '18 at 20:07

score -1 · Answer 5 · answered Feb 05 '14 at 15:52

-1

A directory has to be empty in order for you to delete it, the file system normally requires that.

Therefore normally rsync or rm would recursively delete all the contents first and only then delete the now-empty directory.

If the current user is not the owner of all files, the file systems permissions won't allow you to delete those files. Since they won't be removed, the directory isn't emptied and the delete will fail.

My first guess would be that some files in those directory are owned by another user e.g. the apache or nobody users.

answered Feb 05 '14 at 15:52

HBruijn

72,524
21
127
192

Thanks. The files within the directories are indeed owned by a user within the apache group, however I've just attempted to run this command as that user and got the same errors. – tommarshall Feb 05 '14 at 16:36
Also, if it was a file permissions issue I'd expect running the same command as with 'sudo' to resolve the issue, but this is not the case. However, please correct me on this if I am mistaken. – tommarshall Feb 05 '14 at 16:38
1

Running the command as root should overcome most permission issues, yes. (likely place where that would still fail is on a nfs mount to name one, an immutable file another) A simple check with `ls -la` to see why the directory is still not empty. `lsattr` will display immutable files. – HBruijn Feb 05 '14 at 19:49
Thanks for the additional info, however `lsattr` from one of the directories listed outputs the following: `--------------- ./assets` (no i flag), and the file is not on a nfs mount. Are there any other reasons you could think of why the command would still fail with sudo? – tommarshall Feb 06 '14 at 09:31
Are there any symlinks in there, or just real files? – Marcus Downing Feb 21 '14 at 09:34

rsync 'cannot delete non-empty directory' errors, even with --force option

5 Answers5

Linked