Are there limits to file length piped from find to xargs or using find -exec +

4

In question [ Find and delete all the directories named "test" in linux ] on this site, the best answer talks about using these two commands:

find . -name test -type d -print0|xargs -0 rm -r --
find . -name test -type d -exec rm -r {} +

because they will call rm with a list of directory instead of invoking it many times individually.

Since I cannot comment there due to low reputation, I ask here in a new question:

Is there any limit on the number of files that can be passed to rm using these techniques (aside from realistic system resource bounds)?

From the shell, a command like 'rm *' can exceed the shell's maximum command-line length, but do limits like that apply to this usage of find + or via a pipe to xargs?

simpleuser

Posted 2014-04-02T21:37:24.087

Reputation: 478

Answers

5

In short, no.

The long answer: - Find will run the command specified by exec for every match, so if your find turns up 20 files, it will run 20 seperate instances of rm. - xargs will determine the maximum command length for your shell and add arguments within these limits as you can see with the output of xargs --show-limits mtak@frisbee:~$ xargs --show-limits Your environment variables take up 4050 bytes POSIX upper limit on argument length (this system): 2091054 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 2087004 Size of command buffer we are actually using: 131072

mtak

Posted 2014-04-02T21:37:24.087

Reputation: 11 805

Using the + in find's -exec will cause it to pass a list rather than invoke rm for each file. It seems to pass ~128k of filenames at a time, so it apparently has similar logic to xargs. Thank you for pointing out that option to xargs! – simpleuser – 2014-04-02T22:45:24.937

Thank you for clarifying. I always use find with the -exec rm {} \; option. – mtak – 2014-04-02T22:46:35.597

I did too, and it's slow when deleting thousands of files, so I want to start using the + but was worried about it. Your answer about xargs made me think to write a test.pl to use instead of calling 'rm', which reported the number of args and their length -- so I could verify it behaved intelligently like xargs. – simpleuser – 2014-04-02T22:51:19.067

Asked: 2014-04-02T21:37:24.087

Viewed: 1 003 times

Active: 2014-04-02T22:11:34.360