xargs argument redirection

Question

This seems to be a trivial problem but the solution eludes me ever so cautiously.

I want to clear out many text files; log files. Reason? To save on disk space.

With one text file, this is as trivial as echo '' > path/to/file.txt

However, the files are many. I have resorted to using find and xargs. But I don't know how to get around "input redirection".

I have tried find . -name <regex> | xargs -I target echo '' > target and echo '' > { find . -name <regex> | xargs -I target target }; none worked

I am not well versed in shell-scripting do any help is appreciated.

Thanks.

Answer 1

What you've done in both your examples is put the > where the current shell can see it, so the redirection is being done only once, before the find and xargs commands run. That's your first problem.

Your second problem is that if you quote the > so that it's passed through to xargs, it still won't work because xargs doesn't pass your command through a shell unless you ask it to.

Your third problem is that if you do tell xargs to use a shell to run the command, the shell will do the wrong thing if any filenames have funny characters in them. (Also xargs does the wrong thing with funny characters itself, but that's fixable with -0.)

Other problems, which you may not care about yet, include:

• echo '' doesn't create an empty file, but a file containing a newline.
• the -name option takes a glob, not a regex.
• you might want to add -type f just in case any directories match the glob.

Here's a partially corrected version of the command:

find . -name '*thisisaglob*' -type f -print0 |
xargs -0 -I target sh -c ': > target'

That fixes most of the problems I mentioned. Still there remains the problem of the shell misinterpreting a filename containing shell metacharacters. To fix that, you'd have to give the filename to the shell as a parameter instead of as part of the -c command. That would look like this:

find . -name '*thisisaglob*' -type f -print0 |
xargs -0 -I target sh -c ': > "$1"' fnord target

The "fnord" is a placeholder. It becomes $0 which we don't need.

Now having accomplished the goal of using xargs and redirection together safely, I'll show you how to achieve your goal by using neither of them.

find . -name '*thisisaglob*' -type f -exec truncate -s 0 '{}' +

This requires the truncate command, which is part of GNU coreutils and not a unix standard utility, so it's less portable, but much easier to read, isn't it?

Answer 2

While Alan Curry's answer is thorough, precise and very knowledgeable, I have one question. Why do you want the files to continue to exist, even empty?

My natural recommendation would be:

find . -name <glob> -print0 | xargs -0 rm

If you needed the files to exist for some reason, you could do a series of commands:

find . -name <glob> -print0 | tee /tmp/filelist | xargs -0 rm
cat /tmp/filelist | xargs -0 touch

If you simply wanted to compress the files (e.g. if they were mostly redundant / repetitive information, but you wanted to save space and keep the files), you might try:

find . -name <glob> -print0 | xargs -0 tar -xzf /tmp/logfiles.tgz

Answer 3

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

find . -name '*thisisaglob*' -type f | parallel '>'

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1