Newline-separated xargs

Question

Is it possible to make xargs use only newline as separator? (in bash on Linux and OS X if that matters)

I know -0 can be used, but it's PITA as not every command supports NUL-delimited output.

What is it that you're trying to accomplish. There may be another way to go about it. — Dennis Williamson, Mar 31 '10 at 04:50
@Dennis I use it for various things, often in pipeline with grep, sed, very basic awk etc., mostly to parallelize execution. `find -print0 -name \*.foo -maxdepth 1 | xargs -0 -P4` is way too much to type compared with `ls *.foo | xargs -P4`. — Kornel, Mar 31 '10 at 13:21
Scripts and functions are two excellent ways to reduce typing. — Dennis Williamson, Mar 31 '10 at 13:49

score 77 · Answer 1 · edited Apr 20 '16 at 13:41

77

GNU xargs (default on Linux; install findutils from MacPorts on OS X to get it) supports -d which lets you specify a custom delimiter for input, so you can do

ls *foo | xargs -d '\n' -P4 foo

edited Apr 20 '16 at 13:41

StackzOfZtuff

1,754
12
21

answered Apr 04 '10 at 12:42

James

7,553
2
24
33

I have `alias xxargs="xargs -d '\n'"` in my bashrc. So I can just do things like this: `grep -IRl foo | xxargs sed -i s/foo/bar/g` – tylerl Feb 13 '11 at 00:45
6

This really should be the accepted answer. No need to break out perl. Though `tr` is a good idea too. – Ehtesh Choudhury Dec 03 '12 at 18:36
6

`brew install findutils` and `gxargs` was exactly what I needed on OS X, thanks. – Brad Koch Mar 22 '16 at 19:00

score 23 · Accepted Answer · answered Mar 31 '10 at 01:39

23

Something along the lines of

alias myxargs='perl -p -e "s/\n/\0/;" | xargs -0'
cat nonzerofile | myxargs command

should work.

answered Mar 31 '10 at 01:39

hurfdurf

933
7
11

57

`tr '\n' '\0'` works too. – Kornel Apr 05 '10 at 15:26
3

perl is overkill for a single character. `tr` is faster both for the cpu and for your fingers. – Hello World Apr 23 '15 at 20:09
4

When the only tool you know is radioactive decay, everything looks like a nuclear reactor – Ray Foss Sep 11 '20 at 08:49

score 6 · Answer 3 · edited May 23 '17 at 11:33

6

With Bash, I generally prefer to avoid xargs for anything the least bit tricky, in favour of while-read loops. For your question, while read -ar LINE; do ...; done does the job (remember to use array syntax with LINE, e.g., ${LINE[@]} for the whole line). This doesn't need any trickery: by default read uses just \n as the line terminator character.

I should post a question on SO about the pros & cons of xargs vs. while-read loops... done!

edited May 23 '17 at 11:33

Community

1

answered Apr 01 '10 at 11:20

Charles Stewart

650
6
19

2

This can only emulate `xargs -n1 -d'\n'`, and not the whole `xargs`. – Hello World Apr 23 '15 at 20:12
@HelloWorld - You can bunch arguments together easily enough, so you can simulate `xargs -n$N` as well with only a bit of extra scripting. – Charles Stewart May 05 '15 at 16:33
Yes, I'm among the upvoters, it's just a remark worth mentioning. – Hello World May 09 '15 at 10:43
This is exactly what I do! Plus, you can use the `read` options, like reading more words in separate variables, or change delimiter with `IFS=' '`. – caesarsol Mar 13 '17 at 14:14
Reading the line into an array when you're expecting single lines is overkill at best, but more likely bug-inducing. When Bash reads into an array, it splits input on `$IFS` (normally whitespace), and concatenating the array with `${LINE[*]}` or `${LINE[@]}` does not restore that original whitespace. Using `while read -r line; do ...` is better for most situations, and lowercase variables avoid conflict with built-in/environment variables. – Walf Oct 26 '21 at 02:56

score 4 · Answer 4 · answered Jun 04 '20 at 22:02

4

$ echo "1\n2 3\n4 5 6" | xargs -L1 echo  "#"
# 1
# 2 3
# 4 5 6

answered Jun 04 '20 at 22:02

cmcginty

1,263
15
24

score 2 · Answer 5 · answered Oct 16 '20 at 18:08

please use the -d option of xargs

  --delimiter=delim, -d delim
Input items are terminated by the specified character. The specified delimiter may be a single character, a C-style character escape such as \n, or an octal or hexadecimal escape code. Octal and hexadecimal escape codes are understood as for the printf command. Multibyte characters are not supported. When processing the input, quotes and backslash are not special; every character in the input is taken literally. The -d option disables any end-of-file string, which is treated like any other argument. You can use this option when the input consists of simply newline-separated items, although it is almost always better to design your program to use --null where this is possible.

Example:

$ echo 'for arg in "$@"; do echo "arg: <$arg>"; done' > show_args
$ printf "a a\nb b\nc c\n"
a a
b b
c c
$ printf "a a\nb b\nc c\n" | xargs -d '\n' bash show_args
arg: <a a>
arg: <b b>
arg: <c c>

score -3 · Answer 6 · answered Dec 05 '12 at 06:32

-3

What about cat file | xargs | sed 's/ /\n/ig' this will convert spaces to newlines, using standard Linux bash tools.

answered Dec 05 '12 at 06:32

Omar Al-Ithawi

180
1
9

2

If `file` contains spaces and newlines, then the result of this command will make it impossible to distinguish between them, which seems to be going in the opposite direction of what this question asks for – Max Nanasy Sep 24 '18 at 19:07

Newline-separated xargs

6 Answers6