xargs split at newlines not spaces

Question

here is my problem in short

$ echo 'for i in $@; do echo arg: $i; done; echo DONE' > /tmp/test.sh
$ echo "ac\nbc\ncc\n" | xargs bash /tmp/test.sh 
arg: ac
arg: bc
arg: cc
DONE

Which is what i expect, but

$ echo "ac s\nbc s\ncc s\n" | xargs -d \n bash /tmp/test.sh
arg: ac
arg: s
arg: bc
arg: s
arg: cc
arg: s
DONE

Shouldn't the output be?

arg: ac s
arg: bc s
arg: cc s
DONE

How do I get the 2nd output with xargs?

mklement0 · Accepted Answer · 2018-10-08 12:50:40Z

Try:

printf %b 'ac s\nbc s\ncc s\n' | xargs -d '\n' bash /tmp/test.sh

You neglected to quote the \n passed to -d, which means that just n rather than \n was passed to xargs as the delimiter - the shell "ate" the \ (when the shell parses an unquoted string, \ functions as an escape character; if an ordinary character follows the \ - n in this case - only that ordinary character is used).

Also heed @glenn jackman's advice to double-quote the $@ inside the script (or omit the in "$@"part altogether).

Also: xargs -d is a GNU extension, which, for instance, won't work on FreeBSD/macOS. To make it work there, see @glenn jackman's xargs -0-based solution.

Note that I'm using printf rather than echo to ensure that the \n instances in the string are interpreted as newlines in all Bourne-like shells:
In bash and ksh^[1], echo defaults to NOT interpreting \-based escape sequences (you have to use -e to achieve that) - unlike in zsh and strictly POSIX-compliant shells such as dash.
Therefore, printf is the more portable choice.

^{[1] According to the manual, ksh's echo builtin exhibits the same behavior as the host platform's external echo utility; while this may vary across platforms, the Linux and BSD/macOS implementations do not interpret \ escape sequences by default.}

glenn jackman · Answer 2 · 2014-04-17 21:01:45Z

Your shell script needs to use "$@" not $@

See http://www.gnu.org/software/bash/manual/bashref.html#Special-Parameters

I see in the xargs manual on my machine:

xargs reads items from the standard input, delimited by blanks [...] or newlines

(emphasis mine)

Thus:

$ echo $'ac s\nbc s\ncc s\n' | xargs bash /tmp/test.sh  
arg: ac
arg: s
arg: bc
arg: s
arg: cc
arg: s
DONE

$ printf "%s\0" "ac s" "bc s" "cc s" | xargs -0 bash /tmp/test.sh
arg: ac s
arg: bc s
arg: cc s
DONE

With the former, you get the equivalent of

bash /tmp/test.sh ac s bc s cc s

versus using null-separator

bash /tmp/test.sh "ac s" "bc s" "cc s"

You need to be clear about what the delimiter is with xargs when the data contains whitespace.

$ printf "%s\n" "ac s" "bc s" "cc s" | xargs -d $'\n' bash /tmp/test.sh
arg: ac s
arg: bc s
arg: cc s
DONE

$ echo $'ac s\nbc s\ncc s\n' | xargs -d $'\n' bash /tmp/test.sh  
arg: ac s
arg: bc s
arg: cc s
arg:  
DONE

Note the extra arg in the last case, echo already adds a newline, so you don't need an extra one unless you use echo -n

Good advice, though the real problem lies with not quoting the \n passed to -d. — mklement0, Apr 17 '14 at 20:47
You're absolutely right: both issues need to be addressed to fix the problem (and they each in isolation or combination produce the symptom). Your printf-\0-xargs -0 solution is a nice alternative, because it is more portable (will work on OSX too, for instance, where xargs -d is not available). — mklement0, Apr 17 '14 at 21:01

티스토리

xargs 로 넘기니 스페이스(space.. 빈 공백하나) 에 대해 split 이 일어나더라?

xargs split at newlines not spaces

2 Answers