Invoking vi through find | xargs breaks my terminal. Why?

139

32

When invoking vim through find | xargs, like this:

find . -name "*.txt" | xargs vim

you get a warning about

Input is not from a terminal

and a terminal with pretty much broken behaviour afterwards. Why is that?


This question was explicitly about the why, not about the how to avoid. This was asked, and answered, elsewhere.

DevSolar

Posted 2011-09-15T16:26:21.960

Reputation: 3 860

Related: grep -l .. | xargs vim generates a warning, why? at unix SE

– kenorb – 2015-02-14T12:24:56.007

Related: Terminal borked after invoking Vim with xargs at Vim SE.

– kenorb – 2015-02-19T15:18:29.627

GitHub bug report: vim does not handle STDIN set to /dev/null.

– kenorb – 2017-11-16T11:54:25.310

11Side note: You can perform this operation entirely within vim, not using find or xargs at all. Open vim with no arguments, then run :args **/*.txt<CR> to set vim's arguments from inside the editor. – Trevor Powell – 2013-03-21T13:26:31.180

3@TrevorPowell: In all these years, vim never ceased to amaze me. – DevSolar – 2013-03-21T13:52:31.520

Answers

103

When you invoke a program via xargs, the program's stdin (standard input) points to /dev/null. (Since xargs doesn't know the original stdin, it does the next best thing.)

$ true | xargs filan -s
    0 chrdev /dev/null
    1 tty /dev/pts/1
    2 tty /dev/pts/1

$ true | xargs ls -l /dev/fd/

Vim expects its stdin to be the same as its controlling terminal, and performs various terminal-related ioctl's on stdin directly. When done on /dev/null (or any non-tty file descriptor), those ioctls are meaningless and return ENOTTY, which gets silently ignored.

  • My guess at a more specific cause: On startup Vim reads and remembers the old terminal settings, and restores them back when exiting. In our situation, when the "old settings" are requested for a non-tty fd (file descriptor), Vim receives all values empty and all options disabled, and carelessly sets the same to your terminal.

    You can see this by running vim < /dev/null, exiting it, then running stty, which will output a whole lot of <undef>s. On Linux, running stty sane will make the terminal usable again (although it will have lost such options as iutf8, possibly causing minor annoyances later).

You could consider this a bug in Vim, since it can open /dev/tty for terminal control, but doesn't. (At some point during startup, Vim duplicates its stderr to stdin, which allows it to read your input commands – from a fd opened for writing – but even that is not done early enough.)

user1686

Posted 2011-09-15T16:26:21.960

Reputation: 283 655

20+1, and for TL;DR people just run stty sane – doc_id – 2015-02-11T10:33:46.457

@rahmanisback: The other answers, plus Trevor's comment, all provided ways to avoid terminal breakage in the first place. I accepted grawity's answer, because my question was "why", not "how to avoid" -- that's covered by another question that actually spawned this one.

– DevSolar – 2015-02-11T10:41:19.517

@DevSolar Understood, but think about frustrated people like me who just google how to get rid of that behavior while not -unfortunately - have enough time right now to study "why", which is very interesting nonetheless. – doc_id – 2015-02-11T18:34:49.023

4when my terminal breaks, like this, i use reset instead of stty sane and it works fine after that. – Capi Etheriel – 2015-04-21T17:19:03.213

140

(Following on from grawity's explanation, that xargs points stdin to /dev/null.)

The solution for this problem is to add the -o parameter to xargs.  From man xargs:

-o

      Reopen stdin as /dev/tty in the child process before executing the command.  This is useful if you want xargs to run an interactive application.

Thus, the following line of code should work for you:

find . -name "*.txt" | xargs -o vim

GNU xargs supports this extension since some release in 2017 (with the long option name --open-tty).

For older or other versions of xargs, you can explicitly pass in /dev/tty to solve the problem:

find . -name "*.txt" | xargs bash -c '</dev/tty vim "$@"' ignoreme

(The ignoreme is there to take up $0, so that $@ is all arguments from xargs.)

James McGuigan

Posted 2011-09-15T16:26:21.960

Reputation: 1 501

2How would you create a bash alias out of this? $@ doesn't seem to be translating arguments correctly. – zanegray – 2015-08-31T15:47:27.860

1@zanegray -- you can't make an alias, but you can make it a function. Try: function vimin () { xargs sh -c 'vim "$@" < /dev/tty' vim; } – Christopher – 2017-02-12T18:02:37.353

For a detailed explanation of how the GNU xargs solution works, and why you need the dummy ignoreme string, see https://vi.stackexchange.com/a/17813

– wisbucky – 2018-11-02T20:42:59.093

@zanegray, You can make it an alias. The quotes are tricky. See solution at https://vi.stackexchange.com/a/17813

– wisbucky – 2018-11-02T20:46:54.697

The -J, -o, -P and -R options are non-standard FreeBSD extensions which may not be available on other operating systems. (It was not available on macOS for me because I installed xargs from homebrew (the GNU one)) – localhostdotdev – 2019-05-07T16:56:59.713

GNU xargs has -o since some release in 2017.

– Chris Morgan – 2019-09-19T13:49:58.097

33

The easiest way:

vim $(find . -name "*foo*")

trolol

Posted 2011-09-15T16:26:21.960

Reputation: 347

5This, of course, does not work properly when filenames contain spaces or other special characters, and is also a security risk. – Dejay Clayton – 2015-06-17T16:14:36.027

1My favorite answer because it works for every command that lists files, not just "find" or wildcards. It does require a little trust, as Dejay points out. – Travis Wilson – 2016-09-01T20:50:11.727

1This is will not work with many use cases xargs is designed for: e.g., when the number of paths is very high (cc @TravisWilson) – Good Person – 2016-11-30T19:47:19.623

5The main question was "why", not "how to avoid it", and it's been answered to satisfaction two and a half years ago. – DevSolar – 2014-03-08T09:35:47.620

21

It should work just fine if you use the -exec option on find rather than piping into xargs.

find . -type f -name filename.txt -exec vi {} + 

Chris Wraith

Posted 2011-09-15T16:26:21.960

Reputation: 219

1Also, find is not the only way of getting a list of files that have to be edited simultaneously by vim. I can use grep to find all files with a pattern and try editing them at the same time as well. – Chandranshu – 2014-11-12T08:41:38.763

2Huh... the trick there is the + (instead of "the usual" \;) to get all the found files into one Vim session -- an option I keep forgetting about. You are right, of course, and +1 for that. I use vim $(find ...) simply out of habit. However, I was actually asking for why the pipe operation screws up the terminal, and grawity nailed that with his explanation. – DevSolar – 2013-03-10T18:45:53.277

2This is the best answer and it works on both BSD/OSX/GNU/Linux. – kevinarpe – 2014-02-24T17:17:20.600

8

Use GNU Parallel instead:

find . -name "*.txt" | parallel -j1 --tty vim

Or if you want to open all the files in one go:

find . -name "*.txt" | parallel -Xj1 --tty vim

It even deals correctly with filenames like:

My brother's 12" records.txt

Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

Ole Tange

Posted 2011-09-15T16:26:21.960

Reputation: 3 034

@grawity Using find | xargs -d '\n' works with filenames with spaces in them; it makes xargs only split on line-breaks, which works because find handily emits filenames on per line. (Obviously it wouldn't work with filenames that have line-breaks in them.) – Smylers – 2016-04-16T21:17:36.010

1Not ubiquitously available. Most of the day I am working on servers where I am not at liberty to install additional tools. But thanks for the hint anyway. – DevSolar – 2011-09-16T20:16:41.393

If your are at liberty to do 'cat > file; chmod +x file' then you can install GNU Parallel: It is simply a perl script. If you want man pages and such, you can install it under your homedir: ./configure --prefix=$HOME && make && make install – Ole Tange – 2011-09-20T09:09:12.910

2OK, tried that - but parallel does not open all the files, it does open them in succession. It's also quite a mouthful for a simple operation. vim $(find . -name "*.txt") is simpler, and you get all files opened at once. – DevSolar – 2011-09-20T11:00:32.473

5@DevSolar: Somewhat unrelated, but both find | xargs and $(find) will have big problems with spaces in file names. – user1686 – 2011-09-20T18:34:40.653

2@grawity Correct, but there is no easy way around it (that I know of). You'd have to start fiddling with $IFS, -print0 and stuff, and then you left the realm of a one-shot command line solution and reached a point where you should come up with a script... there's a reason why spaces in filenames are discouraged. – DevSolar – 2011-09-21T09:54:19.160

0

maybe not the best but here it the script I use (named vim-open):

#!/usr/bin/env ruby

require 'shellwords'

inputs = (ARGV + (STDIN.tty? ? [] : STDIN.to_a)).map(&:strip)
exec("</dev/tty vim #{inputs.flatten.shelljoin}")

will work with vim-open a b c and ls | vim-open for instance

localhostdotdev

Posted 2011-09-15T16:26:21.960

Reputation: 111

As for several other answers, note that the actual question was "why", not "how to avoid it". (For which I would still point to Trevor's comment under my question as the most solid way that doesn't require scripting, aliases or anything.) – DevSolar – 2019-08-02T09:25:08.473