command limits when pasting into tcsh (mac OS X)

Question

I encountered a 1024 character limit problem when pasting commands into the terminal application (max OS X 10.5.8) running the TC shell (/bin/tcsh).

This can consistently be reproduced by pasting the following commands into the terminal app:

echo       0       0       0       0       0
echo       1       1       1       1       1
echo       2       2       2       2       2
echo       3       3       3       3       3
echo       4       4       4       4       4
echo       5       5       5       5       5
echo       6       6       6       6       6
echo       7       7       7       7       7
echo       8       8       8       8       8
echo       9       9       9       9       9
echo      10      10      10      10      10
echo      11      11      11      11      11
echo      12      12      12      12      12
echo      13      13      13      13      13
echo      14      14      14      14      14
echo      15      15      15      15      15
echo      16      16      16      16      16
echo      17      17      17      17      17
echo      18      18      18      18      18
echo      19      19      19      19      19
echo      20      20      20      20      20
echo      21      21      21      21      21
echo      22      22      22      22      22
echo      23      23      23      23      23
echo      24      24      24      24      24
echo      25      25      25      25      25

At first, it works as expected, but at some point it fails:

myself@mymac|~> echo       1       1       1       1       1
1 1 1 1 1
myself@mymac|~> echo       2       2       2       2       2
2 2 2 2 2
etc. etc. (good)
myself@mymac|~> echo      21      21      21      21      21
21 21 21 21 21
myself@mymac|~> echo      22      22      22      echo      22      22      22      echo       23      23
22 22 22 echo 22 22 22 echo 23 23
echo      24      24      24      24      24
echo      25      25      25      25      25
(bad - gibberish)

So it gets confused after a while. In fact, I have 44 characters per line plus one for line breaks, and the problems happen on line 23 at character 34. This means, the problems happens after 45*22+34=1024 characters. That is pretty obviously a limit that is set somewhere... anybody knows how to expand it so that it can accept more characters and lines?

I tried the same on a Linux machine with tcsh but I had no troubles. So I am pretty sure this can be tweaked somewhere.

Answer 1

The limit you are encountering seems like the tty type-ahead buffer. It is “hard coded” in the Mac OS X kernel.

• xnu-1504.9.17/bsd/kern/tty.c: look for “input_overflow:”; uses MAX_INPUT
• xnu-1504.9.17/bsd/sys/syslimits.h: MAX_INPUT is #defined as 1024

xnu-1504.9.17 corresponds to 10.6.5, but these bits have not changed since at least 10.4.

If you just want to run the commands and you do not care that the input and the output might end up interleaved in odd ways, then you could temporarily disable tcsh’s command line editing with unset edit before you paste (re-enable it with set edit). This switches the tty to “canonical mode” where the input is line-based. This makes it easier for the shell to “keep up with” the incoming pasted text.

Not all tty-based programs have this issue. GNU Emacs 21.2.1, Vim 7.3, and bash 4.1 all seem to be able to “keep up with” large pastes (I tried 8201 bytes; Vim and bash were slow, but they accepted the whole paste without any problem).