How to clean up output of linux 'script' command

Question

I'm using the linux 'script' command http://www.linuxcommand.org/man_pages/script1.html to track some interactive sessions. The output files from that contain unprintable characters, including my backspace keystrokes.

Is there a way to tidy these output files up so they only contain what was displayed on screen?

Or is there another way to record an interactive shell session (input and output)?

Answer 1

If you want to view the file, then you can send the output through col -bp; this interprets the control characters. Then you can pipe through less, if you like.

col -bp typescript | less -R

On some systems col wouldn't accept a filename argument, use this syntax instead:

col -bp <typescript | less -R

Answer 2

cat typescript | perl -pe 's/\e([^\[\]]|\[.*?[a-zA-Z]|\].*?\a)//g' | col -b > typescript-processed

here's some interpretation of the string input to perl:

• s/pattern//g means to do a substitution on the entire (the g option means do the entire thing instead of stopping on the first substitute) input string

here's some interpretation of the regex pattern:

• \e match the special "escape" control character (ASCII 0x1A)
• ( and ) are the beginning and end of a group
• | means the group can match one of N patterns. where the N patterns are
  • [^\[\]] or
  • \[.*?[a-zA-Z] or
  • \].*?\a
• [^\[\]] means
  • match a set of NOT characters where the not characters are [ and ]
• \[.*?[a-zA-Z] means
  • match a string starting with [ then do a non-greedy .*? until the first alpha character
• \].*?\a means
  • match a string that starts with ] then do a non-greedy .*? until you hit the special control character called "the alert (bell) character"

Answer 3

For a large quantity of script output, I'd hack a perl script together iteratively. Otherwise hand edit with a good editor.

There is unlikely to be an existing automated method of removing control characters from script output in a way that reproduces what was displayed on the screen at certain important moments (such as when the host was waiting for that first character of some user input).

For example the screen might be blank except for Andrew $, if you then typed rm /* and pressed backspace twelve times (far more than needed), what gets shown on the screen at the end of that depends on what shell was running, what your current stty settings are (which you might change partway through a session) and probably some other factors too.

The above applies to any automated method of continuously capturing input and output. The main alternative is taking "screen shots" or cutting and pasting the screen at appropriate times during the session (which is what I do for user guides, notes for a day-log, etc).

Answer 4

An answer to the second part of my question is to use the logging facility in gnu screen: ^A H from within a running screen session. The documentation is at http://www.gnu.org/software/screen/manual/screen.html#Logging

Answer 5

If what you're after is to record your commands (e.g. to later turn them into a bash script), then a reasonable hack is to run script(1), then inside it run

bash -x

Afterwards grep the output file (usually "typescript") looking for lines starting with a "+". The regular expression ^\+ will do the trick.

Answer 6

I used cat filename which removes control characters :-)

Answer 7

If you want to write the output to a file:

col -bp < typescript >>newfile

use unix2dos command to convert file to Windows format if you want

Answer 8

col -bp processes the backspaces as desired (AFAIK). But it mangles the color escape sequences. It might be good to remove the color sequences first, then process the backspaces, if possible.

This is a very common need, and I'm surprised there are not more solutions to it. It is extremely common to script a session, then somebody has a need to review the procedure. You want to cut out all the little typing mistakes, and color escape sequences to create a "clean" script of the procedure for future reference. Simple ASCII text preferred. I think this is what is intended by "human readable", and it is a very reasonable thing to do.

Answer 9

https://github.com/RadixSeven/typescript2txt was written to solve this problem.

It's been 4 years since I last updated/used it, but I don't remember doing anything fancy that shouldn't still work today.

Answer 10

I found the answer that dewtall provided to a similar question on the Unix board to be more effective at removing control characters from the output of script if you are in an environment where Perl is available to you.

dewtall's script:

#!/usr/bin/perl
while (<>) {
    s/ \e[ #%()*+\-.\/]. |
       \r | # Remove extra carriage returns also
       (?:\e\[|\x9b) [ -?]* [@-~] | # CSI ... Cmd
       (?:\e\]|\x9d) .*? (?:\e\\|[\a\x9c]) | # OSC ... (ST|BEL)
       (?:\e[P^_]|[\x90\x9e\x9f]) .*? (?:\e\\|\x9c) | # (DCS|PM|APC) ... ST
       \e.|[\x80-\x9f] //xg;
       1 while s/[^\b][\b]//g;  # remove all non-backspace followed by backspace
    print;
}

To remove the control characters:

./dewtalls-script.pl < output-from-script-that-needs-control-characters-removed

Answer 11

I found a good way to do it. On my system, long output lines are sprinkled with " ^M" (blank space followed by carriage return). The "^M" can be nicely replaced with the null character "^@", which does not display at all when you cat the file.

I capture timing too, so in order to replay the file perfectly, I cannot simply remove " ^M" completely using the commands below (because scriptreplay counts bytes):

tr '\r' '\0' | sed 's/ \x0//g'

I run my script command like this:

script -t -f session.log 2>timing

So, what I do afterwards is:

cat session.log | tr '\r' '\0' > typescript 
scriptreplay -t timing | sed 's/ \x0//g'

The first edit (before replay) retains the number of bytes in the file. The second edit (after the replay) gets rid of white space in random places. (Note that by default scriptreplay looks for input file named "typescript", which is why I did not provide it after "timing".)

Answer 12

dos2unix on the output will also do the trick

Answer 13

One other solution is to use strings which prints only printable characters from a file (or from standard input):

strings -n 1 filename

The -n 1 option sets the minimum length of the sequences to be preserved to one and thus makes sure even single printable characters surrounded by non-printable characters are preserved.

One possible downside of this approach is that strings adds line breaks between contiguous strings of printable characters. For instance a file with content

Foo<SOMECONTROLCHAR>Bar

(where <SOMECONTROLCHAR> is control character or any other non-printable character) would be returned as

Foo
Bar

Another issue brought up in the comments is that some sequences of control characters consist of a combination of both printable and non-printable characters and this approach would only remove part of those.

However, strings does a good job of removing control characters like the backspace mentioned in the question.