Configure .bashrc to save all commands and output

For a new project we are setting up a Linux box that will run several processes. To be better able to backtrack or rollback configurations we'd like to keep a persistent history of all commands entered on the terminal and via ssh as well as save all output to stdout/err to a logfile.

If we make the history persistent we could get something like this:

1  history
2  ls
3  ps
4  ll
5  echo "hi"
6  history

And save the corresponding outputs to logfiles named 1.log, 2.log, etc. Obviously the output shouldn't only be in the logfiles but displayed as well.

Visualized programatically I want that every command entered automatically behaves something like this:

command > /log/$(len(history)).txt 2&>; echo /log/$(len(history-1)).txt

but in a way where I can still see live what happens and interact with CLIs etc.

Is there any way to configure bash like this or any other way to get this behaviour in Linux for both SSH sessions and working locally?

jaaq

Posted 2019-09-10T14:32:43.870

Reputation: 117

Bash can have persistent history, and also ssh, but separately - not on same file. – harrymc – 2019-09-18T17:16:09.810

That'd be okay. But how to duplicate the stdout/err of both bash and ssh into log files and the terminal? – jaaq – 2019-09-19T12:54:13.243

This question reads like a big “XY Problem.” I mean your description is as follows, “To be better able to backtrack or rollback configurations we'd like to keep a persistent history of all commands entered on the terminal and via ssh as well as save all output to stdout/err to a logfile.” Honestly, that sounds like a mess on a good day. A better solution? Use a version control system like Git or even a configuration management tool like Puppet. What you are describing is just — in my humble opinion — a bad way to achieve your state goal.

– JakeGould – 2019-09-20T01:25:36.227

I'd have to agree. If it was me setting up the processes alone, I'd tell everyone to run their code in docker containers, so we don't have to worry about mixing dependencies as much. But as it stands now it doesn't seem that we will implement that. I'd just like to log every change to the machine so I can check out what was changed if I needed to in one central place in a chronical order. – jaaq – 2019-09-21T13:31:56.277

Answers

The usual method for making a typescript of everything displayed on your terminal is to use the script command. However, this command does not handle stderr.

Another approach is to use process substitution with & redirection and exec:

exec &> >(tee -a "$log_file")

$log_file will contain the output of the script and any sub-processes, and the output will also be printed to the screen.

Where:

>(...) starts the process ... and returns a file representing its standard input.
exec &> ... redirects both standard output and standard error into ... for the remainder of the script. Use exec > ... for stdout only.
tee -a appends its standard input to the file, and also prints it to the screen.

harrymc

Posted 2019-09-10T14:32:43.870

Reputation: 306 093

You are probably going to want to look into auditd, which is a daemon that is specifically designed to do this.

Alternatively, as you have found, you can use the bash history, and increase the HISTSIZE / HISTFILESIZE variables to make it keep every command.

Edit: See https://serverfault.com/questions/470755/log-all-commands-run-by-admins-on-production-servers

Sirex

Posted 2019-09-10T14:32:43.870

Reputation: 10 321

But does it keep log files for every command that ends up in the history? Can you elaborate further? – jaaq – 2019-09-19T09:42:05.307

1you'll have to google for it as it's a large topic, but yes. It goes way way beyond logging every command anyone types, and is essentially able to log anything that happens on the system, period. You'd just be wanting a small subset of what it can do. – Sirex – 2019-09-19T20:22:46.467