bash script speed over ssh in background or foreground

I have a curiosity:

I run a bash script over SSH connection that has a lot of output to stdout. does it make any performance difference if I run the process in background or foreground?

I have the feeling that running in background, even if I see the output to stdout anyway, might be faster because bash itself does not depend on SSH latency. Does this make sense?

Andrea Zonca

Posted 2011-07-13T09:22:24.713

Reputation: 111

Answers

As pointed out by sarnold, running the process in background won't make noticeable difference. I think redirecting output to a file will remove the possible bottleneck of the local terminal and thus be faster.

Anyway, if you still need to see the output, try adding these options for ssh:

ssh -c blowfish -C remotehost foo

Where -C will enable compression and -c blowfish will select a cypher algorithm that has a lower cpu impact

user842313

Posted 2011-07-13T09:22:24.713

Reputation: 159

this looks great! what about instead if I redirect to a file and keep another ssh terminal with tail -f? – None – 2011-07-13T13:01:24.377

1That would move the bottleneck to the visualization part. That means that the command will end faster than if tail was in pipe with the command. You could also try to launch the command directly on the target host and redirect the output to a local file and then issue a tail -f from a ssh command. – None – 2011-07-13T13:35:52.013

this is perfect, I do not care if the visualization is slow, but I really do not want to slowdown my script. – None – 2011-07-13T14:03:43.983

You've asked about two cases, but I think you need a third:

without background:
```
ssh remotehost
foo
```
with background:
```
ssh remotehost
foo &
```
output redirected:
```
ssh remotehost
foo > bar
```

Without the background and with the background won't much matter -- bash(1) doesn't actually have to do anything with the standard input, standard output or standard error when you're executing another program. bash(1) never even sees it. It is just waiting for the process to die, so it can give you another shell prompt.

When you run the script in the background, you'll be able to interact with your bash(1) again very quickly, but any output will still go over your ssh(1) channel, potentially very slowly, and potentially the write(2) syscalls in your ssh(1) client might block, causing the pseudo-terminal it creates to block, causing your script to block when it calls write(2). This is all identical to the first case -- the only difference is you can type commands to your bash(1) while the script is sending you output, and maybe kill %1 to kill it, or start other services, or whatever. The script will run at the same speed either way.

In the third case, when you redirect output to a file, the ssh(1) session is no longer a potential bottleneck, and thus not a potential bottleneck for the execution of your script. It can run quite quickly, perhaps much faster than even running it locally without output redirection. (Have you ever seen top(1) output for your terminals when you've run a command that generates a lot of terminal output?)

Of course, a tool like screen(1) (or dtach(1)?) can also let you run a script without sending its output over the terminal session, so maybe a fourth option is required. But there's probably more ways to run a script without forcing its output to be sent over the network...

sarnold

Posted 2011-07-13T09:22:24.713

Reputation: 2 988

thanks, I want to keep seeing the script output to stdout, and I thought that using background might improve script performance. but as you explain it does not. btw, why do you write bash(1) instead of just bash? – None – 2011-07-13T11:17:10.267