So, say I get disconnected from an SSH-session after I've started rsync or cp or any other command that can be long running. Does that command keep running until it's finished after I get disconnected or does it just get killed?
Always wondered this.
This Q&A predates the systemd v230 debacle. As of systemd v230, the new default is to kill all children of a terminating login session, regardless of what historically valid precautions were taken to prevent this. The behavior can be changed by setting KillUserProcesses=no in /etc/systemd/logind.conf, or circumvented using the systemd-specific mechanisms for starting a daemon in userspace. Those mechanisms are outside the scope of this question.
The text below describes how things have traditionally worked in UNIX designspace for longer than Linux has existed.
They will get killed, but not necessarily immediately. It depends on how long it takes for the SSH daemon to decide that your connection is dead. What follows is a longer explanation that will help you understand how it actually works.
When you logged in, the SSH daemon allocated a pseudo-terminal for you and attached it to your user's configured login shell. This is called the controlling terminal. Every program you start normally at that point, no matter how many layers of shells deep, will ultimately trace its ancestry back to that shell. You can observe this with the pstree command.
When the SSH daemon process associated with your connection decides that your connection is dead, it sends a hangup signal (SIGHUP) to the login shell. This notifies the shell that you've vanished and that it should begin cleaning up after itself. What happens at this point is shell specific (search its documentation page for "HUP"), but for the most part it will start sending SIGHUP to running jobs associated with it before terminating. Each of those processes, in turn, will do whatever they're configured to do on receipt of that signal. Usually that means terminating. If those jobs have jobs of their own, the signal will often get passed along as well.
The processes that survive a hangup of the controlling terminal are ones that either disassociated themselves from having a terminal (daemon processes that you started inside of it), or ones that were invoked with a prefixed nohup command. (i.e. "don't hang up on this") Daemons interpret the HUP signal differently; since they do not have a controlling terminal and do not automatically receive a HUP signal, it is instead repurposed as a manual request from the administrator to reload the configuration. Ironically this means that most admins don't learn the "hangup" usage of this signal for non-daemons until much, much later. That's why you're reading this!
Terminal multiplexers are a common way of keeping your shell environment intact between disconnections. They allow you to detach from your shell processes in a way that you can reattach to them later, regardless of whether that disconnection was accidental or deliberate. tmux and screen are the more popular ones; syntax for using them is beyond the scope of your question, but they're worth looking into.
It was requested that I elaborate on how long it takes for the SSH daemon to decide that your connection is dead. This is a behavior which is specific to every implementation of a SSH daemon, but you can count on all of them to terminate when either side resets the TCP connection. This will happen quickly if the server attempts to write to the socket and the TCP packets are not acknowledged, or slowly if nothing is attempting to write to the PTY.
In this particular context, the factors most likely to trigger a write are:
SO_KEEPALIVE). Keepalives amount to either the server or the client infrequently sending packets to the other side, even when nothing would otherwise have a reason to write to the socket. While this is typically intended to skirt firewalls that time out connections too quickly, it has the added side effect of causing the sender to notice when the other side isn't responding that much more quickly.The usual rules for TCP sessions apply here: if there is an interruption in connectivity between the client and server, but neither side attempts to send a packet during the problem, the connection will survive provided that both sides are responsive afterwards and receiving the expected TCP sequence numbers.
If one side has decided that the socket is dead, the effects are typically immediate: the sshd process will send HUP and self-terminate (as described earlier), or the client will notify the user of the detected problem. It's worth noting that just because one side thinks the other is dead does not mean that the other is has been notified of this. The orphaned side of the connection will typically remain open until either it attempts to write to it and times out, or receives a TCP reset from the other side. (if connectivity was available at the time) The cleanup described in this answer only happens once the server has noticed.
As other's have mentioned, once you disconnect from ssh anything running within it is gone.
As @Michael Hampton and others have mentioned you can use tools like tmux or screen to disconnect/reconnect to terminals without loosing their content (i.e. child processes).
Additionally you can put a process into the background using an ampersand & and then use the command disown to disassociate them with the current shell.
# start a command
% sleep 5000 &
[1] 3820
# check it
% jobs
[1]+ Running sleep 5000 &
# disown everything
% disown -a
# check it again (gone from shell)
% jobs
%
# but it's still running on the system
% ps -eaf|grep "[s]leep"
saml 3820 23791 0 00:16 pts/1 00:00:00 sleep 5000
%
No, any programs still attached to the terminal, and not placed into the background with something like nohup, would be killed.
This is why there are virtual terminal solutions like tmux and the older screen which create sessions which continue running even if you are disconnected, and to which you can reattach later.
