0

Imagine performing a short repetitive process, which ends by doing a short lived SSH connection to some server (e.g. transferring a newly generated file via SFTP). This means that there will be a lot of short lived consecutive SSH connecting and disconnecting.

Are there any considerations needs to be taken into account in such situation? E.g. possible SSH connection request limit etc.

Martin Prikryl
  • 7,327
  • 2
  • 36
  • 71
user2340939
  • 103
  • 2
  • 2
    How little is the useful payload to be transferred in each session? How "far" is source from destination? – 178024 Sep 04 '19 at 12:13
  • Let's assume the servers are as far away from each other as possible, and the payload is a large file, but still not too big to be uploaded almost instantaneously. – user2340939 Sep 04 '19 at 12:17

1 Answers1

3

If task is repetitive I would suggest to use persistent(Long lived) ssh connections for that you can use ControlMaster, new connections are expensive and good volume of connection can kill(unresponsive) the host itself. You can use ControlMaster with ServerAliveInterval in your ssh config settings for good example refer this How can I create persistent SSH connection to "stream" commands over a period of time?

asktyagi
  • 2,401
  • 1
  • 5
  • 19
  • 1
    Worth mentioning that this answer implies that the "process" that the OP has, uses OpenSSH `sftp` for the file transfers. And not for example an SFTP library in some programming language. +1 anyway – Martin Prikryl Sep 04 '19 at 12:44
  • I actually want to avoid persistent SSH connection, since the process is done inside an application (Python, to be exact), and this would really complicate things (also because I want the process to be atomic/independent of other processes). So I am aware that the proposed solution would be the best, but given the circumstances, my question is whether I can do this without the persistent connection, and not worry about any critical downsides. From your answer it seems like the actual number of the connections is the key deciding factor here (i.e. asymptotic scale). – user2340939 Sep 05 '19 at 08:48