Slow ssh connection establishement in the same LAN

Question

Hi, connecting to some SSH Linux servers on the same LAN takes about 10 seconds :

me@PC-A $ time ssh X.Y.Z.T :
Warning: Permanently added 'X.Y.Z.T' (RSA) to the list of known hosts.

real    0m10.168s
user    0m0.005s
sys     0m0.004s

But once logged on X.Y.Z.T :

me@serverA $ time ssh localhost :
root@localhost's password:

real    0m0.518s
user    0m0.022s
sys     0m0.010s

Here is the sshd_config :

me@serverA $ egrep -v "^(#|$)" /etc/ssh/sshd_config
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile      .ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem       sftp    /usr/libexec/openssh/sftp-server

and some information on the load :

me@serverA $ free -m
              total        used        free      shared  buff/cache   available
Mem:         128672       27136        3197        4089       98338       89989
Swap:         17407          14       17393
me@serverA $ top | head -15
top - 12:32:41 up 60 days, 20:39,  1 user,  load average: 0.45, 1.46, 1.64
Tasks: 455 total,   1 running, 454 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.3 us,  2.0 sy,  0.0 ni, 96.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13176076+total,  3230140 free, 27788596 used, 10074203+buff/cache
KiB Swap: 17825788 total, 17811452 free,    14336 used. 92147944 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 4624 root      20   0   58824   2384   1480 R  11.8  0.0   0:00.03 top
20726 root      20   0 2086772 596588   3908 S  11.8  0.5   1995:02 bizstorenode
20729 root      20   0 2375864   1.1g   3948 S  11.8  0.9   4161:04 bizstorenode
 4626 root      20   0  159464   3740   2780 S   5.9  0.0   0:00.01 biziodf
 9278 root      20   0 1209128  57952   5000 S   5.9  0.0  87:11.35 /usr/bin/python
 9299 root      20   0  457408  62472   2420 S   5.9  0.0   3859:49 python sagentd/
20719 root      20   0 2063592 547224   3924 S   5.9  0.4   2072:04 bizstorenode
20727 root      20   0 2071220 584648   3908 S   5.9  0.4   2029:08 bizstorenode

Just started the sysstat service, which was not installed, so there is no sar data yet.

EDIT0 : The result of the dig command :

me@serverA $ time dig +short -x X.Y.Z.T

real    0m0.007s
user    0m0.000s
sys     0m0.003s

Just to verify - the address `X.Y.Z.T` is inside the IANA private IP ranges (192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 )? or is that a public IP? My guess is something inside your DNS system (reverse lookup f.e.) is taking very long instead of returning an error immediately — Martin, Jul 19 '22 at 13:24
@Martin It is inside the class B private range. What entry should be added in the reverse DNS zone ? The entry of the X.Y.Z.T server ? It is already in /etc/hosts, is this enough ? — SebMa, Jul 19 '22 at 13:27
you do not need reverse PTR entries - you need a quick answer from your DNS server if those entries exist or not. Check first (for example with `time dig -x X.Y.Z.T`) if the reverse DNS really is the culprit! — Martin, Jul 19 '22 at 13:33
And btw: the existence of an entry inside `/etc/hosts` has nothing to do with reverse DNS - the local hosts file is not being consulted for reverse lookups... — Martin, Jul 19 '22 at 13:39
@Martin Can you please have a look at my EDIT0 ? I also have some `sar` stats available now, tell me the `sar` command you want me to type ? — SebMa, Jul 19 '22 at 14:19
I am not familiar with the `sar` command. Most of the cases, a slow ssh login is caused by DNS (check [here](https://jrs-s.net/2017/07/01/slow-ssh-logins/) ), although your edit might show otherwise... Check if that link helps you further, otherwise I am out of ideas. — Martin, Jul 19 '22 at 15:07
@Martin Thanks a lot for this great [comprehensive guide to fixing slow SSH logins – JRS Systems](https://jrs-s.net/2017/07/01/slow-ssh-logins) , I'll look into it tomorrow because I'm leaving the office right now. — SebMa, Jul 19 '22 at 15:49
@Martin `UseDNS` set to `yes` by default (according to `man sshd_config | grep -A1 UseDNS | grep default`) was the culprit. Adding ``UseDNS no` in `/etc/ssh/sshd_config` and typing `systemctl restart sshd` solved the problem. Can you please convert your last comment to an answer so I can accept it ? — SebMa, Jul 20 '22 at 12:32
@Martin On another server, stopping and disabling the `NetworkManager` service solved the problem. — SebMa, Jul 20 '22 at 16:13
@sebma could you test with `ssh -vvv X.Y.Z.T`, This might highlight where the delay is. edit: nevermiong, after reading the other comments, I see that's already solved. — Saïmonn, Jul 22 '22 at 13:46
@Saïmonn The 10 seconds wait appeared after the last line showed by `ssh -vvv`. But I have another issue that is similar with `PuTTY` but not with `openssh` for which I'm going to post a new question. — SebMa, Jul 26 '22 at 15:12
@Saïmonn Can you please take a look at my other question [here](https://serverfault.com/q/1106646/312306) ? — SebMa, Jul 26 '22 at 16:27

Slow ssh connection establishement in the same LAN

0 Answers0