ssh channel hangs after authentication

Server ranger has just yesterday become unresponsive upon ssh login, after months of working as expected. Authentication succeeds, but then -vvv logging reveals an incredibly long delay being set on the channel established for the session:

debug1: Authentication succeeded (publickey).
Authenticated to ranger ([192.168.1.115]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env SHELL
debug3: Ignored env TERM
debug3: Ignored env DOCKER_HOST
debug3: Ignored env NVM_DIR
debug3: Ignored env USER
debug3: Ignored env NAME
debug3: Ignored env LS_COLORS
debug3: Ignored env HOSTTYPE
debug3: Ignored env _JAVA_OPTIONS
debug3: Ignored env PATH
debug3: Ignored env PWD
debug3: Ignored env XMODIFIERS
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env LOGNAME
debug3: Ignored env LESSOPEN
debug3: Ignored env DISPLAY
debug3: Ignored env LESSCLOSE
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: channel_handler: chan 0: skip for 713631 more seconds
debug3: channel_handler: first channel unpauses in 713631 seconds
debug3: channel_handler: chan 0: skip for 713631 more seconds
debug3: channel_handler: first channel unpauses in 713631 seconds
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
debug3: channel_handler: chan 0: skip for 713631 more seconds
debug3: channel_handler: first channel unpauses in 713631 seconds
debug3: channel_handler: chan 0: skip for 713631 more seconds
debug3: channel_handler: first channel unpauses in 713631 seconds
debug3: channel_handler: chan 0: skip for 713631 more seconds
debug3: channel_handler: first channel unpauses in 713631 seconds

The same occurs on other accounts on this machine, including completely new ones using password authentication (so presumably that rules out .bashrc problems and so on). Likewise, attempting to invoke a completely different command (ssh -vvv user@ranger 'ls ~') yields the same.

The client and server are on the same ethernet network, attached to the same switch.

Client: OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016

Server: Ubuntu 16.04.4, OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016

cemerick

Posted 2018-05-07T13:23:43.827

Reputation: 231

Have you tried restarting the sshd process? Have you tried rebooting? – Tripp Kinetics – 2018-05-07T14:13:53.000

Can you test with PuTTY as client? – user1686 – 2018-05-07T14:32:15.900

@TrippKinetics Yes, and yes. But, it seems it's a client problem! – cemerick – 2018-05-07T18:15:54.083

@cemerick How about clearing .ssh on the client temporarily? – Tripp Kinetics – 2018-05-07T18:17:16.990

@grawity I did, and was shocked to find that it worked. Likewise, a separate git/bash environment. (My usual environment is WSL.) – cemerick – 2018-05-07T18:17:50.287

@TrippKinetics I had already done that; tried again, and still no luck. – cemerick – 2018-05-07T18:19:31.570

@cemerick How about connecting from that client to a different server? – Tripp Kinetics – 2018-05-07T18:30:42.840

I just got the exact same error on Windows Bash (WSL) Ubuntu (2 million second plus unpause), and it seems that moving all the files out of ~/.ssh, closing all terminal windows, and then moving the files back fixed the issue. I have no idea what could have triggered this. – March Ho – 2018-07-12T00:27:19.520

I had the same issue with WSL, I ended up having to remove and reinstall it. – Glen – 2018-10-01T17:17:15.913

Answers

I've just came to the same problem as above while using WSL (Windows Subsystem for Linux), and the solutions mentioned before did not work for me.

I finally resolved the problem by rebooting the WSL. At the PowerShell in the administrative mode just type in: Restart-Service LxssManager

I was inspired by the solution given in Rebooting Ubuntu on Windows without rebooting Windows?

cmick

Posted 2018-05-07T13:23:43.827

Reputation: 111

I'm also on WSL. Killing all running instances of C:\WINDOWS\System32\bash.exe fixed it. Before that, opening new bash instances didn't work.

I assume that others have tried rebooting, without success, so I don't think this solution is widely applicable, but hopefully my data point will help someone figure out what's going on.

I had several bash instances open, and some of them had been open for weeks, so maybe a shared exponential backoff value wasn't getting reset. Earlier today, Windows's network icon was (falsely) reporting that I didn't have network access, so maybe that had something to do with it.

Raijinili

Posted 2018-05-07T13:23:43.827

Reputation: 61

1The large values do indeed look like an exponential backoff problem. I am currently in the same situation and killing all bash instances did not fix it. I am using connection sharing in my SSH config, so maybe there is still a reference somewhere held by the OS. Following this, I remembered that I am using "wslbridge". Killing that fixed the issue. – exhuma – 2018-11-10T09:58:05.347

Ran into this identical issue with WSL openSSH. Every other ssh program had no problem logging into the local device. I tried all the suggestions above with no success even completely uninstalling openSSH-client and reinstalling.

Then I tried the number one thing you're suppose to do when a computer is acting stupid. Shut it off and turn it back on. Everything is back to normal.

jeffpkamp

Posted 2018-05-07T13:23:43.827

Reputation: 121

Not an answer so much as a resolution: I removed and reinstalled ssh, and all's well. ¯\_(ツ)_/¯

cemerick

Posted 2018-05-07T13:23:43.827

Reputation: 231

I experienced this issue a couple of times. Inspired by Marc Ho's comment, I exited all my WLS sessions and it worked right away on the next session I opened. I did not need to remove files from .ssh, though.

I suspect an orphaned lock between sleeps and/or network changes. I happen to have a network drive as Home in my Windows that could be contributing to the problem.

Felipe Carasso

Posted 2018-05-07T13:23:43.827

Reputation: 1