2

I have IPVS configured as:

$ ipvsadm --save
-A -t localhost:<port> -s rr
-a -t localhost:<port> -r <other_hostname>:<port> -m -w 1

It seems to be running:

$ ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  localhost:<port> rr
  -> <other_hostname>:<port>         Masq    1      1          0

but that 1 'ActiveConn' hangs indefinitely, while a direct request responds quickly and as expected:

$ curl https://<other_hostname>:<port>
<response>
$ curl https://localhost:<port>
[hangs]

Nothing seems wrong in dmesg:

$ dmesg | grep IPVS
[ 4795.060820] IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
[ 4795.060932] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[ 4795.061202] IPVS: ipvs loaded.
[ 4795.067663] IPVS: [rr] scheduler registered.

And nothing in journalctl from ipvsadm.

It seems to get stuck in state SYN_RECV:

$ ipvsadm -lnc
IPVS connection entries
pro expire state       source             virtual            destination
TCP 00:56  SYN_RECV    127.0.0.1:58322    127.0.0.1:<port>   <other_ip>:<port>

It seems that it isn't a configuration issue with the real server, since it's an issue with all modes (route, masquerade, tunnel) and testing with httpbin.org rather than my own.

What can cause a request to an IPVS to hang or fail when the 'real server'(s) underlying it respond OK, and how can it be debugged?

OJFord
  • 151
  • 1
  • 1
  • 5
  • I initially asked on SO, but got a close vote suggesting it's a better fit here. https://stackoverflow.com/questions/58040533/requests-to-ipvsadm-server-hang-directly-to-real-server-ok – OJFord Sep 21 '19 at 16:25
  • 1
    It's a firewalling or networking issue. Debugging will be via `tcpdump`. – womble Sep 23 '19 at 01:12
  • @womble firewalling that affects ipvs but not 'direct' requests? The target IP address doesn't appear in `tcpdump` at all. – OJFord Sep 25 '19 at 11:39
  • Its weird, I am having the same issue (completely same setup) with ip_vs on 5.4.0-45-generic with Ubuntu 20.04. I see SYN packets and the SYN_RECV state via ipvsadm, but these packets are never being answered. There is no asymmetric routing involved. If I connect to the VIP from the host it is assigned to, this works fine. – M. Schmidt Sep 10 '20 at 17:21
  • Wish I could help @M.Schmidt, but I never got to the bottom of it. I was using Arch, but also hadn't added any firewall or done anything to the network that I could see would affect IPVS traffic only. – OJFord Sep 10 '20 at 19:26

0 Answers0