Nginx dropping the first client as soon as the second connects

1

I'm trying to configure Nginx to reverse proxy port 445, but every time client A is connected to the share through Nginx and a client B connects I have the connection of client A dropped by Nginx even though he was actively using the share (downloading a big file, for example). It's like Nginx is reusing the connection for client B before client A finishes using it.

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log debug;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

stream {

    server {
         listen 445;
         proxy_pass storage:445;
    }
}

What's missing in the config file above to allow both client A and B to use the share simultaneously without dropping one connection to stablish the other?

Some extra context:
Nginx v. 1.17.1 runing on Ubuntu 18.04.2 LTS virtual machine 4 vCPU and 4Gb mem ;

I have already tried making this control using iptables instead of Nginx to forward the connections on port 445 to the share server and the result was similar: client A has its connection dropped when B connects;

The share works fine if the clients A and B connects directly to the storage share without Nginx between them;

I have tried quite a lot of recomended configurations from Nginx documentation (limit_conn, so_keepalive, reuseport....), but I might have misused them;

From Wireshark I see Nginx sends a [FIN, ACK] packet to client A when client B connects;

Log of Nginx when client A has its connection afected: *[error] 32110#32110: 7 recv() failed (104: Connection reset by peer) while proxying and reading from upstream... but I notice this log is related to a [RST, ACK] packet client A sends to Nginx even after that [FIN, ACK] packet it received.

Edit:
Tried with the newer version 1.17.3 and no success.

Ronaldo

Posted 2019-07-18T15:53:54.347

Reputation: 306

Wait, what's the reason to proxy SMB through Nginx in the first place? – user1686 – 2019-09-10T18:27:07.967

@grawity, I have some legacy apps on a windows server 2003 that I want to migrate to a newer server. But the apps were badly written and have IP and/or DNS hardcoded (I don't have the codes). Database and app were on the same box, but I don't want it like this from now on. My problem is that I can't separete the apps from databases without crashing the apps. The solution I found was to use a Nginx reverse-proxy keeping the old server IP and DNS and redirecting the SMB to the new app server and database to the new database server. Is there a better way to achieve this? – Ronaldo – 2019-09-10T18:52:44.927

Answers

1

I think SMB Server will disconnect you because from its side, the same machine is trying to connect using different users. This is the same using masquerade with iptables and Nginx.

I would continue using iptables, but without masquerading traffic to your SMB server, only allowing forward.

iptables -t nat -A PREROUTING -p tcp -m tcp --dport 445 -j DNAT --to-destination storage:445
iptables -t filter -A FORWARD -d storage/32 -p tcp -m tcp --dport 445 -j ACCEPT

Make the traffic from your SMB server to the networks the clients resides to be routed through the proxy/forwarding server.

Then in the proxy/forwarding server you need to masquerade traffic to your clients networks. Example:

iptables -t nat -A POSTROUTING -d 192.168.0.0/24 -o eth0 -j MASQUERADE

With this, the SMB server will receive traffic from the client's IPs, while the clients communication is with the proxy/forwarding server and should not disconnect when multiple clients connects.

Jerdguez

Posted 2019-07-18T15:53:54.347

Reputation: 76

thank you for your response. I'm trying to implement it, but since I'm new to linux/iptables and just executing the commands as provided didn't work I'm studying the aspects of each option of your code to see if any adjustment is needed.

About the SMB server disconnecting me, I don't think that's the reason. I tested your assumption by connecting via RDP to client A with two different users and we could both use the SMB (no nginx or iptables in the middle) simultaneouly without the connection being dropped. If the SMB server were to blame, this experiment wouldn't work, right? – Ronaldo – 2019-09-12T14:32:42.877

About the tests, depends how you did it, if it was using the app you mentioned or direct connection. I think if you try from the same A client with different credentials to the SMB server (not the user for RDP), the connection would give you an 1219 system error. – Jerdguez – 2019-09-13T04:10:42.670

About iptables, first try flushing (removing) all prior rules: # sudo iptables -t nat -F, # sudo iptables -F and then try the commands in the answer (remember using sudo). In order to view the traffix in the machine, you may use tcpdump in the interface you receive the traffic, maybe eth0 like: sudo tcpdump -nn -i eth0 --port 445. Also, make sure you have forwarding enabled in the proxy/forwarding server: sudo sysctl net.ipv4.ip_forward=1. I did not include many details as you had already test it with iptables... – Jerdguez – 2019-09-13T04:25:44.877