Using Supermicro IPMI behind a Proxy?

Question

This is a SuperMicro server with a X8DT3 motherboard which contains an On-board IPMI BMC. In this case, the BMC is a Winbond WPCM450). I believe many Dell servers use this a similar BMC model.

A common practice with IPMI is to isolated it to a private, non-routable network. In our case all IPMI cards are plugged into a private management LAN at 192.168.1.0/24 which has no route to the outside world. If I plug my laptop into the 192.168.1.0/24 network, I can verify that all IPMI features work as expected, including the remote console.

I need to access all of the IPMI features from a different network, over some sort of encrypted connection.

I tried SSH port forwarding. This works fine for a few servers, however, we have close to 100 of these servers and maintaining a SSH client configuration to forward 6 ports on 100 servers is impractical.

So I thought I would try a SOCKS proxy. This works, but it seems that the Remote Console application does not obey my systemwide proxy settings.

1. I setup a SOCKS proxy. Verbose logging allows me to see network activity, and if ports are being forwarded.

   ssh -v -D 3333 stefanl@gateway.example.org
   

2. I configure my system to use the SOCKS proxy. I confirm that Java is using the SOCKS proxy settings.

3. The SOCKS proxy is working. I connect to the BMC at http://192.168.1.100/ using my webbrowser. I can log in, view the Server Health, power the machine on or off, etc. Since SSH verbose logging is enabled, I can see the progress.

Here's where it get's tricky:

3. I click on the "Launch Console" button which downloads a file called jviewer.jnlp. JNLP files are opened with Java Web Start.

4. A Java window opens. The titlebar says says "Redirection Viewer" in the title bar. There are menus for "Video" "Keyboard" "Mouse", etc. This confirms that Java is able to download the application through the proxy, and start the application.

5. 60 seconds later, the application times out and simply says "Error opening video socket". Here's a screenshot. If this worked, I would see a VNC-style window. My SSH logs show no connection attempts to ports 5900/5901. This suggests that the Java application started the VNC application, but that the VNC application ignores the systemwide proxy settings and is thus unable to connect to the remote host.

Java seems to obey my systemwide proxy settings, but this VNC application seems to ignore it.

Is there any way for me to force this VNC application to use my systemwide proxy settings?

Answer 1

I figured out it's best not to use a socks proxy for this, but instead forward all ports necessary on a localhost IP. To evade any existing services, I use a different IP than 127.0.0.1. Assuming you chose 127.0.0.2, and your server behind the proxy is 192.168.1.1, this is the ssh command to use:

ssh user@proxy-server -L127.0.0.2:443:192.168.1.1:443 -L127.0.0.2:5900:192.168.1.1:5900 -L127.0.0.2:5901:192.168.1.1:5901 -L127.0.0.2:5120:192.168.1.1:5120 -L127.0.0.2:5123:192.168.1.1:5123 -C

Then you can browse https://127.0.0.2 and use the KVM as usual.

The forwarded TCP ports are 5900 and 5901 for control and video, 5120 for virtual CD and 5123 for virtual floppy (I didn't test the latter two). Added -C for compression, though I don't know if anything sent is suitable for compression.

Another, slightly more comfortable (and, in theory, better performing) method on linux, is to use sshuttle, which transparently forwards all TCP connections via ssh using iptables and a python interpreter on the proxy-server.

sshuttle -r user@proxy-server 192.168.1.1

Hint: sshuttle is being packaged in Debian.

What I couldn't forward yet is the UDP port 623, which can be used for ipmitool, a CLI connection for IPMI. There are several tutorials on this, but none worked out for me. Anyways, Java KVM is well enough.

Answer 2

It sounds like a VPN might actually be your best bet. Have an ACL on the router so that the only non-local traffic has to traverse the VPN and you're done. Very simple and secure as well as easy to manage.

Answer 3

Try tsocks, it should let you run any process through a SOCKS proxy by setting LD_PRELOAD which should work in all subprocesses, see this for example usage. Of course, if you are using ssh to create a SOCKS proxy, you'll still have the UDP issue, but this should get around the subprocess issue.

Answer 4

I've used SSH connection to access my home Supermciro server's IPMI from external locations. TCP port forward is pretty easy, main trick is to forward UDP over SSH. I've used socat for this.

My setup, used as example:


1. laptop. I've used MacOS system with Oracle Java installed
2. gateway. Linux server used for SSH port forward. It should have access to IPMI interface over network.
3. server with IPMI. IP address is 192.0.2.2

Start ssh connection from laptop to gateway with forwarding all required ports (80, 443, 623, 5900) plus one extra port for UDP forward. I'm using 1623. Sudo is required to bind to privileged ports (<1024)

user@laptop ~ $ sudo ssh user@gateway \
-L 127.0.0.1:80:192.0.2.2:80 \
-L 127.0.0.1:623:192.0.2.2:623 \
-L 127.0.0.1:5900:192.0.2.2:5900 \
-L 127.0.0.1:443:192.0.2.2:443 \
-L 127.0.0.1:1623:127.0.0.1:1623

Now let's wrap IPMI UDP traffic into TCP socket on gateway.

user@gateway ~ $ socat tcp4-listen:1623,reuseaddr,fork udp:192.0.2.2:623

Last step is to convert TCP to UDP again on laptop. Sudo required once more

user@laptop ~ $ sudo socat udp4-recvfrom:623,reuseaddr,fork tcp:localhost:1623

Finally you can connect to localhost from Supermicro IPMI app, or just open it in browser and download KVM app.

Full IPMI with IP-KVM and remote media is forwarded via single SSH connection.