Net bind capability with systemd

Question

I am deploying Goldfish, an interface for Vault, in production on a server dedicated to secrets management. So security is of prime concern here.

I am trying to deploy the service with systemd on an Unbuntu 16.04 system, giving it the least possible permissions. I want it to run as the non-root user goldfish and to listen on port 443. I have tried multiple options.

Solution 1: using a systemd socket, as suggested in multiple posts I've found such as this one (Goldfish backend is written in Go as in the post). This seems the most precise and secure as it is managed entirely by systemd and opens a single port for a single service. My unit files look like this:

/etc/systemd/system/goldfish.socket:

[Unit]
Description=a Vault interface

[Socket]
ListenStream=443
NoDelay=true  # Tried with false as well

/etc/systemd/system/goldfish.service:

[Unit]
Description=a Vault interface
After=vault.service
Requires=goldfish.socket
ConditionFileNotEmpty=/etc/goldfish.hcl

[Service]
User=goldfish
Group=goldfish
ExecStart=/usr/local/bin/goldfish -config=/etc/goldfish.hcl
NonBlocking=true  # Tried with false as well

[Install]
WantedBy=multi-user.target

Unfortunately I get a "listen tcp 0.0.0.0:443: bind: permission denied". So it seems that it still doesn't get the desired permission, which seems to defeat the purpose of creating a socket for the service. What am I missing here?

Solution 2: giving the CAP_NET_BIND_SERVICE capability to the service. This time I use no systemd socket but the AmbientCapabilities setting in the service (also tried with CpabilityBoudingSet and Capabilities, the latter being surprisingly undocumented in systemd.exec). Adding the CAP_NET_BIND_SERVICE capability to the service should give the service the right to bind to any privileged port.

/etc/systemd/system/goldfish.service:

[Service]
User=goldfish
Group=goldfish
ExecStart=/usr/local/bin/goldfish -config=/etc/goldfish.hcl

# Tried combinations of those:
AmbientCapabilities=CAP_NET_BIND_SERVICE
#CapabilityBoundingSet=CAP_NET_BIND_SERVICE
#Capabilities=CAP_NET_BIND_SERVICE+ep

Still no luck, I always end up with "listen tcp 0.0.0.0:443: bind: permission denied". What am I missing about capabilities management with systemd?

Solution 3: giving the CAP_NET_BIND_SERVICE capability to the executable. This time I give the CAP_NET_BIND_SERVICE capability directly to the goldfish executable, and nothing in the systemd service:

sudo setcap cap_net_bind_service=+ep /usr/local/bin/goldfish

Hurray, it works! The service correctly binds to port 443 and I can use it. However, I now have a binary that can be executed by any user and bind to any privileged port, which I don't really like. What is the relationship between executable capabilities and systemd service capabilities? Shouldn't systemd allow to achieve the same result that I get here but only for a specific process?

Solution 4: putting the service behind a proxy. The solution I'm considering now is to put Goldfish behind an nginx proxy that will be the only one visible from the outside and listening on port 443. This seems like a good compromise as it keeps tight permissions, but it adds a new piece to the setup, with what it adds of complexity and error potential. Does it seem like a superior or lesser option in terms of security and system administration?

So I actually have multiple questions here, but all relating to making a systemd service run as an unprivileged user while listening on a privileged port, and the security implications that it has. Did someone already encounter the same issues?

Thanks for your help!

score 4 · Answer 1 · answered Jun 16 '18 at 09:15

I know this isn't exactly what you wanted and adds "another piece" to the puzzle but you could consider creating a systemd .service file and moving the applications listening port to >1023 (this allows non-root to bind to it), then create an iptables rule that redirects all traffic from port 443 to your new custom port like this:

iptables -t nat -A PREROUTING -i <incoming_interface> -p tcp --dport 443 -j REDIRECT --to-port 8443

In this example, all tcp traffic to port 443 would be transparently redirected to port 8443.

score 3 · Answer 2 · answered Feb 15 '19 at 18:41

The reason Solution 1: using a systemd socket doesn't work is that the binary must be built to accept the socket from systemd. It doesn't "just work" unfortunately.

Systemd passes the socket as file descriptor 3 (after stdin, stdout, stderr). Here's an example program (from my blog post here)

package main

import (
    "log"
    "net"
    "net/http"
    "os"
    "strconv"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello World!"))
    })

    if os.Getenv("LISTEN_PID") == strconv.Itoa(os.Getpid()) {
        // systemd run
        f := os.NewFile(3, "from systemd")
        l, err := net.FileListener(f)
        if err != nil {
            log.Fatal(err)
        }
        http.Serve(l, nil)
    } else {
        // manual run
        log.Fatal(http.ListenAndServe(":8080", nil))
    }
}

You would have to ask goldfish to add support.

For the systemd capabilities (Solution 2) I don't know, but could it be that you also need SecureBits=keep-caps ? systemd should set that automatically, but maybe your version of systemd (8 months ago) didn't? Vault uses it: https://learn.hashicorp.com/vault/operations/ops-deployment-guide#step-3-configure-systemd

Another option would be to use SELinux, although that might be a "now you have two problems".

Personally in your situation I think I would put nginx in front of it. What did you end up doing?

Net bind capability with systemd

2 Answers2