3

I'm trying to set up multiple Apache SSL vhosts, each on a different IPv6 address.

My CentOS7 VPS has a routed /64 IPv6 block assigned to it, let's say 2001:db8:acac:acac::/64, and I can already see packets coming in (tcpdump -nn -i eth0 'ip6 and src or dst net 2001:db8:acac:acac::/64' shows the packets fine).

I am aware I can assign as many individual addresses as I like to eth0 (ip -6 addr add 2001:db8:acac:acac::1234 dev eth0), but I want to get the interface allow apps to bind to any of the 2^64 addresses.

Following advice (see links at bottom), I added a rule (ip -6 rule add from 2001:db8:acac:acac::/64 iif eth0 lookup 200) and a route (ip route add local 2001:db8:acac:acac::/64 dev lo table 200) and now I can ping6 any IP address in the /64 block, and I can connect to services listening on wildcard (e.g. :::22 for ssh) using any address in the /64 block.

The question is: how can I make a program bind to a single address in the /64 block? As no interface owns any of the addresses in the block, I see the following in the apache logs:

... AH00072: make_sock: could not bind to address [2001:db8:acac:acac::1234]:443

I have seen mentions of IP_TRANSPARENT as a possible solution, but cannot find this mentioned in Apache source, only in bits/in.h, included by netinet/in.h.

Has anyone got this to work, either for Apache or for other apps (in particular: dovecot, postfix, bind)?

Relevant articles read before posting this question:

kasperd
  • 29,894
  • 16
  • 72
  • 122
Ashley GC
  • 31
  • 3
  • In case of IPv4 the answer would have been to assign the entire range to the `lo` interface. Unfortunately that only works with IPv4 and not IPv6 - which is kind of ironic as having many IP addresses for a single host would likely happen more often with IPv6 than with IPv4. I am afraid the answer is going to be that a clean solution is going to require a modified kernel. – kasperd Dec 10 '15 at 18:51
  • After digging a bit in the kernel source I found another socket option which may be related to this. It might be that you need to look for `IP_FREEBIND` rather than `IP_TRANSPARENT`. – kasperd Dec 10 '15 at 20:07
  • 1
    I looked at the `apache2` sources and I didn't find `IP_FREEBIND` or `IP_TRANSPARENT` anywhere, so it looks like that may not be supported. In that case assigning the address to some interface may be the only option. – kasperd Dec 10 '15 at 20:25
  • Looking a the man page for `ip(7)`, I see: `IP_FREEBIND`: If enabled, this boolean option allows binding to an IP address that is nonlocal or does not (yet) exist. [...] This option is the per-socket equivalent of the ip_nonlocal_bind /proc interface described below. If I `find /proc/ -name ip_nonlocal_bind 2>/dev/null`, I get one result only: `/proc/sys/net/ipv4/ip_nonlocal_bind`. This makes me think that `IP_FREEBIND` is not implemented for IPv6 yet. – Ashley GC Dec 12 '15 at 09:32
  • In a test C program, I have the code: `int opt = 1; if (setsockopt(listenfd, IPPROTO_IPV6, IP_FREEBIND, &opt, sizeof(opt)) < 0) error(1, errno, "setsockopt failed.");` and it fails with `EPROTO` "Protocol not available". – Ashley GC Dec 12 '15 at 09:39
  • `IP_FREEBIND` does work for IPv6. I tested it on Ubuntu 12.04 and 14.04. I do believe it has existed for long enough for all distributions to have it by now. The reason it fails for you is that you wrote `IPPROTO_IPV6` where you should have written `SOL_SOCKET`. – kasperd Dec 12 '15 at 10:03
  • I changed `IPPROTO_IPV6` to `SOL_SOCKET` and the `setsockopt` now works, thanks for that. However, the `bind` still fails. The code is `int fd = socket(AF_INET6, SOCK_STREAM, 0); serv_addr.sin6_family = AF_INET6; serv_addr.sin6_port = htons(5000); struct in6_addr someaddr = {{{0x20,0x01,0xd,0xb8,...}}}; serv_addr.sin6_addr = someaddr; /*setsockopt calls*/ if (bind(fd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) < 0) error(1, errno, "bind failed");` and the error is "Cannot assign requested address", whether the test app is run as root or a normal user. – Ashley GC Dec 12 '15 at 14:15
  • 1
    I was just looking at anothr page ([Linux & IPv6: How to bind to an arbitrary IPv6 address?](https://serverfault.com/questions/465889/linux-ipv6-how-to-bind-to-an-arbitrary-ipv6-address)) and it said to use `SOL_IP` not `SOL_SOCKET`. And now I can bind to any address! – Ashley GC Dec 12 '15 at 14:22
  • Yeah, I just myself realized that my previous comment said `SOL_SOCKET` where it should have said `SOL_IP`. There is another problem with your code which may be hard to spot at first, but it can come back to give you problems later. `struct sockaddr_in6` introduce two new fields which did not exist with IPv4. Those are `sin6_flowinfo` and `sin6_scope_id`. You should initialized those as 0, because otherwise you'll be using whatever uninitialized data may be at that place in memory. – kasperd Dec 12 '15 at 14:28
  • 1
    I've got `memset(&serv_addr, 0, sizeof(serv_addr));` so that should do, but well spotted. I've posted the full test code in an answer. – Ashley GC Dec 12 '15 at 15:15
  • 1
    I've hacked apr and httpd to add `IP_FREEBIND` functionality, now I'm trying to push some patches into the official sources. The first is into apr: [Apache BugZilla bug 58725](https://bz.apache.org/bugzilla/show_bug.cgi?id=58725). – Ashley GC Dec 12 '15 at 20:48
  • Another possibility would be to patch the kernel sources to add a way to define a prefix which processes are allowed to bind to without using `IP_FREEBIND`. In that case I think `ipv6_chk_addr` in `net/ipv6/addrconf.c` would be the file to change. – kasperd Dec 12 '15 at 21:10

2 Answers2

0

You can assign that address to absolutely any interface. For example, assign it to lo (in addition to ::1). IPv6 is pretty good at having multiple addresses on any interface. Then, after you made the address local, you can listen on it.

UPD: As I see, this idea doesn't differ much from that of mentioned in your first link about assigning address block to lo. Essentially this is the same, but block is degenerate to a single address.

Nikita Kipriyanov
  • 8,033
  • 1
  • 21
  • 39
  • I usually use the `dummy0` interface for such purposes. But I must admit that I don't know exactly what difference there would be between using `lo` and `dummy0` for this. – kasperd Dec 10 '15 at 18:28
  • I haven't tried with ipv6 but with ipv4 any network that is on `lo` will cause the machine to respond to every IP within the network, so if you add, say `192.100.51.2/24` to `lo` the machine will respond to all 256 `192.100.51.*` addresses, not just the `.2` – Eric Renouf Dec 10 '15 at 18:30
  • You've done it wrong. A single address is /32 in IPv4 and /128 in IPv6. So in your case that should be 192.100.51.2/32, or, in question - 2001:db8:acac:acac::1234/128. If you omit the mask, Linux will assume largest possible number. // Update: just tested this, it works. – Nikita Kipriyanov Dec 10 '15 at 18:38
  • @EricRenouf I tried `ip -6 addr add fd6d:fce8:298f::42/64 dev lo` and `ip -4 addr add 172.27.42.3/24 dev lo`. After those two commands I could bind to `172.27.42.8` but not to `fd6d:fce8:298f::43`. So that approach would solve the problem for IPv4 but not for IPv6. – kasperd Dec 10 '15 at 18:39
  • @NikitaKipriyanov He is not doing it wrong if the intention is to get the host to listen to all of the addresses. In fact `ip -6 addr add 2001:db8:acac:acac::/64 dev lo` could be the ideal answer to the question, if the `lo` interface handled IPv4 and IPv6 the same way. But it doesn't work. – kasperd Dec 10 '15 at 18:42
  • @kasperd: question is: how to listen on a SINGLE IPv6 address. – Nikita Kipriyanov Dec 10 '15 at 18:44
  • 1
    @NikitaKipriyanov The question is how to get **one process** to listen only to a single IPv6 address while **the host** is listening on all. And `IP_TRANSPARENT` is not an option because it would require changes to each application (and would only work for applications running as root). – kasperd Dec 10 '15 at 18:47
  • I don't see any obstacles to use AnyIP AND single /128 IP simultaneously. :: will denote "listen on any IP assigned to host" for listening, any of /64 network, while the the particular app will will happily see its address and listen only on that address. In previous example that will be two commands: ip -6 addr add fd6d:fce8:298f::/64 and ip -6 addr add fd6d:fce8:298f::42/128. – Nikita Kipriyanov Dec 10 '15 at 19:34
  • 1
    @NikitaKipriyanov But that still doesn't make it possible for applications to bind to addresses without first assigning each address to an interface. How would you ensure that a process can bind to **any** address in the entire /64? – kasperd Dec 10 '15 at 19:50
  • That's impossible. You have first to assign address to an interface to be able to bind socket to it. The workaround might be to NAT all addresses into a single address and then listen to it, but you'll lose information about to which address the connection was made. – Nikita Kipriyanov Sep 29 '20 at 09:44
0

While I have not yet been able to make apache bind to a single address which is not owned by an interface, I have made a test C program which works, and can bind to any IPv6 address, regardless of whether the address is owned by an interface:

/*
  Test program to bind to an IPv6 address not owned by an interface.
  This code is from public domain sources, and is released into the public domain.
*/

#include <arpa/inet.h>
#include <error.h>
#include <errno.h>
#include <net/if.h>
#include <netinet/in.h> // also includes bits/in.h
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h> // also includes bits/ioctls.h
#include <sys/socket.h>
#include <sys/types.h>
#include <time.h>
#include <unistd.h>

// A real address to use as a sanity check and make sure the program can 
// bind to an address which *is* owned by an interface
#define REALADDR {{{0x2a,0x00, ...}}}

// A fake address to show the program can bind to an address which is *not* 
// owned by any interface
#define SOMEADDR {{{0x20,0x01, 0x0d,0xb8, 0x00,0x00, 0x00,0x00, \
                    0x00,0x00, 0x00,0x00, 0x00,0x00, 0x00,0x01 }}}

int main(int argc, char *argv[])
{
    struct sockaddr_in6 serv_addr;
    int listenfd = 0, connfd = 0, i;

    char sendBuff[1025];
    time_t ticks;

    listenfd = socket(AF_INET6, SOCK_STREAM, 0);
    printf("socket fd is %d\n", listenfd);
    memset(&serv_addr, 0, sizeof(serv_addr));
    memset(sendBuff, 0, sizeof(sendBuff));

    serv_addr.sin6_family = AF_INET6;
    serv_addr.sin6_port = htons(5000);
    struct in6_addr someaddr = SOMEADDR;
    serv_addr.sin6_addr = someaddr;

    // Here's the magic:
    int opt = 1;
    if (setsockopt(listenfd, SOL_IP, IP_FREEBIND, &opt, sizeof(opt)) < 0)
        error(1, errno, "setsockopt(IP_FREEBIND) failed");

    printf("Binding to ");
    for (i = 0; i < 14; i += 2)
        printf("%x:", (serv_addr.sin6_addr.s6_addr[i] << 8) + 
            serv_addr.sin6_addr.s6_addr[i+1]);
    printf("%x\n", (serv_addr.sin6_addr.s6_addr[14] << 8) + 
        serv_addr.sin6_addr.s6_addr[15]);
    if (bind(listenfd, (struct sockaddr*)&serv_addr, sizeof(serv_addr)) < 0)
        error(1, errno, "bind failed");

    if (listen(listenfd, 10) < 0)
        error(1, errno, "listen failed");

    while(1)
    {
        connfd = accept(listenfd, (struct sockaddr*)NULL, NULL);
        printf("accept returned %d\n", connfd);

        // Send some data - the current date and time.
        ticks = time(NULL);
        snprintf(sendBuff, sizeof(sendBuff), "Now is %.24s\r\n", ctime(&ticks));
        write(connfd, sendBuff, strlen(sendBuff));

        close(connfd);
        sleep(1);
     }
}
Ashley GC
  • 31
  • 3