Apache in linux-vserver won't start, can't create socket

Question

During extensive research and testing to write a proper question worthy of stackexchange I found a solution: rebuild the libapr1 package inside the guest.
I thought I'd nevertheless post this information as it could be useful to others.

Problem

When I install libapache2-mod-php5 inside the Wheezy guest and it tries to start up, I get the following:

root@test01:~# /usr/sbin/apache2ctl start
[crit] (22)Invalid argument: alloc_listener: failed to get a socket for (null)
Syntax error on line 9 of /etc/apache2/ports.conf:
Listen setup failed
Action 'start' failed.
The Apache error log may have more information.
root@test01:~# tail /var/log/apache2/error.log
root@test01:~#
root@test01:~# head -n 9 /etc/apache2/ports.conf|tail -n 1
Listen 80

This is the unchanged pristine package installation which by default doesn't manage to start up.

My testing

According to the official documentation, Listen 80 is actually fine. Turning it into Listen 127.0.0.1:80 gives me:

[crit] (22)Invalid argument: alloc_listener: failed to get a socket for 127.0.0.1
Syntax error on line 9 of /etc/apache2/ports.conf:
Listen setup failed
Action 'start' failed.

So why would Apache fail to get a socket? I've other deamons running (i.e. nginx on other Wheezy installation; exim4 listening on port 25 on the same installation) without problems.

Environment

Host

Debian Lenny on 2.6.26-2-vserver-amd64

# vserver-info
Versions:
                   Kernel: 2.6.26-2-vserver-amd64
                   VS-API: 0x00020303
             util-vserver: 0.30.216-pre2772; Dec 13 2008, 04:56:19

Features:
                       CC: gcc, gcc (Debian 4.3.2-1) 4.3.2
                      CXX: g++, g++ (Debian 4.3.2-1) 4.3.2
                 CPPFLAGS: ''
                   CFLAGS: '-Wall -g  -O2 -std=c99 -Wall -pedantic -W -funit-at-a-time'
                 CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W -fmessage-length=0 -funit-at-a-time'
               build/host: x86_64-pc-linux-gnu/x86_64-pc-linux-gnu
             Use dietlibc: yes
       Build C++ programs: yes
       Build C99 programs: yes
           Available APIs: v13,net,v21,v22,v23,netv2
            ext2fs Source: e2fsprogs
    syscall(2) invocation: alternative
      vserver(2) syscall#: 236/glibc
               crypto api: beecrypt
   use library versioning: yes

Paths:
                   prefix: /usr
        sysconf-Directory: /etc
            cfg-Directory: /etc/vservers
         initrd-Directory: $(sysconfdir)/init.d
       pkgstate-Directory: /var/run/vservers
          vserver-Rootdir: /var/lib/vservers


Assumed 'SYSINFO' as no other option given; try '--help' for more information.

Guest

Debian Wheezy, built with vserver $VSERVER build -m debootstrap --hostname $VSERVER --netdev eth0 --context $CONTEXT --interface v$CONTEXT=x.y.z.$CONTEXT/zz -- -d wheezy -m http://apt-proxy:9999/debian/

Research so far

The internets provied me with the following things so far:

• Problem on fedorra fixed by upgrading APR
• Problem fixed by upgrading package
• Kernel incompatibility
• Another solution required a kernel upgrade

My greatest fear, and this is my current conclusion, is that apache inside the virtual server depends on some newer kernel feature which my host doesn't provide. After all, the Wheezy default kernel certainly isn't as old as my 2.6.26.

I want to avoid upgrading the host kernel at all costs.

Why?

• Lack of time and knowledge (hardware is HP server, no idea what to watch out)
• Wheezy does not support vserver anymore (out of the box; for self-installation see 1) ...)
• Already running vservers which are required to be available 24/7 (the whole system is company internal and not exposed to the internet)
• No second same hardware to perform testing

I'm willing to patch Apache

If it's possible to figure out what the problem is, I'm wiling to build a custom deb package for my Wheezy quests.

Answer 1

Solution

As I said in the first sentence, I already found the solution: I rebuilt the libapr1 package inside the guest.

I found the solution by googling for "Invalid argument: alloc_listener: failed to get a socket for (null)", the 5th hit was Invalid argument crap which mentions the kernel upgrade and refers to another blogger talking about Fedora 11 httpd problems:

The issue is related to three kernel calls that are used in apr-1.3.8-1: accept4(), dup3() and epoll_create1(). Without these calls, apache is unable to start.

He mentions the Fedora team fixed so I checked out the bug report: https://bugzilla.redhat.com/show_bug.cgi?id=516331, specifically the second comment:

.. if you build your own APR on that Xen instance, it will correctly pick up the older functions and work ..

That ringed a bell. All I had to do was to rebuilt the libapr1 package because the configure script will automatically figure out that accept4 is not available and will revert to accept. That's how I did it:

• apt-get source libapr1
• tar -xf apr_1.4.6.orig.tar.gz
• cd apr-1.4.6
• tar -xf ../apr_1.4.6-3.debian.tar.gz
• dpkg-buildpackage
  This failed at first because of missing dependencies: apt-get install debhelper autoconf autotools-dev uuid-dev doxygen libtool
• After some time this produced a debian package outside the directory which I installed:
  dpkg -i libapr1_1.4.6-3_amd64.deb
• Then I just started apache and it worked!
  /etc/init.d/apache2 start

On a disk-starved systems, you may want to remove doxygen after compilation: on my system it needed over 600MB together with its dependencies.