Running Apache2 in xenserver

Question

I am trying to run my own squirrelmail email server. This involves the use of apache2, postfix, and squirrelmail.

When I first boot up the server everything will work fine, but after about 1 - 2 hours ports 80 (apache) and 25 (smtp) will be closed to any one outside the network. However, it will still work if I view the website using the servers local ip.

My network only has one router and a Layer 2 switch. The network is not segmented.

Gateway:    10.0.0.1
subnetmask: 255.0.0.0
servers ip: 10.0.25.0

Below are diagnostics from the apache server when it went down for the public ip.

netstat -an | grep 80
tcp6       0      0 :::80                   :::*                    LISTEN
unix  3      [ ]         STREAM     CONNECTED     13180



nmap 10.0.25.0

Starting Nmap 6.47 ( http://nmap.org ) at 2016-05-02 22:27 PDT
Nmap scan report for 10.0.25.0
Host is up (0.000012s latency).
Not shown: 994 closed ports
PORT    STATE SERVICE
22/tcp  open  ssh
25/tcp  open  smtp
80/tcp  open  http
110/tcp open  pop3
111/tcp open  rpcbind
143/tcp open  imap

Nmap done: 1 IP address (1 host up) scanned in 7.43 seconds




ps -aux | grep 80
root       155  0.0  0.6  40808  3296 ?        Ss   21:20   0:00 /lib/systemd/systemd-udevd
root       429  0.0  0.5  37080  2724 ?        Ss   21:20   0:00 /sbin/rpcbind -w
statd      443  0.0  0.5  37280  2936 ?        Ss   21:20   0:00 /sbin/rpc.statd
root       480  0.0  0.5  17724  2664 ?        Ss   21:20   0:00 /usr/sbin/dovecot -F
root       513  0.0  0.4  14236  2180 hvc0     Ss+  21:20   0:00 /sbin/agetty --keep-baud 115200 38400 9600 hvc0 vt102
www-data   695  0.0  1.7 219348  8804 ?        S    21:20   0:00 /usr/sbin/apache2 -k start
root      2808  0.0  1.1  82728  5876 ?        Ss   21:56   0:00 sshd: andrew [priv]
root      3287  0.0  0.4  12732  2168 pts/0    S+   22:05   0:00 grep 80



 systemctl status apache2
● apache2.service - LSB: Apache2 web server
   Loaded: loaded (/etc/init.d/apache2)
   Active: active (running) since Mon 2016-05-02 21:20:23 PDT; 48min ago
  Process: 477 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/apache2.service
           ├─ 643 /usr/sbin/apache2 -k start
           ├─ 694 /usr/sbin/apache2 -k start
           ├─ 695 /usr/sbin/apache2 -k start
           ├─ 696 /usr/sbin/apache2 -k start
           ├─ 697 /usr/sbin/apache2 -k start
           ├─ 698 /usr/sbin/apache2 -k start
           └─1003 /usr/sbin/apache2 -k start

May 02 21:20:23 web-server apache2[477]: Starting web server: apache2.

The servers local ip is 10.0.25.0. The client is 10.1.0.0.

tcpdump -n port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:13:31.449906 IP 10.1.0.0.4043 > 10.0.25.0.80: Flags [S], seq 3670228936, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:13:31.449959 IP 10.0.25.0.80 > 10.1.0.0.4043: Flags [S.], seq 3250350582, ack 3670228937, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:13:31.449984 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [S], seq 446370714, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:13:31.449995 IP 10.0.25.0.80 > 10.1.0.0.4045: Flags [S.], seq 2977754323, ack 446370715, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:13:31.450296 IP 10.1.0.0.4044 > 10.0.25.0.80: Flags [S], seq 1734125982, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:13:31.450346 IP 10.0.25.0.80 > 10.1.0.0.4044: Flags [S.], seq 3475246672, ack 1734125983, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:13:31.450366 IP 10.1.0.0.4046 > 10.0.25.0.80: Flags [S], seq 1502682879, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:13:31.450375 IP 10.0.25.0.80 > 10.1.0.0.4046: Flags [S.], seq 3725546174, ack 1502682880, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:13:31.450380 IP 10.1.0.0.4043 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:13:31.450385 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:13:31.450436 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [P.], seq 1:486, ack 1, win 256, length 485
22:13:31.450469 IP 10.0.25.0.80 > 10.1.0.0.4045: Flags [.], ack 486, win 473, length 0
22:13:31.450753 IP 10.1.0.0.4044 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:13:31.450760 IP 10.1.0.0.4046 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:13:31.452149 IP 10.0.25.0.80 > 10.1.0.0.4045: Flags [.], seq 1:2921, ack 486, win 473, length 2920
22:13:31.452348 IP 10.0.25.0.80 > 10.1.0.0.4045: Flags [P.], seq 2921:3419, ack 486, win 473, length 498
22:13:31.452497 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [.], ack 2921, win 256, length 0
22:13:31.469780 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [P.], seq 486:939, ack 3419, win 254, length 453
22:13:31.470040 IP 10.0.25.0.80 > 10.1.0.0.4045: Flags [P.], seq 3419:3601, ack 939, win 490, length 182
22:13:31.520799 IP 10.1.0.0.4045 > 10.0.25.0.80: Flags [.], ack 3601, win 253, length 0
^C
20 packets captured
20 packets received by filter
0 packets dropped by kernel

After the 1 - 2 hours it wont receive any packets from outside the network. So below is a tcpdump of right after its booted up so that you can see it works fine for a while.

tcpdump -n port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:17:55.192042 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [S], seq 1175674010, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:17:55.192100 IP 10.0.25.0.80 > 10.0.0.1.4092: Flags [S.], seq 1155279685, ack 1175674011, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:17:55.192121 IP 10.0.0.1.4094 > 10.0.25.0.80: Flags [S], seq 2011823322, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:17:55.192131 IP 10.0.25.0.80 > 10.0.0.1.4094: Flags [S.], seq 4263240, ack 2011823323, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:17:55.192136 IP 10.0.0.1.4093 > 10.0.25.0.80: Flags [S], seq 2247299647, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:17:55.192145 IP 10.0.25.0.80 > 10.0.0.1.4093: Flags [S.], seq 1959082678, ack 2247299648, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:17:55.192202 IP 10.0.0.1.4095 > 10.0.25.0.80: Flags [S], seq 2917948577, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
22:17:55.192210 IP 10.0.25.0.80 > 10.0.0.1.4095: Flags [S.], seq 2957320834, ack 2917948578, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
22:17:55.193109 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:17:55.193131 IP 10.0.0.1.4094 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:17:55.193212 IP 10.0.0.1.4093 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:17:55.194606 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [P.], seq 1:495, ack 1, win 256, length 494
22:17:55.194657 IP 10.0.25.0.80 > 10.0.0.1.4092: Flags [.], ack 495, win 473, length 0
22:17:55.194749 IP 10.0.0.1.4095 > 10.0.25.0.80: Flags [.], ack 1, win 256, length 0
22:17:55.196114 IP 10.0.25.0.80 > 10.0.0.1.4092: Flags [.], seq 1:2921, ack 495, win 473, length 2920
22:17:55.196329 IP 10.0.25.0.80 > 10.0.0.1.4092: Flags [P.], seq 2921:3419, ack 495, win 473, length 498
22:17:55.204189 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [.], ack 2921, win 256, length 0
22:17:55.215582 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [P.], seq 495:966, ack 3419, win 254, length 471
22:17:55.215815 IP 10.0.25.0.80 > 10.0.0.1.4092: Flags [P.], seq 3419:3601, ack 966, win 490, length 182
22:17:55.268342 IP 10.0.0.1.4092 > 10.0.25.0.80: Flags [.], ack 3601, win 253, length 0
^C
20 packets captured
20 packets received by filter
0 packets dropped by kernel

This is a tcpdump from when it wasn't working. This is a dump of every port. Notice it only contains ssh packets and nothing to do with apache or postfix.

20:23:38.066007 IP (tos 0x10, ttl 64, id 2649, offset 0, flags [DF], proto TCP (6), length 296)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e1b (incorrect -> 0x8dfc), seq 5140320:5140576, ack 4001, win 520, length 256
20:23:38.066057 IP (tos 0x10, ttl 64, id 2650, offset 0, flags [DF], proto TCP (6), length 296)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e1b (incorrect -> 0x79a8), seq 5140576:5140832, ack 4001, win 520, length 256
20:23:38.066107 IP (tos 0x10, ttl 64, id 2651, offset 0, flags [DF], proto TCP (6), length 328)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e3b (incorrect -> 0x732f), seq 5140832:5141120, ack 4001, win 520, length 288
20:23:38.066157 IP (tos 0x10, ttl 64, id 2652, offset 0, flags [DF], proto TCP (6), length 424)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e9b (incorrect -> 0x8487), seq 5141120:5141504, ack 4001, win 520, length 384
20:23:38.066212 IP (tos 0x0, ttl 128, id 6106, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x3636 (correct), ack 5140832, win 251, length 0
20:23:38.066216 IP (tos 0x10, ttl 64, id 2653, offset 0, flags [DF], proto TCP (6), length 232)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2ddb (incorrect -> 0x85c9), seq 5141504:5141696, ack 4001, win 520, length 192
20:23:38.066254 IP (tos 0x0, ttl 128, id 6107, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x3511 (correct), ack 5141120, win 256, length 0
20:23:38.066258 IP (tos 0x10, ttl 64, id 2654, offset 0, flags [DF], proto TCP (6), length 328)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e3b (incorrect -> 0xed66), seq 5141696:5141984, ack 4001, win 520, length 288
20:23:38.066308 IP (tos 0x10, ttl 64, id 2655, offset 0, flags [DF], proto TCP (6), length 296)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e1b (incorrect -> 0x3b02), seq 5141984:5142240, ack 4001, win 520, length 256
20:23:38.066355 IP (tos 0x0, ttl 128, id 6108, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x32d3 (correct), ack 5141696, win 254, length 0
20:23:38.066363 IP (tos 0x10, ttl 64, id 2656, offset 0, flags [DF], proto TCP (6), length 200)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2dbb (incorrect -> 0x5031), seq 5142240:5142400, ack 4001, win 520, length 160
20:23:38.066457 IP (tos 0x10, ttl 64, id 2657, offset 0, flags [DF], proto TCP (6), length 552)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2f1b (incorrect -> 0x784b), seq 5142400:5142912, ack 4001, win 520, length 512
20:23:38.066505 IP (tos 0x0, ttl 128, id 6109, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x30b5 (correct), ack 5142240, win 252, length 0
20:23:38.066513 IP (tos 0x10, ttl 64, id 2658, offset 0, flags [DF], proto TCP (6), length 328)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e3b (incorrect -> 0x17e5), seq 5142912:5143200, ack 4001, win 520, length 288
20:23:38.066606 IP (tos 0x10, ttl 64, id 2659, offset 0, flags [DF], proto TCP (6), length 456)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2ebb (incorrect -> 0x1077), seq 5143200:5143616, ack 4001, win 520, length 416
20:23:38.066657 IP (tos 0x10, ttl 64, id 2660, offset 0, flags [DF], proto TCP (6), length 200)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2dbb (incorrect -> 0xbfea), seq 5143616:5143776, ack 4001, win 520, length 160
20:23:38.066712 IP (tos 0x0, ttl 128, id 6110, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x2cf1 (correct), ack 5143200, win 256, length 0
20:23:38.066716 IP (tos 0x10, ttl 64, id 2661, offset 0, flags [DF], proto TCP (6), length 504)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2eeb (incorrect -> 0xd7e3), seq 5143776:5144240, ack 4001, win 520, length 464
20:23:38.066807 IP (tos 0x0, ttl 128, id 6111, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x2ab3 (correct), ack 5143776, win 254, length 0
20:23:38.066815 IP (tos 0x10, ttl 64, id 2662, offset 0, flags [DF], proto TCP (6), length 408)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e8b (incorrect -> 0xfdc6), seq 5144240:5144608, ack 4001, win 520, length 368
20:23:38.066850 IP (tos 0x0, ttl 128, id 6112, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x28e5 (correct), ack 5144240, win 252, length 0
20:23:38.066853 IP (tos 0x10, ttl 64, id 2663, offset 0, flags [DF], proto TCP (6), length 200)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2dbb (incorrect -> 0x9d2e), seq 5144608:5144768, ack 4001, win 520, length 160
20:23:38.066908 IP (tos 0x10, ttl 64, id 2664, offset 0, flags [DF], proto TCP (6), length 296)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e1b (incorrect -> 0xf162), seq 5144768:5145024, ack 4001, win 520, length 256
20:23:38.066956 IP (tos 0x10, ttl 64, id 2665, offset 0, flags [DF], proto TCP (6), length 248)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2deb (incorrect -> 0x2bbc), seq 5145024:5145232, ack 4001, win 520, length 208
20:23:38.067006 IP (tos 0x10, ttl 64, id 2666, offset 0, flags [DF], proto TCP (6), length 232)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2ddb (incorrect -> 0x68fb), seq 5145232:5145424, ack 4001, win 520, length 192
20:23:38.067051 IP (tos 0x0, ttl 128, id 6113, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x26d1 (correct), ack 5144768, win 256, length 0
20:23:38.067054 IP (tos 0x10, ttl 64, id 2667, offset 0, flags [DF], proto TCP (6), length 200)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2dbb (incorrect -> 0x3ccd), seq 5145424:5145584, ack 4001, win 520, length 160
20:23:38.067103 IP (tos 0x0, ttl 128, id 6114, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x2503 (correct), ack 5145232, win 254, length 0
20:23:38.067106 IP (tos 0x10, ttl 64, id 2668, offset 0, flags [DF], proto TCP (6), length 232)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2ddb (incorrect -> 0x8871), seq 5145584:5145776, ack 4001, win 520, length 192
20:23:38.067156 IP (tos 0x10, ttl 64, id 2669, offset 0, flags [DF], proto TCP (6), length 296)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2e1b (incorrect -> 0x3056), seq 5145776:5146032, ack 4001, win 520, length 256
20:23:38.067202 IP (tos 0x0, ttl 128, id 6115, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.0.0.37324 > 10.0.25.0.ssh: Flags [.], cksum 0x23a4 (correct), ack 5145584, win 253, length 0
20:23:38.067205 IP (tos 0x10, ttl 64, id 2670, offset 0, flags [DF], proto TCP (6), length 184)
    10.0.25.0.ssh > 10.1.0.0.37324: Flags [P.], cksum 0x2dab (incorrect -> 0x3388), seq 5146032:5146176, ack 4001, win 520, length 144

20707 packets captured
24555 packets received by filter
0 packets dropped by kernel

Please don't use *squirrelmail*. It's development was completely abandoned 3 years ago. Use *roundcube* or any other modern alternative. — drookie, May 02 '16 at 07:30
Your post lacks vital diagnostics and in the same time is overfilled with boring details about what combination do work and what do not, but it's really irrelevant. You should debug listening ports and packet flow, according to the OSI model chart. This is the main reason why there's zero answers at this time. — drookie, May 02 '16 at 08:53
I have used nmap, tcpdump, arp, ping, and I have went though most of the apache and postfix log files. How much more debugging can I do? Are there any specific output results or log files you need to see? The details about my problem were not made to entertain you, they are there to help you solve the problem. I'm sorry if they are a waste of your time, but its a lot better than nothing. If its easier for you, I will even let you ssh into the server there is nothing important on it. — Andrew, May 02 '16 at 10:39
Don't let anonymous people from the internet inside your servers. That's the rule. Second, remove all the unnecessary stuff from your post, and leave only facts describing the situation when the service stops working. Next step - add the diagnostics, showing the networking scheme (doesn't matter if its created by hand on the piece of paper), the `netstat -an | grep 80`(or whatever port you expect the apache to serve) output, the `ps` output showing that apache is alive and the tcpdump output showing hat your VM is receiving packets from the outer world. — drookie, May 02 '16 at 13:15
We need a tcpdump capture taken inside the VM after it stopped responding to the outside worls, not from the situation whan everything works. — drookie, May 03 '16 at 14:16
Ok, ill post the results of the tcpdump in a couple of hours when the vm quits working, but i have done it a couple of times before. There are no packets going back and forth so it will just be a blank tcpdump. — Andrew, May 03 '16 at 22:55
Actually, when I use tcpdump and listen in on a specific port it opens that port up and everything starts working again. Its bizarre. So I'll have to wait a little while longer and then run a tcpdump for every port. Is it in general a good idea to visualize an mail server? — Andrew, May 03 '16 at 23:04

score 0 · Accepted Answer · answered May 29 '16 at 23:38

0

The problem had to do with my router not getting the correct arp information. I fixed the problem by giving my router static arp entries.

answered May 29 '16 at 23:38

Andrew

101
2

Running Apache2 in xenserver

1 Answers1