Trying to install Let's Encrypt certificate on CentOS 6 and Apache but can’t get a connection working

2

I have finally succumbed to the pressure of using SSL for my website that has been up since 1996… getting a certificate used to be expensive, error-prone and time-intensive back then… Today it's only the last two!

My Apache 2 server setup is quite complicated as I use my own Ruby dispatcher and rewrite non-static urls with mod_rewrite to go to the FastCGI dispatcher which does templating and other dynamic stuff, etc…

I'm a software engineer, but not much of a system admin so I end up messing around until everything works and then leave the server config alone for years.

When I installed the server a few years back, I started with a minimal CentOS install and installed most packages (including Ruby, Apache, MySQL, etc) manually and opened only the ports that my server needs.

I have obtained a certificate for both publicspace.net and www.publicspace.net with

sudo /usr/local/bin/certbot-auto certonly --webroot --webroot-path /var/www/html

The usual --apache version did not work for me, because I didn't have a VirtualHost defined. My Apache config is a patchwork perfected over the past two decades. I ended up adding a virtual host definition (and I understand pretty much what it does now), which does work, but I couldn't get the --apache version running anyway.. so I'm integrating the certificate manually.

I tried defining the SSL certificate in the httpd.conf file but things went badly until I realized that an SSL virtual host was already defined in /conf.d/ssl.conf, then I took my own stuff out of the main config file and added the certificate keys to that file instead.

I am now at the stage where, everything seems to be setup correctly, but I can’t get the pages to come up on my web browser.

openssl s_client -connect publicspace.net:443                                  

Gives:

connect: Connection refused
connect:errno=61

When done remotely, but gives a return code of 0 (ok) when invoked from the server itself. Port closed right?

I have opened the 443 port in the Linux firewall but to no avail.

# cat /etc/sysconfig/iptables
# Generated by iptables-save v1.4.7 on Wed Oct 26 17:41:47 2016
*filter
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state -m tcp --dport 22 --state NEW -j ACCEPT
-A INPUT -p tcp -m state -m tcp --dport 10000 --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp -m state --dport 80 --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp -m state --dport 443 --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp -m state --dport 873 --state NEW -j ACCEPT
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A INPUT -p tcp -m tcp -m state --dport 21 --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp -m state --dport 990 --state NEW -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Wed Oct 26 17:41:47 2016
# Generated by webmin
*mangle
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT
# Completed
# Generated by webmin
*nat
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT
# Completed

The httpd server is listening on that port and as tested with the s_client seems to be accepting local but not remote connections?

What am I missing?

Frank R.

Posted 2019-06-20T13:42:40.220

Reputation: 121

“My Apache config is a patchwork perfected over the past two decades.” I have been a webmaster, web developer, system admin and dozens of other things since around 1995 and find it hard to believe your Apache config is this complex for any valid reason. I realize you said “I'm a software engineer, but not much of a system admin so I end up messing around until everything works and then leave the server config alone for years.” But still, this sounds out of control for no reason. – JakeGould – 2019-06-20T13:49:57.620

Additionally, here is a big clue, “…because I didn't have a VirtualHost defined.” You should define a virtual host and move all your configs into that. When I setup any Apache server, all of my config options are in the virtual host with only performance tweaks done to the larger Apache config. I strongly encourage you to migrate your patchwork of Apache config stuff into a similar setup. You will make your life tons easier and avoid issues like this. – JakeGould – 2019-06-20T13:51:44.703

well, it's all in a Virtual Host now and the complexity is mostly the mod_rewrite & mod_fcgi setup.. I'm pretty much certain by now that something is blocking the 443 port, but as I said: I can opensll s_client into it no problem from the server itself, but not from outside. I just get a connection refused 61 error. My iptable (as far as I understand) has the port open.. is there anything besides the firewall that can block the port? – Frank R. – 2019-06-20T14:26:58.573

An Nmap shows 443/tcp closed https so this might all just be a port issue. Are you sure your Apache config is listening on port 443? Do you have a Listen 443 in your ssl.conf? – JakeGould – 2019-06-20T14:35:58.137

1Just solved it.. I was using webmin to change the iptables.. as it turns out (unlike with the apache conf files), webmin does not restart iptables automatically ;-) – Frank R. – 2019-06-20T14:52:30.637

Webmin often gets in the way. So good to hear you figured that out in this case! – JakeGould – 2019-06-20T15:28:22.253

Answers

1

A quick Nmap scan of your site publicspace.net shows the following:

nmap publicspace.net -p22,80,443
Starting Nmap 7.70 ( https://nmap.org ) at 2019-06-20 12:28 EDT
Nmap scan report for publicspace.net (45.76.0.90)
Host is up (0.0027s latency).

PORT    STATE SERVICE
22/tcp  open  ssh
80/tcp  open  http
443/tcp closed https

And that clearly shows port 443 is closed. That can mean either your firewall is not allowing traffic to port 443 or Apache itself is not listening on port 443.

So are you sure your Apache config is listening on port 443? Do you actually have a Listen 443 line in your ssl.conf or your Apache config in general?

Also, you show updated IPtables rules/filters but did you restart/reload these filters in IPtables? Depending on what tool you are using to manage your site, the new IPtables rules/filters won’t take affect unless IPTables setup is restarted/reloaded.

JakeGould

Posted 2019-06-20T13:42:40.220

Reputation: 38 217

Thanks. I use webmin and I assumed (which makes a.. ) that it was reloading the table automatically.. as it turned out it didn't. Thanks for your help! – Frank R. – 2019-06-21T07:31:17.663