2

I have CentOS Linux release 7.4.1708 Linux distro.

I need to install some package. The first command to execute is to add repoistory:

yum install https://extras.getpagespeed.com/release-el7-latest.rpm

When I run it, command hangs and prints errors after some time. Full output from cli is below:

# yum install https://extras.getpagespeed.com/release-el7-latest.rpm
Cannot open:
https://extras.getpagespeed.com/release-el7-latest.rpm. Skipping.
Error: Nothing to do

P.S. I try to install Page Speed Nginx module

Update 1

Seems like my server can't download rpm file:

wget https://extras.getpagespeed.com/release-el7-latest.rpm

Server is connected to the Internet, wget and ping commands works ok on some random resources.

Update 2

I've downloaded file https://extras.getpagespeed.com/release-el7-latest.rpm to my workstation via browser and uploaded it to home server folder.

Then I went to server home folder and executed command rpm -Uvh release-el7-latest.rpm.

#rpm -Uvh release-el7-latest.rpm
warning: release-el7-latest.rpm: Header V4 RSA/SHA1 Signature, key ID 222b0e83: NOKEY
Preparing... Updating / installing...
1:getpagespeed-extras-7-3.el7.gps

Then I searched my package via yum search command. It gives me error, some parts of huge output are below.

yum search nginx-module

extras | 3.4 kB 00:00:00
https://extras.getpagespeed.com/redhat/7/noarch/repodata/repomd.xml:
[Errno 14] curl#7 - "Failed to connect to 2606:4700:30::6812:31e3:
Network is unreachable" Trying other mirror.
https://extras.getpagespeed.com/redhat/7/noarch/repodata/repomd.xml:
[Errno 14] curl#56 - "Callback aborted" Trying other mirror.

I figured out what there're some problems with IPv6 and added ip_resolve=4 setting to /etc/yum.conf file.

Errors changed:

yum search nginx-module
extras | 3.4 kB 00:00:00
https://extras.getpagespeed.com/redhat/7/noarch/repodata/repomd.xml:
[Errno 14] curl#7 - "Failed connect to extras.getpagespeed.com:443; Operation now in progress"

Can problems be related to https connection problems?

Update 3

Error in Update 2 is related to file https://extras.getpagespeed.com/redhat/7/noarch/repodata/repomd.xml, it can't be downloaded.

Direct download from cli has no success too:

# curl --verbose https://extras.getpagespeed.com/redhat/7/noarch/repodata/repomd.xml
* About to connect() to extras.getpagespeed.com port 443 (#0)
* Trying 104.18.48.227...
* Connection timed out
* Trying 104.18.49.227...
* After 86358ms connect time, move on!
* Trying 2606:4700:30::6812:31e3...
* Failed to connect to 2606:4700:30::6812:31e3: Network is unreachable
* Trying 2606:4700:30::6812:30e3...
* Failed to connect to 2606:4700:30::6812:30e3: Network is unreachable
* Failed connect to extras.getpagespeed.com:443; Network is unreachable
* Closing connection 0
curl: (7) Failed to connect to 2606:4700:30::6812:31e3: Network is unreachable

I thought it's https problem on server, but curl https://www.google.com works perfectly. File is downloadable via browser from workstation.

Update 4

Iptables rules are below:

# iptables-save
*filter
:INPUT ACCEPT [71:19593]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [81:64337]
:MYSQL - [0:0]
:MYSQL_WHITELIST - [0:0]
-A INPUT -p tcp -m tcp --dport 3306 -j MYSQL
-A MYSQL -j MYSQL_WHITELIST
-A MYSQL -j ACCEPT
-A MYSQL_WHITELIST -s 10.100.10.6/32 -j RETURN
-A MYSQL_WHITELIST -j DROP
COMMIT

Update 5

Traceroute output:

# traceroute extras.getpagespeed.com
traceroute to extras.getpagespeed.com (104.18.48.227), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
...
30 * * *

userlond
  • 21
  • 1
  • 7
  • 1
    You should try to traceroute to that IP address and show the results. – Michael Hampton Dec 12 '18 at 00:44
  • 1
    The firewall on your server side will not be an issue. **Off topic note:** You are filtering only MYSQL connection all the rest is just accepted as **default policy** on **INPUT** chain. Is it whole output - are you really opening the server for all ports ? Anyway for mySQL filtering you can remove **MYSQL** chain and jump directly to **MYSQL_WHITELIST** just instead of **RETURN** use **ACCEPT**. Last rule for mySQL is **DROP** "the rest" what is not passed for mySQL connection from whitelisted IP until reaching last rule in chain. – Kamil J Dec 12 '18 at 01:10
  • I agree with Kamil J. I successfully tested this ruleset on a centOS VM. – frontsidebus Dec 12 '18 at 04:01

2 Answers2

4

It looks like your web request is timing out when trying to reach extras.getpagespeed.com on port 443 for HTTPS. This host is actually load balanced using DNS between a couple different ip's so it's trying both, then trying ipv6 (which is unreachable because you likely don't have a default route for ipv6 configured in your routing table). You should be able to verify this by opening a tcp connection on 443 using telnet: 

telnet extras.getpagespeed.com 443

if this times out, you likely have a firewall rule blocking your outbound web requests to the ip addresses that your are resolving for extras.getpagespeed.com

You can verify this by checking your firewall settings first on your host: 

iptables-save

for iptables. If you are running firewalld you can also use: 

systemctl status firewalld

to get the state, and then 

firewall-cmd --list-all

if there isn't an active rule blocking outbound tcp connections to destination port 443 to 104.18.49.227 or 104.18.48.227 but your telnet is still timing out, the next step is to figure out which hop in your network is filtering HTTPS requests to those specific ip addresses. You should be able to pretty easily find the next hop in your network using: 

traceroute extras.getpagespeed.com

my hunch is that your router, firewall (ACL's could be being applied from a firewall within your router that does double duty, or a dedicated firewall at the edge of your network, depending on your network topology) has a rule that is catching your HTTPS request and dropping the packets.

I don't think this is a routing issue because we are A- able to reach resolvers just fine, and resolve extras.getpagespeed.com, just fine, and B- we can get port 80 traffic to google, so your default route is working.

frontsidebus
  • 496
  • 2
  • 7
  • Thank's for response. Telnet command fails. I've added iptables rules into quetsion. Can they cause problems? – userlond Dec 12 '18 at 00:30
  • I don't actually think iptables is the problem. I tested from a centOS VM using the exact iptables rules you posted, and I'm still able to reach `extras.getpagespeed.com` via https. – frontsidebus Dec 12 '18 at 03:56
  • Traceroute fails, I'll add output to question. – userlond Dec 13 '18 at 05:02
  • One more thing to try. Since it looks like trace route is failing, likely because your icmp packets are being filtered. You might try installing tcptraceroute and then doing a tcptraceroute on port 443: `sudo tcptraceroute extras.getpagespeed.com 443` – frontsidebus Dec 17 '18 at 01:56
  • Tcptraceroute gives me the same "stars" unfortunately, thanks for responses anyway. – userlond Dec 17 '18 at 03:25
0
  • DNS recolving

DNS recoving seems to be working. You already changed yum config to use IPv4.

  • firewall rules on server

Next to check the firewall (make focus on OUTPUT / outgoing connection). You can try one of following :

iptables-save
iptables -S ; iptables -t nat -S
  • connection

You have mentioned that on workstation it is working. How are the server and the workstation connected? Is it located behind the same connection ? Are you using VLANs? Is there any ACL on router / firewall handling the connection (and / or VLAN routing) ? Is the default GW the same ? Where is the communication terminated (traceroute) - at least what ICMP show...

ip route show
ip address show
traceroute extras.getpagespeed.com

just in case... Isn't there any VPN session active on server / Workstation so the result may be impacted by it (it would be visible in output of previous commands)?

  • one more check

I have tried to reach it on http (port 80) and there can be seen at least redirect for https schema. Try to reach it on port 80. I would not help at the end but can help to trouble shoot if it is something related specifically to port 443. 

    $ curl -4iv http://extras.getpagespeed.com/release-el7-latest.rpm >/dev/null 

    * About to connect() to extras.getpagespeed.com port 80 (#0)
    *   Trying 104.18.49.227...
    * Connected to extras.getpagespeed.com (104.18.49.227) port 80 (#0)
    > GET /release-el7-latest.rpm HTTP/1.1
    > User-Agent: curl/7.29.0
    > Host: extras.getpagespeed.com
    > Accept: */*
    > 
    < HTTP/1.1 301 Moved Permanently
    < Date: Wed, 12 Dec 2018 00:42:10 GMT
    < Content-Length: 0
    < Connection: keep-alive
    < Set-Cookie: __cfduid=d1db394ac95ecd334f0739681096b64ba1544575330; expires=Thu, 12-Dec-19 00:42:10 GMT; path=/; domain=.getpagespeed.com; HttpOnly
    < X-Varnish: 1149403
    < Location: https://extras.getpagespeed.com/release-el7-latest.rpm
    < CF-Cache-Status: MISS
    < Server: cloudflare
    < CF-RAY: 487c264520a7c274-FRA
    < 
    * Connection #0 to host extras.getpagespeed.com left intact

Note: As it is located in CloudFlare there may be some limitation on connection in case it is recognized as potential attack - improper communication, rate of the requests, etc.

Kamil J
  • 1,587
  • 1
  • 4
  • 10
  • 1
    Based on your Update4 ````firewall rules on server```` would not be an issue. Anyway in theory there can be some blocking rule on firewall / router but the most probably it will not be a case... try other stuff I have mentioned also that **http** connection to see the header. – Kamil J Dec 12 '18 at 00:59
  • I'm with you on the DNS resolution looking good and checking the firewall. I don't, however think that this is a cloudflare issue, since I can reach extras.getpagespeed.com via HTTPS request (on port 443) from multiple cloud providers and regions. (I tested ec2 us-east2, DO - Frankfurt, and DO - San Francisco. – frontsidebus Dec 12 '18 at 03:45
  • Cloudflare as a service is ok. Thay are doing some online analysis and providing (at least some) security level as a service directly. The result may be than one source IP (or session as they are using also headers /cookies) may be limited on access while othe may work without any limitation. In case original server from the request has some communication history already it may face other rule set then any other endpoint including your nodes you have tried... Interesting may be if mentioned workstation share connection / IP. – Kamil J Dec 12 '18 at 08:00
  • Begin from the end. `curl -4iv ... ` fails.Traceroute also fails, I'll add the output to the question. – userlond Dec 13 '18 at 05:01
  • Workstation is connected to server via ssh/sftp. It's leased VPS, not dedicated standalone host. – userlond Dec 13 '18 at 08:07
  • 1
    Hmm ok, it seems to me really like some kind ban on IP of your VPS server on CloudFlare side ... Try to open ssh with parameter ````ssh -R 12345:extras.getpagespeed.com:443 ...````. It will open remote port (on VPS) listeing on port 12345. Then add temporary record to /etc/hosts file pointing extras.getpagespeed.com to localhost and try to open the connection to port 12345 (https). In case the VPS ip would be really ban this would help (the connection will go through the ssh session you have created and "from" your client IP. In case of success connection you cannot do much on server side :-(. – Kamil J Dec 15 '18 at 22:16