0

I have a new CentOS7 web server running virtualmin. It hosts a new WordPress site that just went into production.

Starting mid-day today, on the quarter hour, to the second, I am troubleshooting spikes in server load. At various time, but mostly during these spikes, the access_log file fills with hundreds of entries like this:

50.84.83.122 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
50.79.197.249 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
12.147.195.3 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
207.91.154.164 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
173.241.65.202 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
12.19.212.186 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
71.78.224.10 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"
208.69.145.81 - - [25/Mar/2020:22:00:09 -0700] "GET / HTTP/1.1" 301 - "-" "-"

I can't tell if these entries are the cause or the result of spikes in server load, or something else entirely. The lack of user agent data is also puzzling.

I have eliminated wp-cron.php as the culprit (disabled cron and spike still happens, ran manually, doesn't spike).

I've removed all custom 301 redirects in my .htaccess file, disabled a 301 redirect plugin, still get the entries. The only redirect in place right now are via WordPress General Settings where I have https://example.com as the WordPress and Site URLs.

What do these entries mean and how do they relate to spikes in server load?

Edit: addendum - if I search these IP addresses in the rest of the access log, they only have these repeated 301 entries (all the ones I have checked are every few seconds or minutes for over 48 hours) and no requests for any other web pages. I'm almost thinking this is an attack? Would that make sense? There are two other sites on this server (same IP, different SNI) that have normal-looking logs without the 301 floods. I spot-checked IP addresses to see if they are on any blacklists and they came back clean.

Rocky
  • 101
  • 2
  • 1
    check if those entries come from monitoring tools, maybe some webservice which checks latency and/or availability. – Dennis Nolte Mar 26 '20 at 07:17
  • I do have a monitoring service but it inserts a user agent. I've been pouring over logs from this server and the previous server that hosted the same site, which has similar logs. The spikes started happening ~60 hours after DNS changed, and increased sharply over the course of a few hours. My tinfoil hat is making this look more and more like a cheesy DDoS. Filtering out "-" as a user agent in .htaccess his mitigated the issue for the moment. – Rocky Mar 26 '20 at 13:22

1 Answers1

0

It's a DDoS Attack.

With the help of a security consultant, we identified and mitigated a bot attack on a single web site. For future reference, if anyone has a similar issue, these httpd entries were just one clue. That the IP addresses never requested another page, that the logs from a previous host had similar entries, analysis of DNS logs, plus a deep analysis of httpd logs and more, all led to this conclusion. The attack has been mitigated at the network level.

Rocky
  • 101
  • 2