0

Every now and then, I check my munin graphs on my production server to see if there's anything out of the ordinary going on. Today I noticed a strange thing. Identical traffic spikes on all (8) interfaces/ip addresses simultaneously.

Click to see Munin graph

The apache logs don't show any traffic. But if I do a tcpdump for each ip address, I see the exact same thing. I see thousands of these lines:

18:40:50.737105 IP 104.27.163.111.43691 > (mydomain).com.domain:  47751+ [1au] ANY? msn.com. (36)
18:40:50.737198 IP (mydomain).com.domain > 104.27.163.111.43691:  47751- 0/13/1 (247)
18:40:50.756342 IP 104.27.163.111.53917 > (mydomain).com.domain:  7559+ [1au] ANY? msn.com. (36)
18:40:50.756432 IP (mydomain).com.domain > 104.27.163.111.53917:  7559- 0/13/1 (247)
18:40:50.764843 IP 104.27.163.111.38729 > (mydomain).com.domain:  59015+ [1au] ANY? msn.com. (36)
18:40:50.764929 IP (mydomain).com.domain > 104.27.163.111.38729:  59015- 0/13/1 (247)
18:40:50.797139 IP 104.27.163.111.9549 > (mydomain).com.domain:  17799+ [1au] ANY? msn.com. (36)
18:40:50.797253 IP (mydomain).com.domain > 104.27.163.111.9549:  17799- 0/13/1 (247)
18:40:50.800387 IP 104.27.163.111.59499 > (mydomain).com.domain:  33159+ [1au] ANY? msn.com. (36)
18:40:50.800482 IP (mydomain).com.domain > 104.27.163.111.59499:  33159- 0/13/1 (247)
18:40:50.805491 IP 104.27.163.111.17477 > (mydomain).com.domain:  13703+ [1au] ANY? msn.com. (36)
18:40:50.805585 IP (mydomain).com.domain > 104.27.163.111.17477:  13703- 0/13/1 (247)
18:40:50.815385 IP 104.27.163.111.15286 > (mydomain).com.domain:  52359+ [1au] ANY? msn.com. (36)
18:40:50.815481 IP (mydomain).com.domain > 104.27.163.111.15286:  52359- 0/13/1 (247)
18:40:50.816789 IP 104.27.163.111.17279 > (mydomain).com.domain:  45959+ [1au] ANY? msn.com. (36)
18:40:50.816882 IP (mydomain).com.domain > 104.27.163.111.17279:  45959- 0/13/1 (247)
18:40:50.818406 IP 104.27.163.111.33210 > (mydomain).com.domain:  28039+ [1au] ANY? msn.com. (36)
18:40:50.818500 IP (mydomain).com.domain > 104.27.163.111.33210:  28039- 0/13/1 (247)
18:40:50.823323 IP 104.27.163.111.42996 > (mydomain).com.domain:  17287+ [1au] ANY? msn.com. (36)
18:40:50.823417 IP (mydomain).com.domain > 104.27.163.111.42996:  17287- 0/13/1 (247)
18:40:50.837159 IP 104.27.163.111.13627 > (mydomain).com.domain:  22151+ [1au] ANY? msn.com. (36)
18:40:50.837254 IP (mydomain).com.domain > 104.27.163.111.13627:  22151- 0/13/1 (247)
18:40:50.841576 IP 104.27.163.111.38221 > (mydomain).com.domain:  24967+ [1au] ANY? msn.com. (36)
18:40:50.841701 IP (mydomain).com.domain > 104.27.163.111.38221:  24967- 0/13/1 (247)
18:40:50.846206 IP 104.27.163.111.15500 > (mydomain).com.domain:  19079+ [1au] ANY? msn.com. (36)
18:40:50.846299 IP (mydomain).com.domain > 104.27.163.111.15500:  19079- 0/13/1 (247)
18:40:50.956163 IP 104.27.163.111.9152 > (mydomain).com.domain:  60295+ [1au] ANY? msn.com. (36)
18:40:50.956257 IP (mydomain).com.domain > 104.27.163.111.9152:  60295- 0/13/1 (247)

I see this on all ip addresses. Does any one know what's going on?

Zippy1970
  • 157
  • 2
  • 8
  • Looks like someone is trying to query your server for DNS records. If you do not have a DNS server running, I would first block port 53 in the firewall. – Thomas Oct 06 '18 at 17:40

2 Answers2

1

Your DNS server is being used for traffic amplification attack targeted at CloudFlare servers. As you can see, an attacker sends you 36 data bytes from a forged source IP and CloudFlare receives an unsolicited response from you with 246 data bytes in lenght. 246/32 equals circa 7.5 times amplification of the attack bandwidth (i.e. I feed you 1 Mbps, you feed CloudFlare 8 Mbps).

What you should really do, is disable recursive queries from any addresses except your trusted ones and only allow queries from untrusted sources for your hosted domain zones (primary and secondary).

Peter Zhabin
  • 2,276
  • 8
  • 10
  • I added this to my named.conf.options file: `allow-transfer{"none";}; allow-recursion {"none";}; recursion no;` But that didn't help. – Zippy1970 Oct 12 '18 at 15:29
  • Wait, I just noticed outside recursion wasn't even allowed in the first place. I had both `acl recurseallow{ 127.0.0.1; };` and `allow-recursion{ recurseallow; }; recursion yes;` in named.conf.options. – Zippy1970 Oct 12 '18 at 15:48
  • Please test your DNS server with [OpenResolver](http://openresolver.com/) to see your recursion status. Depending on your configuration and BIND version you may need to limit `allow-query-cache` as well. You will still see the inbound traffic, but amount of outbound traffic should be minimal. – Peter Zhabin Oct 12 '18 at 17:01
0

Amplifiers are so amplifiers. This is the Internet, this is hostile and toxic environment. They are asking ANY msn.com query which suppose to return a large answer by UDP using spoofed IP address. This is common the practice to use open DNS resolvers for amplification attack. It works because UDP DNS queries doesn't require a handshake or some sort of it.

First rule: Do not open recursive DNS server into outside world without limiting queries rate and size.

Second rule: Do not open recursive DNS service into outside world if you don't know why you are doing it.

Third rule: If you open recursive DNS service for your clients, always set an ACL using your DNS server implementation itself or the firewall behind.

Yuki Nagato
  • 1