5

I've heard about new kind of DDOS where ntp is used for reflection .

My questions are really simple :

  1. Can you please give details on how they work and clarify? Since ntp is ran over UDP, I suppose there must be some kind of forged packet somewhere?

  2. How is it possible to exhaustively check if some ntp server are vulnerable (and can't be involved in any attack)?

  3. If ever we become the target of such attack, is there any way to mitigate?

As this kind of attack has been widely used in 2014, here are a few more details :

  • You can find more information ont this cve.
  • "Yesterday evening 01/30/2014, starting at 22:15 CET, Witbe network in Paris has been seriously affected by a Distributed Denial of Service (DDOS) attack using NTP amplification."
  • Ouch, 350Gpbs, that hurts http://www.itnews.com.au/News/372033,worlds-largest-ddos-strikes-us-europe.aspx
  • Generic behaviour about ddos can be found here : I am under DDoS. What can I do?
  • Bbc talks about ntp attacks : http://www.bbc.com/news/technology-26662051
  • One more question : If I record correctly, openntpd listens by default on 127.0.0.1 and ntpd listens on 0.0.0.0 - I'm not really sure if all ntp servers involved in reflection ddos needed to serve time publically - Merely, I think some unskilled sysadmins installed ntp to synchronize time locally and left default configuration files.

A simple way to prevent and mitigate this kind of problem would be to listen on 127.0.0.1 by default? I guess this is true for any service (bind9, mysql, ...)?

Community
  • 1
  • 1
    Security folks can use [Nmap's ntp-monlist script](http://nmap.org/nsedoc/scripts/ntp-monlist.html) to find offenders on IP addresses they own. An ISP can check its own AS with this command: `nmap -sU -p123 --script ntp-monlist,targets-asn --script-args targets-asn.asn=1234,newtargets --open -oA ntp-results` – bonsaiviking Mar 20 '14 at 18:19

3 Answers3

15

These attacks have been around for ages, they just became popular again the last couple of months. They work like any regular amplification attack: a host spoofs a query so that the source IP address seems to be the targetted host. The NTP server sends its answer to the spoofed address. Since the answer for specific query types can be quite large and usually is UDP, this can quite rapidly become a problem for the targetted host: it's being swamped with NTP packets.

Unfortunately this isn't a vulnerability in NTP servers, it's just a feature which is being abused. One thing to consider is if you need to run NTP servers which can be queried from the entire internet. If that's not needed, create an access list or firewall policy to block queries coming from untrusted sources. Then, what you can do to check if your NTP servers are vulnerable is doing NTP queries from untrusted sources and verify if you get an answer. But unfortunately there are quite a number of NTP servers which are public by intent (e.g. all servers in pool.ntp.org). If you need to run a public NTP server, you can consider implementing query ratelimiting to reduce impact to the targetted host in case of abuse.

Another, more generic part of the solution is that networks need to implement BCP38, which tells them to filter traffic leaving their networks so sending spoofed packets is impossible. Unfortunately, there are still a large number of networks which do not implement this kind of filtering, so all attacks with spoofed source packets (using any protocol like NTP, DNS or chargen) are still possible.

What you can do to mitigate such an attack depends a bit on your network and tools available, but one thing you should consider is blocking incoming NTP packets from untrusted sources (so check which NTP servers you're using). Of course, this doesn't help if your uplink is congested. In that case, you'll need to ask your ISP to help you filter the traffic.

Teun Vink
  • 1,837
  • 11
  • 14
  • Thanks a lot. What about monlist? Is it the reason why ntp attack became popular? –  Dec 31 '13 at 12:53
  • Monlist is one of the commands often used. The reason it became popular is that there are many NTP servers around which allow queries from the entire internet and the amplification factor is quite high with commands like monlist, so it's easy to abuse and effective. – Teun Vink Dec 31 '13 at 14:05
  • 1
    @TeunVink This is the worst thing i have read in a discussion of ntp DDOS: "But unfortunately there are quite a number of NTP servers which are public by intent." The internet needs public ntp servers. This DDOS has nothing to do with serving time to the public. The problem is allowing monlist queries from untrusted hosts. An ntp server operator can enable noquery in order prevent being used as a traffic amplifier and still provide time services to the public. – dfc Feb 14 '14 at 18:56
  • 1
    @TeunVink Please see these posts from cloudflare for accurate information. This answer is very misleading if not outright incorrect: https://blog.cloudflare.com/understanding-and-mitigating-ntp-based-ddos-attacks https://blog.cloudflare.com/technical-details-behind-a-400gbps-ntp-amplification-ddos-attack – dfc Feb 14 '14 at 18:57
  • @dfc on what point, exactly? What I described are attacks which I've seen 'in the wild', just like many other people at other companies I'm in contact with though anti-abuse workgroups. Cloudflare describes specific types of NTP attacks they saw which use techniques I described to make attacks even larger. Those are not the only techniques around. Also keep in mind Cloudflare has a specific interest in combatting DDoS'es, it's the way they make money. – Teun Vink Feb 14 '14 at 19:19
  • How can a public ntp server be used for a DDOS when noquery enabled? Please provide amplification factors when noquery is enabled. Why is it unfortunate that some ntp servers provide a public service? What does cloudflare's business model have to do with the accuracy of that information? If cloudflare is in the business of combatting DDOSs it seems like they would want to inflate the appearance of problems, not limit them. – dfc Feb 14 '14 at 19:47
  • I never mentioned 'noquery' in my original post and I never claimed that it was not possible to properly configure NTP servers. What I said is that a lot of NTP servers are (sometimes by default) misconfigured allowing external hosts to query them. Often the owner of the server isn't even aware of this.You're right that 'noquery' solves this, unfortunately they often do not configure this. However, I've tracked down dozens of hosts (just like http://openntpproject.org does) in my networks which were unintentional open NTP servers being used in attacks using monlist. – Teun Vink Feb 14 '14 at 19:55
  • There's nothing wrong with offering public NTP servers, as long as it's done responsibly (ratelimiting queries, restricting commands, etc), I never claimed that. There is a problem with public NTP servers without these restrictions and ratelimits. My problem with Cloudflare is that they have a habit of showing off the size of attacks in public while complaining less publicly to other networks about the problems they suffer from when being attacked. – Teun Vink Feb 14 '14 at 19:57
  • The fact that you never mentioned noquery in your answer is the problem! So your problem with cloudflare has nothing to do with the accuracy o the information provided in the links I gave? – dfc Feb 14 '14 at 20:05
  • Fair enough, mentioning noquery would have been a good addition. My other problem with your Cloudflare remark was that you used those posts as if that's the only way NTP is abused. That part has nothing to do with Cloudflare itself. But perhaps I shouldn't have spoken about Cloudflare's motives. – Teun Vink Feb 14 '14 at 20:11
2

My answers:

  1. The attacks use monlist commands (which will show as ntpv2 reserved in tcpdump). These commands are not rate limited by the normal rate limiting. Monlist (and other monitoring commands) will only work from IPs that are allowed to 'query' your server so add 'noquery' to your defaults.
  2. Try a ntpdc -nc monlist yourip from an outside IP to see whether your server answers.
  3. Ratelimit your incoming ntp traffic. Not in the ntpd itself but before it reaches the daemon. How to set this up in linux is discussed in "my rate limiting setup" on the ntp pool mailing list
Koos van den Hout
  • 1,086
  • 6
  • 9
1
  1. Can you please give details on how they work and clarify? Since ntp is ran over UDP, I suppose there must be some kind of forged packet somewhere?

The US-CERT has a great description of this attack at "Alert (TA14-017A) UDP-based Amplification Attacks" and "Alert (TA14-013A) NTP Amplification Attacks Using CVE-2013-5211".

TA14-013A says it best:

Description

UDP, by design, is a connection-less protocol that does not validate source IP addresses. Unless the application-layer protocol uses countermeasures such as session initiation, it is very easy to forge the IP packet datagram to include an arbitrary source IP address [7]. When many UDP packets have their source IP address forged to a single address, the server responds to that victim, creating a reflected Denial of Service (DoS) Attack.

Recently, certain UDP protocols have been found to have particular responses to certain commands that are much larger than the initial request. Where before, attackers were limited linearly by the number of packets directly sent to the target to conduct a DoS attack, now a single packet can generate tens or hundreds of times the bandwidth in its response. This is called an amplification attack, and when combined with a reflective DoS attack on a large scale it makes it relatively easy to conduct DDoS attacks.

The included chart regarding "bandwidth amplification factor (BAF)" shows that NTP exhibits the worst behavior. Since NTP is an extremely common protocol, and many Linux OSes ship with an active NTP server, this problem is particularly bad.

Stefan Lasiewski
  • 22,949
  • 38
  • 129
  • 184