13

What is the proper and safe way to perform a DDoS test without crashing the whole infrastructure?

What different types of DDoS attacks are there and what things should be considered performing such a test?

Also, where can you rent a "botnet" or a similar (cloud) service to perform a realistic test?

Bob Ortiz
  • 6,234
  • 8
  • 43
  • 90
  • 4
    Test it on a replica of the production environment. Since the objective of a (D)DoS is to crash the service, you can't do a proper test unless you genuinely attempt to crash it. Since you probably can't afford that happening on the production environment, do it on a replica where downtime isn't an issue. – André Borie Jul 12 '16 at 18:10
  • 1
    alternatively you could do it by profiling your code and then determining the effects of someone repeatedly hitting computationally expensive functions over and over again. – CaffeineAddiction Jul 12 '16 at 18:14
  • 1
    @CaffeineAddiction that's still not 100% guaranteed. At some point you should just do exactly what an attacker would do and observe the effects. – André Borie Jul 12 '16 at 18:17
  • 1
    @AndréBorie true enough, but where does one go to 'rent' a botnet for a true DDoS attack? – CaffeineAddiction Jul 12 '16 at 18:22
  • @CaffeineAddiction from inside your network you can use hardware you already have. From outside there are legal "botnet as a service" companies exactly for this purpose. Example: Load Impact https://loadimpact.com/ – André Borie Jul 12 '16 at 18:23
  • @AndréBorie loadimpact is not really performing different types of DDoS attacks right? Like SynFloods, PingOfDeath, Smurf, UDP floods, possible others? – Bob Ortiz Jul 12 '16 at 18:25
  • @EvanderConsus I don't know, you'd have to ask them. Of course all they do is rent AWS servers, so you could do the same yourself and then you're free to do whatever attacks you want. – André Borie Jul 12 '16 at 18:28
  • @EvanderConsus Not sure if this was a typo but those are all examples of DoS attacks and not DDoS attacks. DoS attacks like those are largely based off weaknesses with the networking protocols and are mostly obsolete these days where DDoS is almost always a flood of traffic intended to knock your server offline that looks indistinguishable from legitimate traffic. – DKNUCKLES Jul 12 '16 at 19:26
  • 1
    A friend of mine works at a company Red Wolf Security that offers a controlled DDOS against production services. – eel ghEEz Jul 12 '16 at 21:12

4 Answers4

12

If you are performing load testing at high packet rates, the most safe way is to isolate it completely from the rest of the network. For example, you can connect two servers by direct 10GBps link without switch, and use another LAN connection on benchmarking server to ssh to one server to run the test.

Another way is to provision servers in Public Cloud like AWS for short period of time and run it there. You don't risk impacting your infrastructure. It's OK to do it, cloud infrastructures are resilient so if you flood your server with packets it's just small bit in a lot more of traffic. Of course if would be good to provision big, fast machine, or even dedicated one so that you can really see how much load your software takes.

Regarding software, you can use generic ab which is "Apache Bench" and it's part of Apache packages. You can test resilience of your web server (static files from httpd) and application server (dynamic ones from PHP, Ruby, Java). You might look for other, professional load testing software for specific protocols which may include API and video streaming testing. There are many of them depending on what protocol you are using (like REST etc).

Then proceed with usual network stack tuning as well application tuning to gain more performance (same on the client side like ulimit). Keep track record of your results. And also, try to analyze the logs properly with either free software like AWStats or commercial Sawmill. Avoid using grep ;-) Analytics is what will show you real results. Also note that ab is good for httpd but sometimes has issues with other web servers. And in AWS you can use Load Balancer which shows real-time stats as well. Record network stats with Nagios, Zabbix etc to see how the network stack performed (e.g. dropped connections, packet rate, CPU usage etc).

Never hire DDoS attacks as these are criminal gangs and you risk a lot, not only legally for yourself but also for your ISP, your business and your customers.

Aria
  • 2,706
  • 11
  • 19
5

The single best plan is not to do this. It rarely serves any purpose, as DDoS will work against you. That's a given. It is so easy to create a high volume attack now it's not even worth trying to do more than the usual load tests.

You are much better served by implementing DDoS mitigation.

If, however, you must test, try the following:

  • test a non-production copy
  • build a multiplier into load testing when running OAT/UAT prior to go-live
Rory Alsop
  • 61,367
  • 12
  • 115
  • 320
  • 1
    Agreed - IMO if you have the money to build a replica of your production environment for testing purposes, you have the money to pay for DDoS mitigation technologies. – DKNUCKLES Jul 12 '16 at 19:21
2

Fundamentally, the way a Distributed Denial of Service works is by flooding a companies bandwidth with to much traffic. The actual attacks may differ in source and style, but they share the same goal. There is no point in testing a DDoS attack, because no matter how much bandwidth you have it is always possible for it to be overwhelmed. It depends entirely on the physical capabilities of the hardware.

Now, that isn't to say that there is no way to mitigate a DDoS. You can purchase services from various companies that are capable of detecting a DDoS, and sending your traffic to a scrubbing center to only let legitimate traffic through. However, these services are still limited by the amount of bandwidth that flows into their equipment. Unlike other Denial of Service attacks, there is no software fix for a DDoS.

Do not try to hire services to DDoS your network. These services are at best legally questionable, and at worst are organized crime. If you insist on testing your network, other answers provide perfectly reasonable ways to go about it.

TheGiantPossum
  • 139
  • 3
1

Beside asking the how you should consider asking the why.

Case 1: you want to check how your service will be impacted by a DDoS -- this is possibly a reasonable thing to do. I would say that these are load tests taken to the extreme.

In that case you need to test a clone of your whole service, including the infrastructure. You do not know what will fail between your service frontend in a DMZ and the backend database in a LAN. It can be the application, a firewall on the way, the database,... As you can imagine, it will not be a simple task and the outcome may or may not be interesting.

You do not need a DDoS for that - a multiple DoS will be enough (you are interested only by the load)

Case 2: you want to assess your capacities to resit a DDoS -- in that case the test is useless.

If the attack is volume based (as opposed to something like SLOWLORIS) then the moment you detect a packet incoming, it is too late. There are companies which will sell some vaporous appliances which magically fix the problem but they do not work*.

You need to block this attack upstream, so

  • you should talk to your ISP for upstream traffic blackholing (passably useful, maybe to avoid a crash further down your application)
  • or look for some cloud based solutions which may work or not - they are in practical terms your only solution (your ISP may provide such a solution as well).

*the idea is that they will be fast enough (= way faster than your service processing capacities) to decide whether a packet is "good enough" or not to be accepted. Beside the capacity to correctly decide, the premise breaks when your network pipe is full.

WoJ
  • 8,957
  • 2
  • 32
  • 51