As far as I know, I have never heard of or seen any large scale web sites like Amazon, Microsoft, Apple, Google, or Ebay ever suffer from DDoS. Have you?
I have a personal philosophy that the bigger you are, the more of a target you are for such attacks. Imagine the brownie points you would get if you could bring down a major website.
Yet, such sites have always remained sturdy and seemly invincible. What security measures have they implemented and can these be applied to smaller businesses?
They generally have a very layered approach. Here are some things I've either implemented or seen implemented at large organizations. To your specific question on smaller businesses you generally would find a 3rd party provider to protect you. Depending on your use case this may be a cloud provider, a CDN, a BGP routed solution, or a DNS-based solution.
Bandwidth Oversubscription - This one is fairly straightforward. As you grow larger, your bandwidth costs drop. Generally large organizations will lease a significantly larger capacity than they need to account for growth and DDoS attacks. If an attacker is unable to muster enough traffic to overwhelm this, a volumetric attack is generally ineffective.
Automated Mitigation - Many tools will monitor netflow data from routers and other data sources to determine a baseline for traffic. If traffic patterns step out of these zones, DDoS mitigation tools can attract the traffic to them using BGP or other mechanisms and filter out noise. They then pass the clean traffic further into the network. These tools can generally detect both volumetric attacks, and more insidious attacks such as slowloris.
Upstream Blackholing - There are ways to filter UDP traffic using router blackholing. I've seen situations where a business has no need to receive UDP traffic (i.e. NTP and DNS) to their infrastructure, so they have their transit providers blackhole all of this traffic. The largest volumetric attacks out there are generally reflected NTP or DNS amplification attacks.
Third Party Provider - Even many fairly large organizations fear that monster 300 Gbps attack. They often implement either a DNS-based redirect service or a BGP-based service to protect them in case they suffer a sustained attack. I would say CDN providers also fall under this umbrella, since they can help an organization stay online during an attack.
System Hardening - You can often configure both your operating system and your applications to be more resilient to application layer DDoS attacks. Things such as ensuring enough inodes on your Linux server to configuring the right number of Apache worker threads can help make it harder for an attacker to take down your service.
While there are no real counter measures for DDOS, there are someways to control it.
First is by using a Content Delivery Network, Using several data centers across the world to serve contents to visitors from different geographical areas. This helps to eliminate single point of failure and makes it harder to exhaust resources or saturate the links and balance the attack load.
Another way is to work closely with major backbones, ISPs and respective organizations to block the attacker IPs in the most specific network as possible to prevent their traffic from reaching their targets.
Hope it helps.
As a mid-sized company, we use a DOS mitigation service to reduce the risk of our website from being knocked offline. Our site resolves to the provider's IP address. The provider then forwards the request to our webserver. Our webserver only communicate with the provider.
They then use their tools to determine if certain attacks are actual attacks by using a variety of monitoring and correlation tools. If there is deemed to be an attack, the provider does not forward the request to our web servers and soaks up the attack. In order to be able to perform this type of mitigation, your capacity must exceed that of what the attacker is trying to deliver. With larger companies that normally expect a larger bandwidth capacity, I would expect that they either outsource to ISPs or create an internal system to perform the same mitigation strategy.
My company has dealt with DDoS attacks up to 180gbps and here are my techniques that I have used to mitigate.
The size of a website doesn't only make it a bigger target, things that also play a significant role are:
Motives for DDoS attacks include but are not limited to the following:
Also (from one of the comments):
There are many different types of DDoS attacks and how they are initiated, first you need to get the points I listed above in order, then your DDoS attacks will likely decrease. This doesn't mean that you will not experience them anymore, you simply give people less of a motive to attack you.
On a technical level, there are multiple things to consider because most businesses have multiple nodes in their infrastructure. In some cases, each node requires a different type of approach. In my case these nodes were an API, a game server, an authentication server, a database, and a social server. Step 1 was to make sure that you are never exposing an IP address that does not need to be exposed. In my case those were the authentication server, database and social server. Generally limiting the points of failure is a good approach to start with. Protection is incredibly expensive, and it's only good to have the most resilient protection where you really need it most.
After you have determined which points are required to be public, you can protect each function individually in the way they need to be protected. theterriblevitrium gave an excellent answer to techniques, here are my 2 cents on that one.
Feel free to ask any questions!
I'm not sure if this was what you were getting at with Automated Mitigation mentioned by @theterribletrivium, but they also use load balancers to distribute traffic evenly to separate servers in order for them to run as fast as possible.
Although it's not the most effective way to evenly distribute users to servers, Google uses what is called Round-robin DNS. Round-robin DNS will return multiple IP addresses and the user will connect to one of those IP addresses. The problem with this however is that the same IP address can be determined by multiple computers to connect to, rendering the other servers faster and not used.
They use a similar setup for dealing with the large amounts of information that is stored. Google uses what it calls BigTable to store information related to Google Maps, Blogger, YouTube, GMail and more. It is reported that Google uses hundreds of thousands servers in order to store all of this information and to make their websites running as fast as possible.
They use software (that they probably developed themselves) to host their websites and without using a large amount of memory and CPU. The most popular web server, Apache, is most definitely not used by these large websites because of how it fails handle such heavy loads and it is affected by the C10k problem. The C10k problem causes web servers (such as Apache) to fail and sometimes shutdown when more than 10,000 connections are made to the web server at the same time (which Google probably has a lot more than 10,000 connections at any one time).
The servers and the hardware they use are top of the line. According to the Wikipedia article on the Google platform though, Google doesn't use hardware that performs the best, they use hardware that is the best bang for the buck.
If you think about it, websites such as Google, Amazon, Microsoft, and Apple are technically always under a DDoS attack. But they have such advanced technologies in place that allow their websites to be accessed by everyone without being shutdown.
