This Serverfault question has some good general tips on DDoS mitigation - there are lots of things you can try but a significant DDoS requires help from your ISP, as mentioned.
It would help to know the web server you are using on Windows and Linux - presumably IIS and Apache.
A few options in decreasing order of usefulness:
- Use Linux kernel firewall (iptables) to block - the xtables-addons approach is simple but ipset can handle larger numbers of IP address ranges. Similar setup on Windows. Using a separate physical firewall would be better, then it could front-end both Linux and Windows, reducing setup and offloading the servers.
- Use mod_security on Apache - this could potentially work across Windows and Linux as long as you use Apache on both. Since using GeoIP involves quite a lot of setup to keep the GeoIP blocks up to date, this could reduce overall maintenance once configured.
- Use DNS server to block on GeoIP - OK for a casual DoS that uses your domain name. However this is useless against anyone simply using your IP address to DOS you.
See this question on GeoIP blocking as well and the ddos and geoip tags (also added to your question.)
A DDoS mitigation service (aka "clean pipes" service) may be the best option for serious DDoS: they front-end your traffic and filter out the DDoS, leaving you only with valid site traffic, subject to how well they filter. They have huge pipes and are focused on this problem so will probably do a better job than an in-house solution, and a lot depends on having a big enough pipe to absorb a DDoS so their hardware/software can filter it. BlockDOS.net is a reasonably priced service, Prolexic and Verisign are more top-end and much more expensive.
If that's too expensive, it might help to re-host onto Amazon EC2, which can route all traffic via a specific EC2 instance (like a VPS) - there's a specific AMI (VPS image) that is intended to frontend your web servers, which would be on separate EC2 instances. Amazon EC2 also make it possible to spin-up new server instances to handle the increased load. You might still get charged for the DDoS traffic (possibly more than for a mitigation service), and would pay for the extra servers, so this needs some investigation. Other cloud VPS providers may have better DDOS policies or in-house DDoS mitigation services.