Preventing Sendmail Brute Force Attack on Solaris 10

Question

I want to dynamically block specific connections that use the same IP address based on a rate or connection limit. Is this possible using Solaris/IPF or some sendmail extension? I want to limit sendmail login attempts to prevent brute force attacks.

In Linux it's easily handled on the iptables firewall layer, but I haven't been able to figure out a way to use ipf to limit it on the firewall layer. Sendmail has a built-in rate limit and connection limit, but it appears to be applied to all users so if we're experiencing a DOS or DDOS it would block all our users instead of just the attacker.

Answer 1

I solve this by adding another rule in syslog / rsyslog to pipe mail.* messages to a fifo in /etc/mail/mailban/syslog_fifo

I then made a daemon to read syslog_fifo, parse the sendmail messages, and act on what is found. The history of each ip address and activity is tracked through a 1.5 million(!) row mysql table. Offending ip addresses are added to a ban chain in iptables for various durations/ports depending on various criteria, and life goes on sweetly...

A simple cron task runs every hour and releases old ip addresses and updates the status in the database accordingly.

Now I made the software automatically compile and sanitize log records, find the responsible abuse address for the ip and then sends a report informing the ISP of unsociable behaviour. It works in about 5% of cases, and helps a little to clean up the net.

This also then requires a blacklist of delinquent ISPs that don't accept or ignore these type of complaints, and this evolves over time.

I also use another similar solution for ssh, pop3, httpd attacks.

I don't know of any other software that does this, but I could help you develop a solution although I'm most familiar with RedHat/Fedora.