There are a few things, options, settings etc; that you could set in order to improve application and underlying server hardening. Here are my recommendations on what I believe to be a good balance between easy implementation and effectiveness.
Limiting Connections
The biggest instant impact, that may not cause too much difficult to configure depending on the specific option you go for, would be to limit who can connect to the application.
Black listing is certainly an option but as you have described the attacker is constantly changing IP addresses. I would recommend you do not do this.
A white list would be a better suggestion and allowing only certain IPs to access the application. This would stop the attack in its tracks.
You could combine this with a VPN, this would mean those working from home (with most likely dynamic IPs) could easily connect to the application without having to get someone to give them access.
Secure Passwords
If this slow attack has in anyway resulted in an account being breached, you really need to question your password policy. There are many many references on this site giving password advice, I will not go in depth here.
If your password is worth its salt (haha) this attack shouldn't phase you.
Change the default username
If you can, change the default username from admin, you will throw a lot of people. I mean, first user I always try is admin.
Do Not!
- Do not disable the account after X tries, you are just asking for a DoS.
- Do not (or at least avoid) blacklist so many IP addresses, they are probably popular VPN services, which many people use a lot.