Do as many mitigations as you can.
Your goal is to force a wide variety of potential attackers to spend more effort, resources, computer time (and thus electric bill), and most especially "skilled" (rare or scarce, and thus valuable) person-hours. No one protection works against every threat. Not every threat can be protected against while still remaining on the internet.
However, a large number of threats (script kiddies and low skilled or low resourced attackers) can be defeated almost all the time by having a defense in depth - many layers of good, solid security, each of which should - theoretically - be enough to defeat many threats on its own. Thus, when your system's iptables rules let someone in by accident, the SSHD certificate authentication keeps them out. When your SSHD service has a flaw, your VPN prevents anyone from seeing it. When both your VPN and your SSHD have a flaw at the same time, your firewall rules prevent most of the world's IP addresses from even attempting it, leaving only locals and large botnets to be able to try and exploit the vulnerabilities, which you patch as soon as possible.
Thus, I say again, put in as many mitigations at as many levels as possible. You cannot know in advance which ones have undiscovered vulnerabilities, and which order those will be exploited in.
Make certain you only allow SSH v2
sshd_config
Stay patched
Use your iptables to allow ONLY the IP addresses or ranges you actually use
Set up your SSH to allow access ONLY by certificate, not by username/password, per the Ubuntu help site SSH/OpenSSH/Configuring page
sshd_config
If you use an encrypted home directory
- AuthorizedKeysFile /etc/ssh/%u/authorized_keys
If not, the normal setting for home directories can work
And set up as strong a certificate as you can. Generating keys is covered at the Ubuntu help site SSH/OpenSSH/Keys page
I'd start with a 4096 bit RSA key or an ed25519 key with a long, strong passphrase. Be sure to generate it locally, type in a long, strong passphrase when prompted, use -o to ensure the new OpenSSH 6.5 key format, and make sure the server never has anything but the public key.
ssh-keygen -o -t ed25519 -f ~/.ssh/id_ed25519
ssh-keygen -o -b 4096 -t rsa -f ~/.ssh/id_rsa
To check length on an existing key, use ssh-keygen -e -f mykeyfile
If you can, restrict keys in the authorized_keys file to a specific IP address or set of hostnames
from="192.168.1.2" ssh-rsa ABCDE...012 test@test.test
from="*.test.test,!BadPC.test.test" ssh-ed25519
ABCDE...012 test@test.test
If you're on OpenSSH 6.8 or higher, you can restrict authorized keys to certain types as well in sshd_config
- PubkeyAcceptedKeyTypes ssh-ed25519*,ssh-rsa*
Use fail2ban to make brute force attempts take more effort (i.e. make them use a botnet instead of only one machine, or make them take a lot of time)
Setting up certificates (keys) is superior if you're asking which one to do first
Reduce your login grace time
If you're using certificates, you have the option to crank it WAY down unless you use very, very slow connections
Otherwise, well, how fast do you type?
Control your cipher suite choices with information from Mozilla.org Security/Guidelines/OpenSSH
Since it's your server, and only you are getting in, you can select a very, very tight grouping such that only the most modern clients are allowed. Perhaps
HostKey /etc/ssh/ssh_host_rsa_key
- sudo ssh-keygen -o -N '' -b 4096 -t rsa -f /etc/ssh/ssh_host_rsa_key
or
HostKey /etc/ssh/ssh_host_ed25519_key
or both (most preferred first)
remove all dsa keys entirely; they're fixed at a pathetic 1024 bits for SSH
Use ssh -Q directives to determine what your system supports for the cipher suite options
KexAlgorithms curve25519-sha256@libssh.org
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com
Or whatever specific choices make you feel safest
- And keep up with the literature; when there's a flaw in something you've chosen, choose something that doesn't have known flaws at that point in time.
Use your iptables (or firewall) with blocklists from I-blocklist
Use your iptables (or firewall) with geographical or ISP based IP lists; one such source of geographical lists is maxmind.com
Switch to a port in the dynamic (ephemeral) port range of 49152 to 65535 per RFC6335
And only bind to the IP addresses required
Block everything other port on that server via iptables; chances are you're getting hit by a LOT of attacks
Especially harden your MySQL database, if any - I see a LOT of MySQL (and SQL Server) attacks on my IDS, and I've never run either one or had either port open. Set them to local, non-routable IP's only, etc., long passwords, disable them, etc.
Especially disable any web servers, set them to local, non-routable IP's only, etc.
Use port knocking per the Ubuntu help site or a DigitalOcean article
Rate limit new connection attempts, per this answer to ServerFault question "Hundreds of failed ssh logins"
iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 5 --name SSH --rsource -j DROP
iptables -A INPUT -p tcp --dport 22 -m recent --set --name SSH --rsource -j ACCEPT
Consider using chroot per the Debian documentation
"OpenSSH SFTP chroot() with ChrootDirectory"
Ensure rhosts is disabled with the sshd_config option
IgnoreRhosts yes
RhostsRSAAuthentication no
RhostsAuthentication no
Add more restrictions to the pre-authentication unprivileged process
UsePrivilegeSeparation sandbox
If you aren't using fowarding or tunnelling, disable it in sshd_config
X11Forwarding no
AllowTcpForwarding no
GatewayPorts no
PermitTunnel no
Prevent other insecure setup in directories or permissions
StrictModes yes
And then go through your permissions and settings until it works again :)
Disable host based authentication
HostbasedAuthentication no
Note that per This Serverfault answer to "OpenSSH daemon ignores ServerKeyBits directive", the old ServerKeyBits sshd_config directive is no longer applicable, since you don't allow SSH v1 in the first place.
Don't allow root in sshd_config
PermitRootLogin no
Don't allow different users - perhaps service users on some package you install - to have other options
PermitUserEnvironment no
Install a separate, all-up firewall like pfSense or a Ubiquiti Edgerouter Lite between that machine and the outside world.
And then use the built-in VPN IN ADDITION TO your hardened SSH login
And use the blocklists mentioned above at this level also
As DeerHunter mentioned, changing ssh and/or iptables and/or other firewall settings without any other way in can result in a lack of ability to SSH in.
