Unfortunately, your question is application specific, and scenario/situation specific. The short answer is, simply, no (or rather it shouldn't).
Your swap partition should be seen as an extension of your RAM. To prevent your SWAP getting full, you might (prior to getting more RAM) try to make your SWAP partition larger. But do get more RAM as well, SWAP can be 1000X slower than RAM.
So, to answer your question fully, ask the following question: What happens to a PC (and it's running applications) when it runs out of RAM? Or RAM and SWAP?
It would refuse to start up new applications... It would refuse to supply new memory blocks to applications requesting memory (if these applications can recover from such a refusal, they will log an error and continue to run, most applications will crash, or exit gracefully as they cannot fulfil their purpose if they cannot allocate memory).
So if your key file was indeed locked (which I don't dispute could have happened), it was most likely due to something SSH/SSL related, like the daemon hanging because it couldn't allocate memory, or the daemon restarting for the above-mentioned reason.
If using openssh-server, change your /etc/ssh/sshd_conf to not allow password authentication, and try to recreate a scenario where you would have zero RAM and zero SWAP space left. I really suspect sshd will just crash, to be honest.
1Next time this issue presents, use the verbose switch with ssh (ssh -v user@server.whatever) from the command line. It'll give you some clues. You can also have a look at the logs for more information. – Alex Atkinson – 2015-05-04T15:17:21.320