A good solution is to have the key on a dedicated hardware device that will do all your crypto operations without even revealing the key material to the host computer. You can use any PKI card supported by OpenSC, an OpenPGP smartcard (supported by both GnuPG and OpenSC) or a Yubikey (which in this case will behave just like an OpenPGP card).
For OpenSC-supported cards, install OpenSC and tell OpenSSH to use it:
ssh -I /usr/lib/opensc-pkcs11.so user@example.com
For GnuPG you can use the GPG agent as an SSH agent which will expose the card's keys through that. Note that unless you need passphrase/PIN caching I recommend using OpenSC over GPG.
On Windows, you should use either your card's minidriver (how Microsoft calls software like OpenSC) - most of them can be downloaded automatically provided you allow your system to search for drivers online. If no official minidriver is found you can use OpenSC's Windows build which includes a generic minidriver. Note that for OpenPGP cards there is a third-party minidriver that works better than OpenSC's one (in fact I had no luck getting OpenSC's minidriver to work with the OpenPGP card, even if the PKCS11 library worked fine proving that OpenSC was talking to the card).
Once the minidriver is installed, software will be able to talk to the card using the system's standard Crypto API (the card appears like any other certificate in the user's cert store). PuTTY-CAC is such a program that can take advantage of those certs (and cards), it includes PuTTY itself as well as Pageant, PuTTY's equivalent of an SSH agent. If under Cygwin/MSYS you can use ssh-pageant
to convert a running Pageant into an SSH agent that can be used by the standard ssh
(a direct bridge between CAPI and SSH would've been nicer, but no such thing exists yet).