Yes, malicious software running as your user can do anything that you can do as a user. There is no privilege separation between different applications running as the same user. This has also been noticed by xkcd.
To protect your SSH key specifically, you can add a passphrase, as mentioned by galoget, but that will only help marginally -- a malicious program could also scrape memory when the key is in use or log your keystrokes for your passphrase. (Though there are some mitigations for those attacks.)
A better option is to use a hardware token of some sort -- I use a Yubikey 4 with an OpenPGP key for SSH authorization. Though my key might be abused when unlocked, the key material itself cannot be stolen, and it is only unlocked for a short period of time. (You could set this timeout to near 0 and require user interaction for each use of the key.)
Another option is to store your key under another user account and use su
to switch to that account before SSHing.
In any case, if you have malicious software running as your user on your computer, you're going to have a bad time.