The best solution for this problem would be renting a cloud server and host your applications on their site as you don't need to expose your home network and are personally secure if the server got hacked in some way.
If you still would like to host it local I would suggest an VPN tunnel if you need your PC connected to your whole network for many reasons. If you put your computer in a DMZ or make a own network for that and cut it off the other devices I would suggest a SSH server. Reasons for that are:
- Both, VPN and SSH, are very robust and secure because the add some strong encryption. Anyway, it's easier for you to connect with OpenVPN or other software and you can use more features of your home network as you tunnel the whole traffic. These are coming in the future as you are surely expanding your solutions so better start with a full setup than always a short term solution.
- As you are thinking of SSH or VPN you got the security problem with both solutions. If one get compromised they get your network (VPN connects you to a network and SSH to a particular PC but if this PC is in your local network it doesn't matter which one get compromised). So better start with a protocol that fits all your needs and more.
- One big drawback will be that it's harder to setup and maintain a VPN Server than a SSH Server.
- VPN offeres more privacy as it adds another layer and works as a direct connection to your own network.
So if you are thinking of these reasons you can select your needed protocol. If you just want to put one PC on the internet, which is cut off the network (Either DMZ or physically), SSH will be enough as it's easier to maintain and install. If this PC needs to be connected to your network you can use a VPN as it doesn't matter in security perspective and you can use the privacy point.
I know I missed RDP here but as you stated you don't like it I took it out off my considerations.