A Virtual Private Server is a server hosted using a dedicated allocation of hardware (including RAM, disk space, etc), but with other servers/users being hosted on the same box. Unlike cloud hositng, a VPS won't share resources with other users hosted on the same physical server, usually by zeroing out shared resources when switching users, making a VPS more secure & reliable than shared cloud hosting at a fraction of the cost of a dedicated server. You can host a VPN server on a VPS, but it's not the same thing. Though a VPS does share resources like RAM and CPU, there's much nmore separation than there is in a shared cloud environment, which is controlled by the hypervisor.
There are many advangtages to using a VPS. By chosing the right data centre location, you can imporve page serve times, take advantage of privacy laws in a particular country, and improve latency for time sensitive applications like FOREX trading.