From what I understand with OpenVZ you just increase privvmpages and vmguarpages for the swap memory, and even though the guest shows it as RAM the host will swap it were possible.
Am I right here or does it work differently
Update: Even setting the RAM to 3gb (swap+memory) for the container it still messes up because some programs allocate a lot of memory they don't use. I'm switching back to a full virtualization solution but a good answer to this would be nice.