On large systems, 8GB of physical RAM, we usually allocate 2GB of swap. These are loaded database servers running Oracle or PostgreSQL. For years, I've never seen swap hit even under heavy load. Heavy load is roughly 100 to 150 users doing around 10,000 SQL reads, and maybe 2,500 writes per minute.
We also adjust the swapiness level up to discourage swapping and we turn off unneeded processes we know we don't need running. (Custom Linux builds)
Like Eddie, says above, 1x physical RAM is a good rule of thumb for anything under 4GB. I do custom Linux builds for a large, Fortune-500 company and these are the things we typically do, and we haven't had any issues in over the last 5 or so years I've been consulting for them.
On larger systems: 64bit Linux with 32GB and 64GB of RAM, these are our Oracle database servers, and we typically keep 2GB of swap around for things like Eddie points out above - idle processes will be moved to swap depending on your swapiness settings.