My rule of thumb is "anything over zero is bad". Question becomes "how bad?". For webservers and database servers its very bad as the trip to disk is likely enough to have a very large performance impact on whatever transactions spilled over. In fact it will often cause a snowball effect until the kernel oom-kill's something.
For a fileserver or something that does asynchronous/batch workloads, as long as its not happening constantly you may be ok with it.
Remember the first line of output from vmstat is an average. Its only if you pass it a number of seconds to update on that you start to see if swapping is occurring now.
If you have sysstat setup a "sar -B" will show you historical trends.