A multiple core cpu is like having multiple cpu's on a single motherboard.
What multi-core cpu's give you depend on what you are running... it's a matter of will you be running any software on the server that takes advantage of the multiple cores?
Where multi-core cpu's are immensely helpful is where you are running an application that is written to take advantage of them; that is, specifically, multi-threaded applications.
Many (perhaps most) legacy apps were written for a single cpu model, so for these it makes little or no difference.
But an app written as a multi-threaded app will be able to take full advantage of those extra cores. As an example, we run an app that used to be single-threaded. It was modified by our programmers to be multi-threaded, and it now runs almost 20x faster on our dual xeon, 24 core dell (each xeon has six physical core, and each is "hyper-threaded", making for 12 logical cores.
Another potential area where multi-core cpu's could be immensely advantageous is where you are running virtual machines. I know that Vmware lets you assign one or more cores to each virtual machine; in these cases, the performance can increase linearly, depending on the application. If the app is i/o bound, multiple cores will still help, but not to the same extent as when it is cpu bound.