Clock frequency
Each clock cycle, the CPU tries to do work. The faster the clock, the more work it can try to do. However, the CPU is often waiting on stuff to work on to come in from RAM or I/O devices, so this is often the limiting factor in performance more than clock speed. If what you are doing involves heavy math computation or lots of encryption, then a faster clock frequency benefits.
If the type of work you are doing is not math heavy, a slower clock will save power. So many CPUs these days will allow the OS to manage the clock speed in some way for this reason.
Cache
Anything a CPU can work on has to live in RAM. External RAM in DIMMs are slower than the CPU - meaning things that occupy external RAM will make the CPU wait, but cache RAM is at or near the same speed of the CPU. So the CPU's memory controller will try to keep frequently accessed things in cache.
Cache RAM will greatly affect performance as anything a CPU works with has to be in RAM, at least temporarily. You want as much of it as possible for most applications and operating systems.
Cores
If you have 2 cores, then on each clock cycle. basically the equivalent of 2 CPUs can try to do independent work at the same moment. They are still limited by RAM and I/O. Programs and operating systems must be written to distribute work among multiple cores (all modern ones are now).
Many programs will use threads to work on things - threads being a section of a program that can complete or work independently from the main program. On a multi-core system, more than one thread can execute at once, meaning the work completes faster. Programs that don't divide work into threads (older programs, some games) won't benefit, but if you run two such programs, then you'll still benefit since each such program can be on it's own CPU core.
Given that a modern OS does many things in the background and competes for CPU resources against programs to some extent, it's almost always a benefit to have multiple cores, and the more the better. It may not matter too much to have more than 2 if you mostly run old programs or a single non-threaded program all the time.