Garbage collection usually indeed refers to the heap memory.
Note that memory leaks and out of memory errors are not synonymous.
A memory leak is a situation where some chunk of allocated memory (e.g., an object in a OO language) is no longer needed, but for some reason cannot be deallocated/collected (e.g., the programmer forgot to clear some reference to it).
You can have a memory leak that does not cause an out of memory error (if the leak is sufficiently small), and you can get such an error without having a memory leak (if you legitimately try to allocate more memory than's available to the process).