Virtual memory and commit charge

1

1

I've read a lot about commit charge and something still is bugging me. On my mac (and probably on linux too), what is the relation between VM size and and Commit memory (windows). I think I fully understand commit after some testing, but in windows it doesn't seem that the amount of virtual memory ever exceeds the swap space + ram. IN OSX (and linux) the VM size seems to almost exceed the size of my harddrive! I'm guessing it either includes shared memory more than once, or it includes allocated but untouched or written to memory. Let's say I make a malloc(2gb) (obviously not like this), is different from actually writing to that entire 2gb.

So this brings me to my question, what exactly is VM size measuring in OSX and linux. Is it a total of all the malloc calls for example (or total possible VM for instance, including practically 4gb for each process on a 64 bit machine), and how does this compare to the Commit (limit,peak,use) on windows? Does windows not let you allocate more than your swap + ram limit, as OSX and linux do and why?

In the images below you can see that in windows swap + ram equals commit (2gb). In OSX my VM size is over 200gb on a about 200gb harddrive.

windows task manager showing 2gb commit max (I think)

windows control panel showing 1024mb 1(gb) swap plus 1(gb) ram

osx process manager showing over 200gb VM size

rubixibuc

Posted 2012-02-14T22:05:55.863

Reputation: 1 492

Answers

2

VM size is the total amount of address space used by the process. Neither operating system permits you to exceed available RAM+swap with backed VM, but you can with unbacked VM. Essentially, VM size (especially on 64-bit operating systems) is a meaningless measurement. Unbacked virtual memory (virtual memory that doesn't, and can't, require RAM or swap) is essentially free, so there's no reason to care how much of it is being used.

On a 64-bit operating system, you can open a 2GB file read-only and map the entire file into your process address space. That "consumes" 2GB of VM, but doesn't actually require any significant resources at all. And since it's a read-only mapping, it can't ever require swap.

Commit charge measures backed virtual memory, that is virtual memory that could ultimately result in the consumption of physical memory or paging/swap space. The operating system has no idea how much of this memory will ultimately require backing, so it generally won't allow more than RAM+swap to be allocated. (So far as I know, neither Windows nor OSX permit overcommitment of backed virtual memory. Linux does.)

David Schwartz

Posted 2012-02-14T22:05:55.863

Reputation: 58 310

@Stu No. Commit charge is more like the RAM plus swap that would be reserved if overcommit was disabled. Or, to put it another way, the maximum amount of RAM+swap that could be required without the OS having an opportunity to say "no, you can't have more". It's like the balance in the account, minus any checks written but not yet deposited. – David Schwartz – 2014-07-29T02:06:16.237

As for as VM size in OSX, is this just the amount of virtual memory reserved (such as reserved memory in windows) or is it the total of all the logical memory spaces allocate to all processes? Like processes * extent of vm mapped per process? I'm picking up on subtle difference between reserved (virtual) memory and page-file backed memory. What is the exact difference? – rubixibuc – 2012-02-15T03:53:42.137

There is a difference between virtual memory and page-file backed memory. The difference is that when these are present as physical memory, page-file backed memory may need to be written to the page file before the page can be discarded. For example, a read-only mapping of a file is virtual memory, but it can never need to be written to the page file (because it's already on disk as a regular file). On the other hand, regular malloced memory, if modified, will need to be written to the page file before the pages can be evicted from physical memory. – David Schwartz – 2012-02-15T06:30:45.623

Just so my little linux-understanding brain gets it, "commit charge" == "swap in use"? – Stu – 2013-05-16T17:48:04.710

Asked: 2012-02-14T22:05:55.863

Viewed: 2 232 times

Active: 2012-02-14T22:32:59.750