How do I find what's eating up all of my system's memory? (not SuperFetch)

How does one diagnose/discover memory-related problems, if Task Manager appears to not be reporting correctly?

I must have a memory leak or something -- I'm at 75% usage of my 12 GB of memory, but Task Managers listings of processes' memory usage isn't adding up (yes, "Show processes from all users" is checked).

It seems like this "phantom" memory usage grows in relation to how long the system's been up. It shows that there are 118 total processes. Besides the top 5, all of them are under 50 MB.

The top process is firefox, using 2.6 GB. Adobe Premiere at 900 MB. Plugin-container at 300 MB. Pale Moon at 275 MB. Explorer.exe at 94 MB.

How in the world can I find what's using up the rest of my memory? It seems as if Task Manager isn't seeing everything that's being used. Perhaps there's some kind of memory leak? Or program's aren't releasing used memory properly?

Resource Monitor reports (click for larger version):

NOTE:

As my main OS drive is an SSD, SuperFetch is disabled on my system.
And I understand the principal that unused ram is "wasted" ram, so to speak. However, when I see my memory usage nearly used to peak capacity, it's stopping me from running/loading subsequent programs (from my own concern), and even sends me to task to close many down (with hardly much effect on decreasing memory usage, at least noticeably and within short time period).
When the memory fills up, I hate seeing the 1GB page file on my SSD expand to the 6GB limit I've set, and me "sweating" with the knowledge that my "poor" SSD is likely being ground down to dust as Windows heavily/rampantly turns to using it as a full-fledged memory stick. (On that note, should I just hard-cap the page file to a static 1GB?)

Coldblackice

Posted 2012-06-17T11:40:44.527

Reputation: 4 774

4just a sidenote: used RAM is a not a bad thing per se. clever caching of things often speed things up. if you do not have a problem like "omg, programm xyz starts up so slow i can barely take it because of it has to throw out other things out of the ram" .. then it might be not a problem at all. – akira – 2012-06-17T12:20:54.320

possible duplicate of Discrepancy between memory in use reported by the task manager and memory used by programs also perhaps check out Tool for viewing used and free memory on windows system

– Ƭᴇcʜιᴇ007 – 2012-06-17T14:13:36.487

Thanks, but since SuperFetch is disabled on my system (with the main SSD OS drive), the question is different. Updating original question. – Coldblackice – 2012-06-17T15:49:34.697

@Coldblackice Just because the one (unaccepted) answer doesn't help you doesn't mean it's not the same question. The OP of the other question even said changing SuperFetch didn't help. :) – Ƭᴇcʜιᴇ007 – 2012-06-17T17:12:13.737

True, but it isn't the correct answer in this case, giving need for another question. Question updated to reflect this specific. – Coldblackice – 2012-11-02T17:56:39.623

Answers

You can usually gain better insight into what is using up your memory with tools like RamMap.

^{click to enlarge}

Often, the memory can be used up by memory mapped files that have been pulled into RAM to increase performance.

Take note of the Active column in RamMap. That is the memory that is actually currently being utilized. Indicators regarding free memory can often be misleading. This includes Windows tools like Task Manager.

Further information

I also wanted to address some additional points from your question.

However, when I see my memory usage nearly used to peak capacity, it's stopping me from running/loading subsequent programs (from my own concern), and even sends me to task to close many down (with hardly much effect on decreasing memory usage, at least noticeably and within short time period).

Your reasoning is perfectly fine. But, unless you're experiencing issues due to low memory situations, you should refrain from this practice. The operating system should handle this task and flush data out to the paging file when physical memory is required.

Possibly, you're also using the wrong indicator for your practice. You want to have an eye on your Commit Charge (Process Explorer terminology).

enter image description here

The commit charge is the sum of all used memory. You can see in the screenshot, there is also a limit to it. That limit is the size of my physical memory + the size of my paging file. If that limit was to be hit, that would be an issue.

Which brings us to our next topic.

When the memory fills up, I hate seeing the 1GB page file on my SSD expand to the 6GB limit I've set, and me "sweating" with the knowledge that my "poor" SSD is likely being ground down to dust as Windows heavily/rampantly turns to using it as a full-fledged memory stick. (On that note, should I just hard-cap the page file to a statci 1GB?)

In my opinion, there are several things wrong here. In general, it should be strongly discouraged to set limits on the size of the paging file. Mark Russinovich explains this in the videos I linked above (much better than I ever could).

Only bad things can come from that. What is Windows supposed to do when it needs to flush out more than 6GB of data to the paging file but you capped it at 6GB? That is a major issue.

Let Windows handle the paging file. The only thing I would strongly advise you to do, is moving the paging file away from your SSD. Tom Wijsman explains this excellently in his blog post Maximizing the lifetime of your SSD, which I recommend you to read.

Der Hochstapler

Posted 2012-06-17T11:40:44.527

Reputation: 77 228

1@Oliver Salzburg, one thing to keep in mind is that Windows 7 was designed for spinning rust, not SSDs, and 1-4GB of RAM, not 12. It's not gospel, it should be obvious that there are times when one-size-fits-all defaults don't apply, and the solution isn't to keep advocating to don't think about it and never touch those defaults no matter what when someone wants to more deeply understand an issue. That's fine for noobs, but not for everyone. – SilverbackNet – 2015-01-29T06:52:32.390

Re pagefile on an SSD, Microsoft has this to say (https://blogs.msdn.microsoft.com/e7/2009/05/05/support-and-qa-for-solid-state-drives/): "Should the pagefile be placed on SSDs? - Yes. Most pagefile operations are small random reads or larger sequential writes, both of which are types of operations that SSDs handle well. [...] In fact, given typical pagefile reference patterns and the favorable performance characteristics SSDs have on those patterns, there are few files better than the pagefile to place on an SSD."

– Jamie Hanrahan – 2018-09-29T03:49:07.133

Re size of pagefile - Resource Monitor shows, at the moment captured, 2.5 GB on the modified page list. It should be well under 1 GB (on my machine atm it is about 500 MB). The pagefile is supposed to exist so that Windows can flush the modified page list to it! Windows has already decided that that stuff should be evicted from RAM, but you have constrained your pagefile so that Windows can't do that. – Jamie Hanrahan – 2018-09-29T03:57:44.927

Note that if your server is out of usable memory, RamMap will crash. Really turns into a chicken-and-the-egg problem to try to get a handle on the case. – Demonslay335 – 2019-11-11T17:25:35.837

1Ahhh, I see. Yes, the "phantom" memory usage appears to be coming from the Mapped File category (and specifically, the Standby column of this). So Task Manager doesn't report these files being mapped, just that the memory is being used? – Coldblackice – 2012-06-17T15:58:32.670

Additionally, with so much memory being used with Mapped Files -- does the fact that my memory seems to be quickly and frequently brushing up against the 12GB limit mean that I should have more memory? Or will Windows just expand to fill that up with as many files as it can map, as well (and understandably for efficiency; why not use ram)? – Coldblackice – 2012-06-17T16:01:03.587

@Coldblackice: I strongly advise you to watch the videos I linked in my answer. In your question you mention several points that concern me. I will address those in my answer shortly. – Der Hochstapler – 2012-06-17T16:04:26.533

@Coldblackice: I added a few more bits to my answer. Hopefully you'll consider it useful :) – Der Hochstapler – 2012-06-17T16:23:36.540

2Thanks for the help and addendum. As to your note about the page file and SSD's -- I've heard the exact opposite (and hence why I have it the way it is). Not saying that I have the definitive "right" by any means, but I've read that page file is -best- on an SSD because of the SSD's speed benefits. And as to degrading the life of the SSD, I heard that that's negligible, as long as the pagefile is limited. – Coldblackice – 2012-06-18T12:02:44.207

1As to letting Windows manage it -- Windows wants to allow my pagefile to expand up to 18GB! Unfortunately, I can't afford that amount of space on my SSD for paging. I had presumed that 12GB of memory would be more than ample for Windows to work with, which memory, oddly enough, Windows seems to quickly slurp up as much of it available as it can, seemingly caching anything it can get its hands on :) Thus, I fear that if I let it have however big of a pagefile it wants, it will find enough files to map into memory to quickly bridge to filling up the page file in its entirety. – Coldblackice – 2012-06-18T12:11:28.843

1@Coldblackice: I'm sorry I couldn't convince you. Please only take 1 thing away from this, there are very few people who have a proper understanding of Windows memory management. Yet, a lot of people are quick to offer recommendations (obviously, me included). Ask yourself, if limiting your paging file size is useful, why is it not the default? Also, a memory-mapped file is very unlikely to be flushed out to the paging file, because it already exists in a file (the one that was mapped to memory). So your fear in this regard is, most likely, misplaced. – Der Hochstapler – 2012-06-18T12:18:20.147

Appreciate your help in guiding and pointing this out to me! – Coldblackice – 2012-06-20T17:46:16.403

In my case the reason for high memory usage was a hyper-v virtual machine for which I had Dynamic Memory settings enabled for RAM.

RamMap tool helped me to locate the root cause.

First, I noticed in RamMap that Driver Locked consumed more then 29GB out of my 32GB RAM. Then, I found some article that explained that Driver Locked high memory usage can be caused by the HyperV Dynamic Memory enabled, and then I just confirmed that.

Dragan Nikolic

Posted 2012-06-17T11:40:44.527

Reputation: 131

If you see the snap is not showing all the process and the shown process itself taking 5.8 GB Memory. from the shown snap it show that there are a lot unnecessary background program are running. Using msconfig remove the unnecessary startup programs and also increase your virtual memory (1.5 times of your physical memory).

tiki

Posted 2012-06-17T11:40:44.527

Reputation: 66

How do you increase your virtual memory? And what's the reasoning behind the factor 1.5? – Der Hochstapler – 2012-06-17T14:46:36.083

While increasing virtual memory (by increasing the size of the pagefile) is valuable, it may only lead to a lot of paging, which will slow the machine waaaay down. If all unnecessary programs have been prevented from running, then the best thing to do is install more RAM. – Fran – 2012-06-17T16:15:15.660

To increase your virtual memory follow the path : Go to MY COMPUTER properties->select "Advance system settings"->go to 'advance' tab->select the "settings" under PERFORMANCE->go to advance tab->under Virtual Memory Select Change-> Select Custom size to give min. and max. size......It is recommended to increase your Virtual memory upto 1.5x of your physical memory(ex:if you have 4GB RAM then you can assign upto 6GB virtual memory). But FRANs suggestion is also correct. Please try to minimize your unnecessary background application. – tiki – 2012-06-17T18:46:28.887

So is it proven that it's overall better to increase or decrease the page file, in terms of raw, overall performance? Is the 1.5x really necessary? – Coldblackice – 2012-06-20T17:48:00.723

You can try Sysinternal's RAMMAP but it shows memory usage in a different way than Task Manager & Resource Manager and is therefore not too helpful.

Task Manager & Resource Monitor don't show the filecache and the size of it. It's part of the "In use" part of the memory (See Resource Monitor). The file cache can grow to a VERY large size, especially when one keeps the computer running for hours. Overall program memory usage goes up as time goes by, as a result of a growing file cache & memory leaks. Some programs simply have a lousy memory management.

One way to minimize memory usage is reducing the amount of processes. (Why are 118 processes running ? Why do you have multiple instances of Explorer.exe running ?)

There's a program that helps you to regularly reduce memory usage & regularly collapse the file cache. (PcWinTech's CLEANMEM). Reducing memory usage this way also reduces or even eliminates the need for Windows to write data form the memory to the swapfile (on your SSD). So, it increases the lifespan of your SSD. Reducing memory usage also makes more "Standby" memory available, allowing Windows to preload more files in the Standby memory, speeding up your system.

Willy2

Posted 2012-06-17T11:40:44.527

Reputation: 17

1Are you affiliated with this software? The last paragraph sounds like marketing speak to me, and I see you're recommending the same thing on 2 old posts. – Arjan – 2014-05-04T17:36:34.407

1Yes, I have helped the author of the program to improve the program. But I don't have any (financial) reasons to promote the program because the program is free. – Willy2 – 2014-05-04T17:47:24.173

RAMMap showed that C:\Windows\SoftwareDistribution\DataStore\DataStore.edb mapped file was wasting 400 Mb of my precious 4 Gb RAM.

Moreover Windows Update was constantly slowing down the system in process of updating that huge file.

That may be a common problem for many lower-end PCs.

See PC Boots then writes giant datastore.edb file slowing the computer down for possible resolution.

Vadzim

Posted 2012-06-17T11:40:44.527

Reputation: 974