Windows allows programs to request memory (Virtual Address Space) to be allocated to them even if they do not plan on doing anything with the memory. This allows the program to use the memory at a later point and they don't need to spend the extra time during a time critical portion of code to check it is available for use.
Windows does will not overcommit memory, so say you have 10MB of RAM and one program asks for 5MB (but only uses 2MB of it) and a 2nd program asks for 6MB. Windows will deny the request for the 2nd program because it can not guarantee if both programs wanted to use their full allotments that it the memory would be available to them.
If it did allow the requests to be made to allocate without enough space available it you would need to either crash the program that attempts to write to memory that it does not have space for or forcibly terminate another program to make space. (Linux does allow overcommiting of memory if you want to allow it via /proc/sys/vm/overcommit_memory
and follows the "forcibly terminate another program" approach when it encounters a memory write it can not fulfill)
When you have a pagefile and a program asks for 5MB (but only uses 2MB of it) it puts those 2MB memory that it is actually using in to ram and the other 3MB are allocated on the pagefile. Because it is just a "allocation" and nothing was written or read from it yet no IO performed and the speed of the drive does not matter. Now when that 2nd program comes along and asks for 6MB there still is plenty of space left available for it to fulfill the request.
Now on to your situation. First of all, when you go to check on how much ram is being used are you checking "Working Set" or "Commit Size". "Working set" that would be that "only uses 2MB" we saw in the previous example, "Commit Size" would be the "5MB" requested you did above.
When you have no pagefile your sum total of "Commit Size" must be smaller that the total ram in the system, for systems with pagefiles your sum total of Commit Size can be up to the total ram in your system + the pagefile size.
I do not know if when you went to check the ram available if task manager displays the total of "Working Set" or if it will change over to the total of "Commit Size" when you don't have a page file, you may be been much closer to your limit than you think.
A 32 bit program can allocate up to 2GB of virtual address space and a 64 bit program can allocate up to 7 TB for itself. So any programs you open can quickly take up very large chunks of address space, being that windows is designed and has been tuned over many years to work with a page file I think giving the user a warning when you have reached 66% of your total capacity is very reasonable.
I know many people assume that they will get better performance without a pagefile, and this was true back in the era of Windows 98, but a lot of things have changed and memory management is much better now. You state in your question that you disabled the pagefile due to high disk use, how did you measure the disk usage and how do you know it was going to the pagefile? If you are getting very large amounts of IO when you still have lots of free ram that may be a indicator of a different issue which we may be able to help solve.
1This question probably sets a new record for "most answers that don't actually answer the question". – Hugo Zink – 2016-08-29T19:43:21.870
Have you considered reducing the noise from the HDD? E.g. Hard Drive Silencing: Sandwiches & Suspensions.
– Andrew Morton – 2014-01-30T21:59:00.363