Fresh MongoDB Using 100MB of Virtual Memory?

Question

I'm having a problem with virtual memory usage using MongoDB on a server. NOTE: I have only a local user account without root privs.

The server's VM limit (from ulimit -v) is 120MB.

With an empty database, MongoDB is reporting 100MB of virtual memory used.

> db.serverStatus().mem 
{
    "bits" : 32,
    "resident" : 23,
    "virtual" : 100,
    "supported" : true,
    "mapped" : 0
}

As soon as I try to create my first data entry (through db.addUser in the admin database), it's failing with an out of memory error.

done allocating datafile DBPATH/admin.0, size: 16MB,  took 0.244 secs

ERROR:   mmap() failed for DBPATH/ admin.0 len:16777216 errno:12 Cannot allocate memory

I understand that with a small VM limit and 32-bit mongodb, I'm not going to be able to create gigantic databases, but that's OK. I don't need gigantic databases.

What I can't understand is why it is using so much virtual memory out of the gate, and why it can't create a single 16MB database table.

I have read the accepted answer on this other question. It clarified how mongodb uses virtual memory, but it didn't help me to understand what's going wrong here.

score 4 · Accepted Answer · answered Mar 07 '13 at 18:21

4

You need to provide adequate resources for your MongoDB server -- that means LOTS of RAM.
You are limiting it to virtually none, so you should not expect it to work unless you set more reasonable memory use restrictions.

You seem to be laboring under the mistaken assumption that 100MB is a large amount of memory.
It's not.

A simple shell (let's take plain old vanilla Bourne sh as an example) is nearly 10MB Virtual Size.
A more complicated shell (say the Bourne-Again Shell bash) is just under 25MB.
An Apache web server worker can have a VM size of 150MB or more.

A database engine, even a NoSQL one, that is only 100MB virtual size and communicates over the network is frankly a programming miracle. If you expect to be able to do anything with that miracle you're going to have to give it enough RAM to actually hold and work with data.

answered Mar 07 '13 at 18:21

voretaq7

79,345
17
128
213

I appreciate the info but could have done without the snark. I am aware that 100MB is not a lot by modern standards, but MongoDB promises that the [virtual memory usage](http://docs.mongodb.org/manual/reference/server-status/) will be: "slightly larger than mapped" in typical installations, and recommends sizing your [RAM requirements](http://docs.mongodb.org/manual/faq/diagnostics/#how-do-i-calculate-how-much-ram-i-need-for-my-application) based on your working data set. Asking why the VM use was nearly ten times the data set size did not seem to be an unreasonable question. – Lynn Mar 07 '13 at 19:19
The key words there are "in typical installations", which you're quite clearly not one of. – ceejayoz Mar 07 '13 at 19:20
1

No snark intended, @Lynn. I've no way of gauging your experience level, but the fact that you seem surprised at a 100MB virtual size for an unloaded DB server lead me to think the illustrative examples might be useful - if not to you, then perhaps to the next person who comes along wondering the same thing. (It would also probably be more reasonable for Mongo to say the *resident* size will be slightly larger than mapped - in your case the VM size is bounded more by shared libraries than your data set.) – voretaq7 Mar 07 '13 at 19:38
@voretaq7 - Apologies if I misread your answer. Thank you for responding. – Lynn Mar 07 '13 at 19:41

score 1 · Answer 2 · answered Mar 08 '13 at 11:31

The virtual memory usage will represent the entire size of the data files (including empty space), not just the data in the database. When you call mmap() on a file, it just maps that all into virtual memory - it does not care what's in the file.

If you have a low virtual memory limit, then that host is simply not going to be suitable for MongoDB beyond a very small data set. You will be helped by running with the smallfiles option, but only a little.

Working set and resident memory are a completely different thing to the virtual memory requirements for MongoDB. For virtual memory, you will need to have at least the total size of your data files available, and if you run with journaling (you should, but it's generally not possible with 32-bit) that requirement doubles. As a general statement you should have a host with unlimited virtual memory to run MongoDB, if there are limitations imposed it's probably going to cause you a lot of problems.

It's not part of what you are asking, but I would also generally not recommend using the 32-bit version of MongoDB for anything besides some basic testing.

Fresh MongoDB Using 100MB of Virtual Memory?

2 Answers2