3

Our mongodb process is consistently using >100% of our CPU (this is on an Ubuntu 64-bit server on Linode) and we're casting about for performance improvements.

One suggestion we found was that MongoDB and NUMA don't work well together: http://docs.mongodb.org/manual/administration/production-notes/#production-numa

I notice that /proc/sys/vm/zone_reclaim_mode is already set to 0 and numactl is not currently installed.

My question: How do I find out whether our server is already NUMA-enabled and whether we should follow the advice at the above link to start Mongo with the following command?

numactl --interleave=all /usr/bin/local/mongod

Here's the contents of /proc/cpuinfo (with the first 3 sections suppressed since the output is identical except for the "processor" lines):

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2650L 0 @ 1.80GHz
stepping        : 7
microcode       : 0x70a
cpu MHz         : 1800.077
cache size      : 20480 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes hypervisor lahf_lm ida arat epb pln pts dtherm
bogomips        : 3600.15
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:
David Edwards
  • 148
  • 5
dreeves
  • 238
  • 1
  • 3
  • 9

1 Answers1

6
dmesg | grep -i numa

This will tell you if your server is numa enabled.

numactl -s will tell you if numa is enabled on a per process basis.

If production nodes shouldn't use numa, then you probably shouldn't. I would suggest rather than relying on turning it off via numactl, you can and probably should usually turn numa off completely via BIOS.

I've seen numa completely crap up server performance before with other Java applications when I was doing some performance tests. To see it screw up a C/C++ app like Mongo is surprising!

I'd like to disclose that your original issue, is different from your question. Changing the numa settings might not solve your 100% cpu issue. You might find after eliminating the numa card, you still have 100% cpu usage, which will require more diagnostics to get to the root of the problem (separate thread maybe).

Zippy Zeppoli
  • 200
  • 1
  • 8
  • Says NUMA is turned off. Thanks so much, Zippy! Starting another thread is probably a good idea, though I'm at a loss at the moment for what to even ask next. :) Suggestions welcome! Thanks again for the great answer. – dreeves Jan 22 '13 at 00:39