9

I've got a CentOS 5.10 (32-bit) server running on VMWare. It's allocated 4 GB of RAM.

If I run dmidecode -t 17 | grep Size | grep MB I see: 

Size: 4096 MB

Yet when I run free, I see: 

             total       used       free     shared    buffers     cached
Mem:       3107140    1239244    1867896          0        332     400464
-/+ buffers/cache:     838448    2268692
Swap:      2096472          0    2096472

Why is there a discrepancy between the total amount of memory free reports and the dmidecode output?

The kernel I'm running is: 

2.6.18-371.4.1.el5 #1 SMP Thu Jan 30 06:09:24 EST 2014 i686 i686 i386 GNU/Linux

Admittedly, the kernel is not running PAE but I thought that was only necessary for memory exceeding 4 GB.

I know I'm missing something simple - can someone please elaborate?

Additional Notes/Observations

It definitely looks like my kernel is reserving a bunch of memory for other stuff. Here's what I see in /var/log/dmesg:

Linux version 2.6.18-371.4.1.el5 (mockbuild@builder17.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)) #1 SMP Thu Jan 30 06:09:24 EST 2014
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 000000000009f800 (usable)
 BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
 BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bfef0000 (usable)
 BIOS-e820: 00000000bfef0000 - 00000000bfeff000 (ACPI data)
 BIOS-e820: 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
 BIOS-e820: 00000000bff00000 - 00000000c0000000 (usable)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
Warning only 4GB will be used.
Use a PAE enabled kernel.
3200MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000f6bf0
Memory for crash kernel (0x0 to 0x0) notwithin permissible range
Mike B
  • 11,570
  • 42
  • 106
  • 165

3 Answers3

18

With a 32-bit kernel, you only have 4GB of available address space. Some of this address space has to be used by the (virtual or physical) hardware in the system, such as video cards, NICs, etc., for their own purposes. This usage is usually between 256MB-1GB depending on how much address space the particular hardware needs.

Since that address space is used by hardware, the corresponding RAM is generally inaccessible to a 32-bit system.

You have a couple of options:

  1. The preferred option is to run a 64-bit operating system. This dramatically expands the address space, so there is plenty of room for all the RAM and hardware. It also breaks the 2GB/3GB 32-bit limit on applications while maintaining the ability to run 32-bit programs. In general, any system with 2GB of more of RAM should run a 64-bit OS to avoid these issues.
  2. Another option is to run a 32-bit kernel with PAE enabled. This will unhide the RAM, but each process will still be limited to 2GB/3GB of address space, depending on the particulars of the kernel build. Since 64-bit OSes will run 32-bit applications perfectly well, this has no advantage and many disadvantages (such as a lack of an upgrade path).
Michael Hampton
  • 237,123
  • 42
  • 477
  • 940
  • Thanks. That makes sense but how can I check specifically how much is "hidden"/consumed by hardware for other purposes? Would that be under `/proc/meminfo` ? – Mike B Oct 29 '14 at 17:50
  • @MikeB Specifically, I'm not sure offhand, though it's obvious that somewhere around 800 MB is lost. – Michael Hampton Oct 29 '14 at 17:51
  • For the purpose of my initial question, I think it's answered but the next question is "WHY?". It looks like there is another thread covering this (http://unix.stackexchange.com/questions/97261/how-much-ram-does-the-kernel-use) so I'll try digging some more and may have questions later. Thanks! – Mike B Oct 29 '14 at 18:12
  • As professional system administrators, we care about this, but only up to a point - where and how it affects operations. I think I've addressed that aspect. – Michael Hampton Oct 29 '14 at 18:16
  • Agreed. I'll post to the Unix SE side if needed. Thanks again for your help. – Mike B Oct 29 '14 at 18:19
  • 2
    @MikeB `/proc/iomem` will show you the memory used by devices where Linux has a driver for. The e820 memory map (at the very beginning of a `dmesg` of a freshly booted kernel) will show you what your BIOS/EFI thinks which regions are reserved. Matching them against each other is AFAIK a manual task with no tool support. – mihi Oct 29 '14 at 18:45
5

The output of the free command does not count reserved kernel memory and a few other small bits. You will see this discrepancy even in a 64-bit kernel and even with <2GB RAM.

John
  • 8,920
  • 1
  • 28
  • 34
3

The critical line from your physical RAM map is this one:

 BIOS-e820: 0000000100000000 - 0000000140000000 (usable)

This line shows that 1 GB (0x40000000 bytes, hexadecimal) of your system's physical RAM is being mapped by the BIOS above the 4GB limit, making it inaccessible by a 32-bit system without PAE.