2

According to numactl, this dual CPU Opteron box is UMA rather than the expected NUMA:

$ numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
node 0 size: 65534 MB
node 0 free: 381 MB
node distances:
node   0 
  0:  10

I think it should be NUMA because there are four 4-core CPUs*. As I understand it, each CPU has its own memory channel; if a CPU needs to access memory in a non-local node, it must use hypertransport, which costs more time than accessing memory in the local node. AMD explains it here.

The motherboard has 16 RAM slots, 8 for each CPU. All 16 are populated with 4GB each for a total of 64GB. Some more particulars:

Why is numactl reporting that this box is UMA?

*There are two CPU dies per package, so the motherboard only has two CPU sockets.

Wayne Conrad
  • 635
  • 1
  • 7
  • 20

1 Answers1

1

The BIOS is hiding the NUMA reality behind the node interleaving setting. Setting that to Disabled will give you a true NUMA system as far as the OS is concerned. Not many systems really use NUMA effectively, which is why motherboard manufacturers default to make everything equally slow rather than, let the OS figure out what needs fast and slow access.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • 1
    I wouldn't suggest changing this setting. This box is so close to UMA that the overhead involved in managing the NUMA will exceed the benefits. And then there's the penalty you pay for not interleaving across all memory channels. Leave it the way it is, unless you're experimenting with NUMA specifically. – David Schwartz Jul 19 '12 at 01:14
  • @David, Thank you. After doing some reading, I think I understand the reason for your recommendation. We'll leave the box set for UMA. – Wayne Conrad Jul 20 '12 at 15:27