-1

I have a VM and in it a process that consumes a lot of memory (~200GB). Some sort of in-memory DB. I need to run it on a standard laptop and I cannot recompile it or see the code.

I've added 256GB of swap space with pri=32767 in /etc/fstab and vm.swappiness=100 in /etc/sysctl.conf but it just won't load the DB quick enough (stuck on 4GB after 24h; doesn't seem to grow anymore).

(EDIT: I cloned the machine, changed the RAM from 256GB to 4GB, added hard disk, formated it as ext4 and created a swap file of size 256GB (dd -> mkswap -> swapon...))

Why did it stop growing?

I suspect the slowness is caused by it being a swap space, therefore the OS is busy "swapping" (load -> not enough room -> deciding what to swap out...).

I'm looking for a way to "add more memory" but make the OS treat it as normal memory. Or maybe my swap configuration is wrong?

I know it'll hurt performance, but it is acceptable for me.

The VM is CentOS 6.

assafmo
  • 139
  • 3
  • 1
    It seems to be not acceptable performance wise as you asked why it is blocked. When it looks frozen, do you see high paging (`vmstat 2` or `sar`) – eckes Aug 28 '17 at 09:26
  • @eckes Yes, I see high paging with `vmstat`. Is there a way to make it treat swap as normal memory? – assafmo Aug 28 '17 at 09:34
  • `iotop -ao` shows the process is 98% swapin – assafmo Aug 28 '17 at 10:04
  • 1
    It does treat it as normal memory, virtual memory to be precise. It is just slow. – eckes Aug 28 '17 at 10:22
  • 1
    You did exactly what you say you want to do -- it is treating the disk as normal memory and the performance is, not surprisingly, atrocious. – David Schwartz Aug 28 '17 at 10:48

1 Answers1

3

Looking at http://www.corsair.com/en-us/blog/2015/september/ddr3_vs_ddr4_generational, memory bandwidth is around 4 GB / s (the article is a bit old, but let's use this number for reference) for both writes and reads.

Then, for hard disks, the bandwidth is roughly 100 MB / s for sequential operations (http://www.buildcomputers.net/hard-disk-speed.html). For random access, the bandwidth is much smaller.

So, even if the software accessed memory in sequential manner, RAM would be 40 times faster. However, the memory access for the software is most likely very random, getting small blocks here and there, which makes total performance of RAM much higher.

Then there is the issue how swap actually works. Every operation where the processor accesses data, has to use the RAM for the data. If the data is not in memory, then some part of RAM is swapped out to HDD and the needed part is read from HDD to memory. When there is too much "hot" data in RAM, there will be constant swapping going on, which leads to a situation like you described.

It is not possible for processor to use HDD directly as RAM, because the addressing and interface is completely different.

Tero Kilkanen
  • 34,499
  • 3
  • 38
  • 58