Linux - Use disk as RAM

0

I have a VM and in it a process that consumes a lot of memory (~200GB). Some sort of in-memory DB. I need to run it on a standard laptop and I cannot recompile it or see the code.

I've added 256GB of swap space with pri=32767 in /etc/fstab and vm.swappiness=100 in /etc/sysctl.conf but it just won't load the DB quick enough (stuck on 4GB after 24h; doesn't seem to grow anymore).

(EDIT: I cloned the machine, changed the RAM from 256GB to 4GB, added hard disk, formated it as ext4 and created a swap file of size 256GB (dd -> mkswap -> swapon...))

Why did it stop growing?

I suspect the slowness is caused by it being a swap space, therefore the OS is busy "swapping" (load -> not enough room -> deciding what to swap out...).

I'm looking for a way to "add more memory" but make the OS treat it as normal memory. Or maybe my swap configuration is wrong?

I know it'll hurt performance, but it is acceptable for me.

The VM is CentOS 6.

assafmo

Posted 2017-08-28T10:18:36.477

Reputation: 111

3“I need to run it on a standard laptop” Well, you won’t. Even if you could get it to start, an in-memory database isn’t built to cope with the slowness of disk access. You might as well perform whatever the program does by hand with pen and paper and it’d probably be faster. – Daniel B – 2017-08-28T10:20:53.750

2If there were no difference between swap and real RAM all our systems would have a very small RAM. If you need the speed of real RAM, you need real RAM. – xenoid – 2017-08-28T12:12:13.547

Answers

1

Have you tried multiple swap files? 4 gigs sounds like you're hitting the file size limit for a 32 bit system.

I found this tutorial at centos.org which pretty cleanly laid out how to make swap files manually.

To add a swap file:

  1. Determine the size of the new swap file in megabytes and multiply by 1024 to determine the number of blocks. For example, the block size of a 64 MB swap file is 65536.
  2. At a shell prompt as root, type the following command with count being equal to the desired block size:

dd if=/dev/zero of=/swapfile bs=1024 count=65536

  1. Setup the swap file with the command:

mkswap /swapfile

  1. To enable the swap file immediately but not automatically at boot time:

swapon /swapfile

  1. To enable it at boot time, edit /etc/fstab to include the following entry:

/swapfile swap swap defaults 0 0

The next time the system boots, it enables the new swap file.

After adding the new swap file and enabling it, verify it is enabled by viewing the output of the command cat /proc/swaps or free.

codykochmann

Posted 2017-08-28T10:18:36.477

Reputation: 200

You might want to add details on how this is done for everyone that doesn't know how to do it. – Ramhound – 2017-08-28T13:38:51.330

this was a clean resource on how to basically do it. https://www.centos.org/docs/5/html/5.2/Deployment_Guide/s2-swap-creating-file.html

– codykochmann – 2017-08-28T16:46:07.740

I like this tutorial a little more though because they include how to manage your permissions https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-centos-6

– codykochmann – 2017-08-28T16:46:48.137

@Ramhound It is a 64bit system. Thanks. – assafmo – 2017-08-29T07:08:30.960

@codykochmann The material should be in the answer body. A tutorial on another website doesn't do us any good – Ramhound – 2017-08-29T11:12:30.090

1

@assafmo Even if the processor is 64bit, that doesn't mean the filesystem is. For example, FAT32 is limited to 4Gb files. FAT16 and FAT12 are even worse.

– jpaugh – 2017-08-31T18:31:56.937

I forgot about that. :) – codykochmann – 2017-09-01T00:06:21.567

Who runs CentOS on a FAT partition? – Ken Sharp – 2019-02-12T13:33:04.107