2

I'm attempting to run a Python script that requires a lot of Virtual Memory. I'm on Linux Mint 17.3 64 bit and Python 2.7.

As the Python process grows, I can see (using free -m) the amount of RAM use increase, but the amount of swap space used actually appears to be decreasing.

I tried changing vm.swappiness from 60 to 90, but that doesn't appear to have helped.

It's not a matter of the swap space being totally unused - some of it is in use. But it's not being used as much as it probably should be.

BTW, I'm in a VMWare instance.

How can I run this large process without a MemoryError, and without changing the code?

Thanks!

PS: Here's some free -m output:

$ while : ; do free -m; date; echo; sleep 10; done
cmd output started 2016 Fri Aug 19 08:47:39 AM PDT
             total       used       free     shared    buffers     cached
Mem:          7968       4764       3203         31        215        335
-/+ buffers/cache:       4214       3754
Swap:         8189       3452       4737
Fri Aug 19 08:47:39 PDT 2016

             total       used       free     shared    buffers     cached
Mem:          7968       4880       3087         31        215        343
-/+ buffers/cache:       4321       3646
Swap:         8189       3451       4738
Fri Aug 19 08:47:49 PDT 2016

             total       used       free     shared    buffers     cached
Mem:          7968       4982       2985         31        215        343
-/+ buffers/cache:       4423       3544
Swap:         8189       3449       4740
Fri Aug 19 08:47:59 PDT 2016
user1084684
  • 155
  • 1
  • 7

1 Answers1

0

It turned out that although the process started out growing slowly, it suddenly grew (or attempted to grow) a lot, and hit an out of memory condition.

I was able to verify this with strace (Debugging with syscall tracers | stromberg.dnsalias.org). I could see it mmap'ing smallish chunks over and over, and then right before the error it would mmap a large chunk.

So I added a lot of swap space, and the error went away.

Thanks to all who read this.

user1084684
  • 155
  • 1
  • 7