Memory cache is full not sure what is using it and if it is normal

Question

The situation is that amazon bots are download ~600k images off my server. My web app deployed on apache server is extremely slow probably because ram usage is 100%

some statistics:

root@407096:/home# free -m
             total       used       free     shared    buffers     cached
Mem:         16384      16380          3          7          0      16149
-/+ buffers/cache:        231      16152
Swap:         8192        122       8069

root@407096:/home# top

top - 16:18:29 up 21:04,  2 users,  load average: 0.13, 0.32, 0.33
Tasks:  44 total,   1 running,  42 sleeping,   1 stopped,   0 zombie
%Cpu(s):  1.2 us,  0.9 sy,  0.0 ni, 97.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  16777216 total, 16774160 used,     3056 free,        0 buffers
KiB Swap:  8388608 total,   125904 used,  8262704 free. 16535440 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                     
13710 www-data  21   1 2223944  36712   4996 S  12.3  0.2 116:39.27 apache2                                                                                     
14420 www-data  20   0  645668  12780   3000 S   1.3  0.1  63:23.26 apache2                                                                                     
13798 www-data  20   0  774092  11276   3004 S   1.7  0.1  63:36.86 apache2                                                                                     
13883 www-data  20   0  840732  11160   3004 S   1.0  0.1  63:19.59 apache2                                                                                     
14511 www-data  20   0  510772   9976   3004 S   1.3  0.1  62:05.79 apache2                                                                                     
 2402 root      20   0  565640   8196   4392 S   0.3  0.0   6:30.79 python                                                                                      
  631 mysql     20   0  644352   6836   4576 S   0.0  0.0   0:48.47 mysqld                                                                                      
20680 root      20   0   89564   3656   2780 S   0.0  0.0   0:00.00 sshd                                                                                        
 1168 root      20   0   60512   3228   1840 S   0.0  0.0   0:15.22 supervisord                                                                                 
17856 root      20   0   90912   3176   3048 S   0.0  0.0   0:00.55 sshd                                                                                        
20295 root      20   0   90912   3164   3044 S   0.0  0.0   0:00.06 sshd                                                                                        
11353 root      20   0   88420   2676   2332 S   0.0  0.0   0:04.31 apache2                                                                                     
  468 root      20   0   61316   2508   2400 S   0.0  0.0   0:02.32 sshd                                                                                        
14985 postfix   20   0   40120   2484   2360 S   0.0  0.0   0:00.00 tlsmgr                                                                                      
20469 root      20   0   18392   2284   1556 S   0.0  0.0   0:00.05 bash                                                                                        
20457 root      20   0   39244   2112   1036 S   0.0  0.0   0:00.38 redis-server                                                                                
17870 root      20   0   18296   1968   1588 S   0.0  0.0   0:00.22 bash                                                                                        
    1 root      20   0   33312   1936   1368 S   0.0  0.0   0:00.83 init                                                                                        
20467 root      20   0   40808   1804   1428 S   0.0  0.0   0:00.00 sudo                                                                                        
20681 sshd      20   0   62736   1532    772 S   0.0  0.0   0:00.00 sshd                                                                                        
20307 root      20   0   18276   1528   1528 S   0.0  0.0   0:00.02 bash                                                                                        
  389 syslog    20   0  184192   1524    968 S   0.0  0.0   0:02.53 rsyslogd                                                                                    
20665 root      20   0   19828   1460   1080 R   0.0  0.0   0:00.03 top                                                                                                   
  666 postfix   20   0   27412   1444   1356 S   0.0  0.0   0:00.04 qmgr                                                                                        
 1143 root      20   0   83552    372    372 S   0.0  0.0   0:00.00 saslauthd                                                                                   
 1144 root      20   0   83552     24     24 S   0.0  0.0   0:00.00 saslauthd                                                                                   
 1168 root      20   0   60512   3228   1840 S   0.0  0.0   0:15.21 supervisord

I'm not sure what is using up cache. Is it possible that apache is using cache for image downloading and therefore my web app hosted on the same apache is laggy ?

Buffers/Cache are not linked to a process but using [ftools](https://code.google.com/p/linux-ftools/) you could somehow indentify the content. — Xavier Lucas, Oct 14 '14 at 13:33
@Marijus Marijus To the kernel virtual memory management. And what's cached is called pages. It's usually just file blocks put inside RAM for faster access. Nothing to worry about, that's an optimization and memory will be reallocated if needed by other tasks. — Xavier Lucas, Oct 14 '14 at 13:42

score 0 · Answer 1 · answered Oct 14 '14 at 16:59

cached is a value that's displaying the amount of data stored in RAM that's a filesystem-level cache. The way it works is that data is read from/written to the cache first, then dumped to disk later. Commonly used data is also kept around for faster read access (think shared object files, etc). This improves I/O dramatically, and is a feature within the Linux kernel. This usage is prioritized so that if an application requests more RAM, the cache is freed up to make room for it. With this in mind, the cache is not causing your problem.

The following is just conjecture, and there's no significant proof that this is the case, this is just based on past experience.

Amazon's CPU's are slightly slower than other servers/VPS's. I've proved this crudely with several basic applications and scripts doing nothing but incrementing counters in while loops for a given amount of time. AWS instances consistently underperformed in comparison to dedicated machines, and other VPS's. This is likely due to the shared platform nature of EC2 (other instances running on the same physical host as yours). Does your app under-perform consistently on EC2 but works fine in your development environment? For $5/month you can try running it on a DigitalOcean VPS to see whether EC2 is to blame or whether there's something else going on.

Memory cache is full not sure what is using it and if it is normal

1 Answers1