6

I would like to optimize APC some more but I am not sure where I could do something. First here is the screenshot after one week of running with the current configuration: APC Dashboard

I have now the following points that I am unsure of:

  1. Do I see it correctly that the fragmentation happens because the cache is used as an user cache too?
  2. Why does the fragmentation bar tell me 100% of only 5.8MB when I allocated 192MB in total?
  3. Is this just a rendering problem that the circle under "Memory Usage" is not fully closed? Because the MB values below do add up. (To say is that the circle looks nice after a restart it just becomes like this when the cache gets more and more fragmented.)
  4. Since the hit rate is really good I am not sure if the fragmentation is a big problem or not. Do you think that I can still optimize it?

I am mostly interested in getting those questions answered. Only then I can understand APC better and make adjustments myself.

Some detail information: On this server is Drupal and Magento running. Drupal is using it as a user cache too.

The question for me is now how I could optimize it. I could allocate more RAM but I am not sure if this really helps a lot.

UPDATE: Here is the configuration:

; The size of each shared memory segment in MB.
apc.shm_size = 192M

; Prevent files larger than this value from getting cached. Defaults to 1M. 
apc.max_file_size = 2M

; The number of seconds a cache entry is allowed to idle in a slot in case
; this cache entry slot is needed by another entry.
apc.ttl = 3600

As you can see it is pretty minimal at the moment.

Raffael Luthiger
  • 2,011
  • 2
  • 17
  • 26

3 Answers3

5

Do I see it correctly that the fragmentation happens because the cache is used as an user cache too?

No, fragmentation can happen when a file's opcode cache size has changed, and it won't fit into the 'slice' it occupied before - under the hood it's a bit more complicated, but that's the gist.

Why does the fragmentation bar tell me 100% of only 5.8MB when I allocated 192MB in total?

It's telling you 100% because out of your free, available cache space, 100% of it is implicated in fragmentation (meaning the barrier to entry for that 'slice' has to fall within a fragmented size).

Is this just a rendering problem that the circle under "Memory Usage" is not fully closed?

Yes, don't trust it; the values printed 'within the slices' are incorrect at times as well.

Since the hit rate is really good I am not sure if the fragmentation is a big problem or not. Do you think that I can still optimize it?

Absolutely! I'd increase the cache. You can have an allocation of 100MB, and have 10MB cached, and still have a 100% hit rate.

A successful setup should have little pruning (AKA: few to no gc) - and elbow room for expansion, you want a little extra space; (more than 5MB) because the effects of fragmentation will 'complicate' things that want to enter the cache.

Shoot for 10% free, non-fragmented space, continue monitoring and increasing the size. Also know that pushing large amounts of new code files will have an effect on the cache as well.

thinice
  • 4,676
  • 20
  • 38
  • Thank you for the complete answer. I have a few question though: I have set the apc.stat setting and I am sure that no files changed. What exactly is then changing the op-code cache size? Do you know what algorithm is used for the memory management? Why I come up with this question: Could it be that it takes a lot longer to find a cached op-code if all those op-codes are spread over a lot of RAM? – Raffael Luthiger Jan 26 '12 at 16:25
  • 1
    It's unlikely there's a performance impact when opcodes are in RAM, the lookup method is essentially based off of the file locations themselves - something that can happen extremely fast unless you have tens of thousands of files cached. As for what's changing in your cache, it could be a mixture of ttl, pruning and user cache in your case. TTL being a bigger culprit. – thinice Jan 26 '12 at 16:44
  • I should add that you _do_ have a lot of little user variables, which are definitely a part of the problem. From a programmer's point of view, those variables can be extremely important for speed's sake, unless the code is using them ineffectively. – thinice Jan 26 '12 at 17:01
  • Thank you for pointing out the TTL. Maybe I should look at it this some more. – Raffael Luthiger Jan 26 '12 at 22:27
1

Regarding Question 3,
I cant exactly tell you why the circle is not fully closed, but it looks the same for my setup, after a few days uptime and with 10% fragmentation.

2,
I think that's a glitch in the APC Statistics Page, mine tells me "10.34% (771.7 KBytes out of 7.3 MBytes in 89 fragments) " where 7.3 MBytes is excatly the amount of memory that should be free according to the memory usage stats.
If you take a look at your screenshot, you will notice that it looks like its doing the same thing there
5.8 of 5.8 fragmented and 5.8 free, so either the fragmentation is somehow calculated on the free memory or the numbers are just wrong.

See my screenshot for comparison:
APC Stats

Niko S P
  • 1,182
  • 8
  • 15
  • OK. You are guessing too. But I think you are on the right track. Do you use "user cache" too or is all only "file cache"? Then this could answer question 1. – Raffael Luthiger Jan 23 '12 at 09:20
  • I am not using user cache at all, so your fragmentation issue might come from using usercache. I would recommend to setup a simple check which calls some of the urls on this server and compare the latencies, then do a APC opcode Cache clear and see wether the latencies have gone down, this would answer question 4. – Niko S P Jan 23 '12 at 14:22
0

You should have a look at the "Cache full count" counter, >1 it indicates you run out of memory for APC, in this case you ran out 39 times in 3 days which seems a lot.

It seems more likely your use of user cache is causing this fragmentation than the use of php file caching, because it tends to cache many small different objects, which end up occupying memory just to reference their existence.

The first thing I'd do is try to up the allocated APC memory (safely, 2Go should not be the solution) until the "cache full count" no longer happens, then I would concentrate on the number of objects pushed in user cache.
If you can limit the number of them (rather than their size) it would be great but you could explore different possibilities like memcache to store that user cache and dedicate APC to file caching.

Posting you current APC configuration would help a lot as we could have a look at TTL values which may be important in this case too.

Shadok
  • 623
  • 5
  • 10
  • I can not change the code as I am only responsible for the server but not for the website. So whatever Drupal is doing with the user cache stays like this for the moment. I can only make recommendations to the webmaster. I added the configuration. – Raffael Luthiger Jan 18 '12 at 11:41
  • And can you give me some answers to question 2 and 3? – Raffael Luthiger Jan 18 '12 at 11:42
  • 2
    39 gc cycles for 100million hits, 100% hit ratio, and you think the cache is struggling??? It looks to me that this is a near optimal setup - yes adding more memory will reduce fragmentation and GC requency - but that usually means stealing it from something else - in this instance I see no evidence of a requirement to do so. – symcbean Jan 18 '12 at 12:26
  • 1
    @symcbean you just said it yourself, adding memory will reduce fragmentation and limit transaction from/to the apc cache, thus limiting the risk of cache slam. All that for the cost of a few dozens Mo of RAM, isn't that great or at least worth trying ? – Shadok Jan 18 '12 at 13:14
  • Interesting discussion. But I would like to have more information about question 2 and 3 too. :) – Raffael Luthiger Jan 18 '12 at 22:08