5

Java application dies from time to time with PermGen space error, but when I look at the memory usage it seems low from what I can tell.

It is a Tomcat application, plus there is SOLR server running (under the same tomcat).

JVM params in catalina.sh:

-Xms1024m 
-Xmx2048m 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/path/to/logs

Error in catalina.out:

java.lang.OutOfMemoryError: PermGen space
Dumping heap to /path/to/logs/java_pid22335.hprof ...
Heap dump file created [107041478 bytes in 1.823 secs]
Exception in thread "pool-5-thread-1" java.lang.OutOfMemoryError: PermGen space

Now the first strange thing is that memory dump is only 100Mb, while heap limit is 2048Mb. When I used to have "proper" out of memory errors, dump file was close in size to the heap limit.

Second strange thing is memory usage shown by jmap -heap 22335 seems pretty normal (when this command was run the java app was still down):

Attaching to process ID 22335, please wait...   
Debugger attached successfully.
Server compiler detected.
JVM version is 11.3-b02

using thread-local object allocation.
Parallel GC with 16 thread(s)

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 2147483648 (2048.0MB)
   NewSize          = 2686976 (2.5625MB)
   MaxNewSize       = -65536 (-0.0625MB)
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 88080384 (84.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 566689792 (540.4375MB)
   used     = 13467648 (12.84375MB)
   free     = 553222144 (527.59375MB)
   2.3765467792297907% used
From Space:
   capacity = 327680 (0.3125MB)
   used     = 0 (0.0MB)
   free     = 327680 (0.3125MB)
   0.0% used
To Space:
   capacity = 327680 (0.3125MB)
   used     = 0 (0.0MB)
   free     = 327680 (0.3125MB)
   0.0% used
PS Old Generation
   capacity = 1431699456 (1365.375MB)
   used     = 86216248 (82.22222137451172MB)
   free     = 1345483208 (1283.1527786254883MB)
   6.021951579200712% used
PS Perm Generation
   capacity = 88080384 (84.0MB)
   used     = 88080080 (83.99971008300781MB)
   free     = 304 (2.899169921875E-4MB)
   99.99965486072358% used

I looked at the memory dump file, nothing unusual there. Tried increasing heap limit, no difference.

Any ideas what could it be and why the dump file is so small, so is the memory usage while the app continues throwing out of memory errors?

serg
  • 245
  • 1
  • 3
  • 7
  • Don't have enough reputation to comment, but... I noticed that your `jmap -heap` shows ` MaxNewSize = -65536 (-0.0625MB)`. I don't think that looks normal! – Neal Gokli Dec 05 '13 at 02:03

2 Answers2

6
PS Perm Generation
   capacity = 88080384 (84.0MB)
   used     = 88080080 (83.99971008300781MB)
   free     = 304 (2.899169921875E-4MB)
   99.99965486072358% used

There's your culprit. PermGen is used for class definitions, and you seem to have a lot of them!

Try cranking up your PermGen size by adding this to your Tomcat launch options:

-XX:MaxPermSize=256M

...but if your application is smashing into your current limit, this change will only give you some extra time between crashes.

Monitor how it behaves and how quickly the usage grows, but be ready to do some digging in your application code for problem class definitions.

Shane Madden
  • 112,982
  • 12
  • 174
  • 248
  • Thanks. Is the a way to check the current limit? After the tomcat restart with new setting that section still shows 73Mb capacity and 99% usage with 0.07Mb free. – serg Aug 11 '12 at 00:36
  • Where'd you put the setting? Seems like it didn't take. – Shane Madden Aug 11 '12 at 00:40
  • Inside catalina.sh into `CATALINA_OPTS` string. Same place were Xms and Xmx are set. – serg Aug 11 '12 at 00:44
  • I run `jinfo -flag MaxPermSize ` and it showed 256mb, while without it 88mb. So I think that was it, thanks again. – serg Aug 11 '12 at 00:54
3

Max PermGen size is set by a separate command line switch, try -XX:MaxPermSize=128M, that should be enough for PermGen.

Alex
  • 7,789
  • 4
  • 36
  • 51