Java application response is slow if Linux cache consumes entire system free memory

Question

I am running two Java-based application on my server with 32GB of RAM,16 core CPU. Apart from this I just have MySQL community server 5.6.16. The Java applications are assigned 10 GB of overall memory. The Xmx and Xms are same for both applications.

Now the problem is over the period of time (in about 3-4 hours) Linux assign entire free memory to cache and the Java applications slow down drastically. As soon as I clear the cache memory, the applications start responding normally.

Here is the output of the free command:

$ free -m
             total       used       free     shared    buffers     cached
Mem:         32095      31767        328          0       1380      11245
-/+ buffers/cache:      19141      12953
Swap:        29999       3918      26081

The Heap+Non heap memory utilization for both Java applications are not crossing 75% at any time.

What could be the reason behind this? Is this Linux memory management issue?

Thanks in Advance