1

We have slow running JAVA program on a Solaris 10 Zone. I ran a truss on the pid and noticed following messages getting generated even when program is not doing anything:

pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/29:    pollsys(0x00000000, 0, 0xFFFFFFFE242FF970, 0x00000000) = 0^M
/29:            timeout: 0.050000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/29:    pollsys(0x00000000, 0, 0xFFFFFFFE242FF970, 0x00000000) = 0^M
/29:            timeout: 0.050000000 sec^M
/22:    lwp_cond_wait(0x101585840, 0x101585828, 0xFFFFFFFE250FE960, 0) Err#62 ETIME^M
/63:    pollsys(0xFFFFFFFE135FE458, 1, 0xFFFFFFFE135FE380, 0x00000000) = 0^M
/63:            fd=46 ev=POLLIN rev=0^M
/63:            timeout: 0.100000000 sec^M

It is also preceeded/followed by:

lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M
/62:    pollsys(0xFFFFFFFE137FE4D8, 1, 0xFFFFFFFE137FE400, 0x00000000) = 0^M
/62:            fd=32 ev=POLLIN rev=0^M
/62:            timeout: 0.100000000 sec^M
/63:    lwp_mutex_wakeup(0xFFFFFFFF7EAF2AE0, 0)         = 0^M
/51:    pollsys(0x00000000, 0, 0xFFFFFFFE14DFEEC0, 0x00000000) = 0^M
/51:            timeout: 0.010000000 sec^M
/37:    lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M
/37:    lwp_mutex_wakeup(0xFFFFFFFF7EAF2AE0, 0)         = 0^M
/61:    lwp_mutex_timedlock(0xFFFFFFFF7EAF2AE0, 0x00000000) = 0^M

Would we know what this might mean? Anyway to figure more on it? I read we can run dtrace, but is there any other utility we can run to get where the slowness is happening. The app takes a lot of time to run basic queries, which works much smoother on a windows server.

uSlackr
  • 6,337
  • 21
  • 36
turtle
  • 11
  • 1
  • 2

2 Answers2

1

Your best bet is to use the Java debugger and see what the the JVM threads are doing. It's really hard to tell from a system call trace what's going on.

JOTN
  • 1,727
  • 1
  • 10
  • 12
  • Thanks JOTN, the same app is working just fine on Linux Jrockit. So I was thinking its probably less to do with java and more to do with underlying OS/hardware. I can enable java debugger – turtle May 03 '11 at 18:17
  • In that case I would look at a difference in the JVM as the likely problem. Maybe default heap size isn't big enough? I did hit one issue with a release of the Oracle JVM where it started doing reverse DNS lookups on socket connections as a new security measure. The connecting IP wasn't in DNS so it hung waiting for a response. – JOTN May 03 '11 at 22:21
1

You're looking at the JVM manipulating mutexes (locks). Waiting for a lock can make a thread sit around not using much CPU time, but not making any progress(!)

--dave

davecb
  • 211
  • 2
  • 5