I am having troubles with Torque + Maui.
The problem is the following: I have 2 queues, each queue has 10 associated nodes. If i submit 10k jobs to the first queue and i submit 1 job to the second one, the job in the second one remains in Q state.
[root@localhost bin]# ./showstart 366808
job 366808 requires 2 procs for 1:00:50
Earliest start in 00:00:00 on Fri Dec 18 16:27:21
Earliest completion in 1:00:50 on Fri Dec 18 17:28:11
Best Partition: DEFAULT
[root@localhost bin]# ./checkjob -v 366808
.....
PE: 2.00 StartPriority: 3
job can run in partition DEFAULT (80 procs available. 2 procs required)
[root@localhost bin]# ./diagnose -n
Total Nodes: 20 (Active: 10 Idle: 10 Down: 0)
[root@localhost server_logs]# cat 20151218 | grep 366808
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;366808.localhost.cluster;ready to commit job
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;366808.localhost.cluster;ready to commit job completed
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;366808.localhost.cluster;committing job
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;svr_setjobstate;svr_setjobstate: setting job 366808.localhost.cluster state from TRANSIT-TRANSICM to QUEUED-QUEUED (1-10)
12/18/2015 16:23:47;0100;PBS_Server.44477;Job;366808.localhost.cluster;enqueuing into dev, state 1 hop 1
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;svr_enquejob;jobs queued job id 366808.localhost.cluster for dev
12/18/2015 16:23:47;0008;PBS_Server.44477;Job;req_commit;job_id: 366808.borcluster1.cm.cluster
[root@localhost log]# cat maui.log | grep 366808
12/18 16:25:06 MPBSJobUpdate(366808,366808.localhsot.cluster,TaskList,0)
12/18 16:25:06 INFO: job '366808' Priority: 1
12/18 16:25:06 INFO: job '366808' Priority: 1
12/18 16:25:06 MJobPReserve(366808,DEFAULT,ResCount,ResCountRej)
12/18 16:25:06 MJobPReserve(366808,DEFAULT,ResCount,ResCountRej)
As we can see, is not a resources problem, since Maui says clearly that the job can be executed.
The job gets executed only if i force it with qrun
