0

I am trying to configure a JBoss AS 7.1.1 cluster in domain mode with mod_cluster at the apache2 proxy. All hosts are running Ubuntu and are Rackspace Cloud Servers.

My problem is that when I hit my app at http://project.example.com/ I just get a 503

JBoss log reports:

org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy proxy-ip/proxy-ip:6666, configuration will be reset

The mod_cluster management page has this information:

DUMP output

balancer: [1] Name: main-server-group Sticky: 1 [JSESSIONID]/[jsessionid] remove: 0 force: 0 Timeout: 0 maxAttempts: 1
node: [1:1],Balancer: main-server-group,JVMRoute: slave:server-two,LBGroup: [],Host: project-jb-node1,Port: 8159,Type: ajp,flushpackets: 0,flushwait: 10,ping: 10,smax: 26,ttl: 60,timeout: 0
node: [2:2],Balancer: main-server-group,JVMRoute: master:server-one,LBGroup: [],Host: project-jb-master,Port: 8009,Type: ajp,flushpackets: 0,flushwait: 10,ping: 10,smax: 26,ttl: 60,timeout: 0
node: [3:3],Balancer: main-server-group,JVMRoute: master:server-two,LBGroup: [],Host: project-jb-master,Port: 8159,Type: ajp,flushpackets: 0,flushwait: 10,ping: 10,smax: 26,ttl: 60,timeout: 0
node: [4:4],Balancer: main-server-group,JVMRoute: slave:server-one,LBGroup: [],Host: project-jb-node1,Port: 8009,Type: ajp,flushpackets: 0,flushwait: 10,ping: 10,smax: 26,ttl: 60,timeout: 0

INFO output:

Node: [1],Name: slave:server-two,Balancer: main-server-group,LBGroup: ,Host: project-jb-node1,Port: 8159,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 26,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [2],Name: master:server-one,Balancer: main-server-group,LBGroup: ,Host: project-jb-master,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 26,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [3],Name: master:server-two,Balancer: main-server-group,LBGroup: ,Host: project-jb-master,Port: 8159,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 26,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [4],Name: slave:server-one,Balancer: main-server-group,LBGroup: ,Host: project-jb-node1,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 26,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

After a restart the apache error.log has:

[Sun May 26 19:17:51 2013] [notice] Apache/2.2.22 (Ubuntu) mod_cluster/1.2.0.Final configured -- resuming normal operations
[Sun May 26 19:17:52 2013] [warn] manager_handler STATUS error: MEM: Can't read node
[Sun May 26 19:17:53 2013] [warn] manager_handler STATUS error: MEM: Can't read node
[Sun May 26 19:17:55 2013] [warn] manager_handler STATUS error: MEM: Can't read node
[Sun May 26 19:18:00 2013] [warn] manager_handler STATUS error: MEM: Can't read node
[Sun May 26 19:18:26 2013] [error] proxy: CLUSTER: (balancer://main-server-group). All workers are in error state

If I hit http://project.example.com/index.html I get this in the apache error.log:

[Sun May 26 19:34:33 2013] [warn] proxy: No protocol handler was valid for the URL /index.html. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

The hostnames (project-jb-master etc) are mapped to IPs in /etc/hosts and I can connet using telnet and can ping.

Googling around this I find lots of mentions of the instance-id so I set this in the full-ha profile:

<subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">
    <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
    <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>
    <virtual-server name="default-host" enable-welcome-root="false">
        <alias name="localhost"/>
        <alias name="project.example.com"/>
    </virtual-server>
</subsystem>

but the original message is still there.

Simon Gibbs
  • 61
  • 1
  • 11

3 Answers3

1

Multicast isn't currently allowed on Rackspace Cloud Servers. The traffic volume of multicast traffic ends up overloading the switching layer and it degrades network performance for all traffic.

For this reason the configuration should look something like this:

Apache config:

Listen host:6666

<VirtualHost host:6666>
  EnableMCPMReceive
  <Location /mod_cluster-manager>
    SetHandler mod_cluster-manager
  </Location>
</VirtualHost>

ServerAdvertise is not enabled

JBoss config:

<subsystem xmlns="urn:jboss:domain:modcluster:1.1">
  <mod-cluster-config advertise="false" connector="ajp" proxy-list="host:6666">
    ...
  </mod-cluster-config>
</subsystem>

proxy-list

Defines a comma delimited list of httpd proxies with which this node will initially communicate. Value should be of the form:

address1:port1,address2:port2

advertise

Using the default configuration, this property can by manipulated via the jboss.mod_cluster.proxyList system property.

If enabled, httpd proxies will be auto-discovered via multicast announcements. This can be used either in concert or in place of a static proxyList.

mod_cluster Documentation

Federico Sierra
  • 3,499
  • 1
  • 18
  • 24
0

Set your Apache HTTP Server to LogLevel debug. Then you will be able to see in the error_log, what messages are AS7 instances sending. Furthermore, update to mod_cluster 1.2.6. It is fully backward compatible with 1.2.4 and it contains tons of important bug fixes.

Michal Karm Babacek
  • 155
  • 1
  • 7
0

proxy: No protocol handler was valid for the URL /index.html. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

Make sure you have loaded mod_proxy_http and/or mod_proxy_ajp.

JorSol
  • 299
  • 2
  • 9