We are connecting with some JMS clients to ActiveMQ Artemis (v2.14.0) brokers running in a cluster configuration. Today, we have noticed that the clients connect to the acceptor that is dedicated to cluster communications and are wondering why it happens.
Here are the relevant broker.xml snippets:
<configuration xmlns="urn:activemq" xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xsi:schemaLocation="urn:activemq:core ">
[…]
<connectors>
<connector name="netty-connector">tcp://${ipv4addr:localhost}:61618?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;useEpoll=true</connector>
</connectors>
<acceptors>
<!-- Acceptor for every supported protocol -->
<acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;connectionsAllowed=10000</acceptor>
<!-- AMQP Acceptor. Listens on default AMQP port for AMQP traffic.-->
<acceptor name="amqp">tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpMinLargeMessageSize=102400;amqpDuplicateDetection=true</acceptor>
<!-- STOMP Acceptor. -->
<acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>
<!-- HornetQ Compatibility Acceptor. Enables HornetQ Core and STOMP for legacy HornetQ clients. -->
<acceptor name="hornetq">tcp://0.0.0.0:5445?anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;protocols=HORNETQ,STOMP;useEpoll=true</acceptor>
<!-- MQTT Acceptor -->
<acceptor name="mqtt">tcp://0.0.0.0:1883?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=MQTT;useEpoll=true</acceptor>
<acceptor name="netty-acceptor">tcp://0.0.0.0:61618?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;useEpoll=true</acceptor>
<acceptor name="artemis-tls">tcp://0.0.0.0:61617?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true;sslEnabled=true;keyStorePath=/var/lib/artemis/certs/keystore.jks;keyStorePassword=${keyStorePassword};enabledProtocols=TLSv1.2</acceptor>
</acceptors>
<broadcast-groups>
<broadcast-group name="cluster-broadcast-group">
<broadcast-period>5000</broadcast-period>
<jgroups-file>jgroups.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<connector-ref>netty-connector</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="cluster-discovery-group">
<jgroups-file>jgroups.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="artemis-cluster">
<connector-ref>netty-connector</connector-ref>
<retry-interval>500</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>STRICT</message-load-balancing>
<!-- <address>jms</address> -->
<max-hops>1</max-hops>
<discovery-group-ref discovery-group-name="cluster-discovery-group"/>
<!-- <forward-when-no-consumers>true</forward-when-no-consumers> -->
</cluster-connection>
</cluster-connections>
</core>
</configuration>
The intention is to use port 61616 (plain TCP, acceptor artemis) and 61617 (TLS, acceptor artemis-tls) for client connections. The brokers shall use port 61618 (acceptor netty-acceptor) for cluster-internal communications. However, during topology discovery, the broker sends port 61618 back to the client (instead of the expected port 61616). While this works when all communications is plain TCP, things get weird when netty-acceptor is configured for TLS while client connections are not. The client then shows messages like
2020-08-24 17:58:13,833 | WARN | Thread-1 (ActiveMQ-client-netty-threads) | i.n.c.ChannelInitializer | Failed to initialize a channel. Closing: [id: 0x5bb533bc]
java.lang.Exception: Failed to find a store at /var/lib/artemis/certs/truststore.jks
at org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.validateStoreURL(SSLSupport.java:278)
So, the broker passes its internal configuration to the client. (/var/lib/artemis/certs/truststore.jks exist in the Artemis container only.)
How can we configure the broker for dedicated client and cluster connections and ensure that the client never receives the cluster endpoint? Also, can we configure TLS independently for client and cluster connections? The docs are not really helpful here.
EDIT
Rethinking this issue leads to a related question: How to ensure that clients connecting via TCP always receive the TCP port during discovery and clients connecting via TLS receive the TLS port?
UPDATE
A closer look to the Artemis docs reveals this explanation:
Although this value can be configured on the server, it is downloaded and used by the client.
Okay, that explains a bit. However, I still cannot figure out how to tell the clients which connector to use. In the RedHat AMQ 7.2 docs, chapter 6.3 says
The connector above would be referenced by a client, or even the broker itself, when making a TCP connection to the specified IP and port, 10.10.10.2:61617.
suggesting the client matches IP address and ports to find an appropriate connector. However, this does not seem to work with Artemis v2.14.0.
