2

My team is in the process of building an application that will be deployed on AWS EC2, and will communicate with on-premise legacy systems via IBM MQ (formerly known as WebSphere MQ, formerly known as MQSeries).

We already have IBM MQ queue managers on our premises. Do we also need to deploy one in EC2? Do we need to deploy them on the EC2 boxes where our application will run?

I am new to IBM MQ, although i have some experience with RabbitMQ. There are people in the company with a lot of experience with IBM MQ, but none of cloud applications. I've been told several times by them that we need to run a queue manager on the box with our application, which can then forward to other queue managers, but on a cloud machine with an ephemeral disk, that doesn't make any sense to me - it doesn't add any reliability.

We could deploy a queue manager on an EC2 machine with an EBS volume, which would make a bit more sense, and have our applications talk to that. But is that really any better than just talking directly to the existing queue managers on our own premises?

To add some bonus fun, we're not deploying the application directly on EC2, but on Cloud Foundry deployed on EC2, so the app instances are running inside containers that we don't have much influence over.

Tom Anderson
  • 387
  • 1
  • 11

1 Answers1

4

Worrying about the type of volume that the queue manager will be deployed to isn't what you should be worried about. IBM MQ has specific guidelines on what volumes are supported and what volumes are not supported. Have a look in the MQ Knowledge Center for more information.

MQ applications can run either locally or remotely (to a queue manager).

(1) MQ applications that connect locally (bindings mode) to a queue manager, can make application development and support a lot easier because there is NO network involved in getting and/putting messages.

(2) MQ applications that connect remotely (client mode) to a queue manager, is a lot more complex. Someone else recently posted a similar question on the MQ ListServer and here is T.Rob Wyatt's excellent response.

There are a whole host of issues when consolidating from stand-alone QMgrs to shared hubs. The one you are asking about is among the most mechanical and uninteresting of the bunch. As FJ points out there are some security implications and the connectivity issues you yourself have stumbled across. Let me give you a high-level round-up of some others. The degree to which you experience most of these depends on how much the environment is currently stand-alone and how much sharing it currently has. The exception is data integrity when switching to client so I'll talk about that first.

No messages lost or duplicated, order mostly assured:

This requires XA transactionality. XA imposes its own set of design constraints, among which is that the app in question cannot fail over to a different QMgr. MQ generally behaves as everyone seems to expect in that no messages are duplicated, lost or disordered under normal circumstances.

Dupe messages, disordering possible:

This requires at least single-phase commit. If you GET a message off a queue under syncpoint and the client app gets a 2059 on the next API call, then you can be assured that the message will be rolled back and not lost. But the app will see it again and if it was processed correctly the first time it will appear to be a dupe. Also, if the app got a 2059 on a COMMIT after PUTting a message, it has no choice but to assume the COMMIT failed and issue the PUT again. Unlike the "functional duplicate" from the GET, this generates multiple identical messages.

The channel agent holding the transaction with one or more GETs won't release the messages until MQ or TCP times it out and kills the orphan connection. Since the rollback doesn't necessarily happen before the app reconnects, it is likely to get subsequent messages not under that syncpoint. When the orphaned channel is killed and the messages reappear on the queue, they will be delivered but the order will be disrupted.

Dupe, missing messages, disordering possible: Client connections that use no XA and no single-phase COMMIT can lose or duplicate messages or disorder them for the reasons stated above.

Roger
  • 141
  • 2
  • It's also worth noting that the issues raised are endemic to async messaging and not to IBM MQ specifically. So much so that the JMS specification makes an exception that dupe messages caused by session exceptions are "functional duplicates" and do not violate once-and-only-once QOS. – T.Rob Oct 15 '16 at 04:21