Cluster a custom application

Question

I have a custom application which is written by java. I want to know is it possible to cluster this application by pacemaker. For example copy this application in two different nodes and setup a cluster by pacemaker. In one time only one application work and when primary node goes down my application start to work on second node in simple word i need a way to cluster my simple application like clustering nginx server with packemaker,

And also is there any solution to cluster two OS. If first OS goes down second OS start to serve request ?

Thanks in advance

---

To clarify my scenario. I have a java application which process some messages. It is not like a service and acts like a local application. I want to be sure one instance of my application is always running in two or more nodes of my cluster and when one instance of my application is running in one node of my cluster other instances not allowed to work. I Know how pacemaker work and use float ip as a resource. I want to know is there any solution to add my application as resource or pacemaker monitor my application and when failed switch to other node.

---

Updated: for more information I want to deploy application on physical servers and I cannot use virtualization and VM. Furthermore methods like Vmware FT has some limitations like max no of VCPU and also RAM. I think it should be much easier if I can use pacemaker as a cluster resource manager but I dont know how to introduce my application as a resource and how to route all traffic to my primary server and when server or application goes down traffic and requests route to other node.

Unfortunately my application is not like a service. normally my application gathers information from different servers and sends them to another servers thus methods like loadbalancer is not suitable for my scenario.

I need a floating IP for my cluster and when serverA goes down All traffic switch to ServerB( Input and Output) also cluster start Stopped application on serverB. And when ServerA backs online all requests (Input or Output ) route to ServerA and Stop App on ServerB.

Answer 1

I don't quite understand what it is you are asking for exactly, but in general:

You can always create a fail-over cluster with a cluster manager like pacemaker. That requires no special support in the application itself.

In a fail-over cluster the cluster manager will ensure that only one instance of the application is running at any one time, with all the unique resources that application needs ( i.e. ip-address, storage etc.).

If one node fails, then the resources will be transferred and the application will be started on another node (the application is not yet up and running) reducing the amount of downtime to the absolute minimum (in theory).

---

Other cluster modes, where the application is running concurrently on more than one node at the time, will often require special support in either the application or the infrastructure.

Other clustering modes typically come with a form of loadbalancing to distributing requests to the active instances of the application.

Often (not always) the application will work out-of-the-box when the loadbalancer directs all users to one active node and the second is a hot standby with the application already running but not getting used until the primary fails.

To use active-active loadbalancing where some users are directed to one node and other concurrent users get directed to another node, that requires either an application specifically designed for horizontal scaling and loadbalancing or a really dumb, non-interactive application.

Answer 2

I'm going to make a few assumptions in my answer based on some of the details in your question, because I have to in order to construct a less generic answer. Apologies if this information is not correct.

Assumptions:

• Your Java application must be run on only one node at a time, and is stateful.
• This application has no exotic running state, will respond to very simple start and stop operations, and can be easily monitored.
• This application has no LSB init script or systemd unit file to control its state

Pacemaker uses what are called resource agents (RA) to control the state of applications. These RAs are bash or python scripts in most cases, and application-specific ones are available to better interact with those applications beyond just starting and stopping processes. However, there are several resource agents that are designed to handle "arbitrary" applications, such as the one you describe.

The "Anything" resource agent should take care of this situation, and is very generic. See its git page for details on how exactly it should be implemented. However, here is an example primitive defined via crmsh:

primitive p_anything_java-app ocf:heartbeat:anything \
    params binfile="/opt/executable-file" monitor_hook="/opt/monitor-script" \
    op start interval=0 timeout=60 \
    op stop interval=0 timeout=60 \
    op monitor interval=30 timeout=60

That's just an example. You probably will need to define more parameters than that to get things to behave consistently if this is anything but the simplest of applications.

Alternative to the "Anything" RA, is is the LSB or systemd RA, either of which interact with standard systemd unit files or LSB init scripts. If your application can be managed within that spec, it would likely be best to make a unit file or init script and use this method instead, as it gives a reliable way to run your application both clustered and unclustered without having to define all of your various relevant paths and parameters in Pacemaker. This is also a better way to implement resource monitoring, which is critical in an HA resource.

---

Concerning your "floating" IP address, this can be established using the IPaddr2 primitive. IPaddr2 will utilize GARP to announce the presence of a Pacemaker-assigned secondary IP address of an interface, which can be collocated with your application in question to accomplish inter-node failover networking. This does require operating cluster nodes on a common subnet and broadcast domain.

See the comments within the RA to get usage details.

You can also get usage details from "crmsh" if you have the relevant resource agent installed, via:

crm ra info IPaddr2

Keep in mind that these resources will need to be both collocated and ordered together to ensure they run on the same node and in the proper start/stop order. It seems that this can be addressed with a single "group".