3

In a cluster of applications nodes, what are the advantages/disadvantages of placing your application binaries on a shared mount and having each of the nodes mount that, rather than having separate copies of the binaries on each node?

Is there anything that makes one approach or the other particularly bad?

Thanks

Brendan Maguire
  • 313
  • 1
  • 2
  • 11

1 Answers1

6

When I started in the financial industry, I was running systems that were comprised of gateway systems that consumed/massaged data, compute nodes to make trading decisions, NFS servers to serve applications/centralize logging and a lot of networking to bind it all together. We also centralized crontab scheduling and application monitoring configs and placed those on NFS mounts.

This was NFSv3-based, and the benefits were, um...

  • Easy application distribution: One mount. One place to make changes.
  • Simple management of the data related to those applications.

Downsides:

  • Susceptible to errors: One mount. One place to screw up changes.
  • Scalability: Unless you're also clustering your storage, the central mount will have an upper limit on the number of clients it can serve effectively.
  • Single point of failure (SPOF): Again, clustered storage can mitigate this, but it's a consideration. Assuming NFS, what happens if you need to reboot the server? Will cluster nodes hang on boot if the NFS server is missing or unavailable?
  • Permissions: How will you manage permissions across cluster nodes. Local service accounts? Matching UIDs/GIDs? In my case, I used NIS and later, LDAP. A directory service would be helpful here.
  • Performance: You're dependent on networking and the switching infrastructure here. Performance of things like application loading, logging, etc. will be faster on local disk than over-the-wire.
  • Tuning NFS is hard, and most people don't do it well :)

These days, you can deploy and manage your software via a configuration management system (Puppet, Chef, Ansible, etc.). Or even better, use the native packaging method of the operating system (RPM, .deb) and deploy it with configuration management:

  • This removes the SPOF.
  • Localizing the application should have a performance benefit.
  • Simplifies networking: E.g. what if you expand to multiple locations?
  • Can provide more granular control: e.g. distribute a binary to a subset of systems, versus the entire cluster.
  • Self-documenting.

Given that, I'd probably recommend separate copies of the applications instead of using a shared mount.

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • 2
    I don't disagree with your conclusion, but on the subject of NFS tuning - the #1 suggestion is to ask the vendor/check the docs of your NFS server for which NFS mount options your clients should use. The #2 suggestion is to use a dedicated network with jumbo frames. (I imagine you probably know all that, but hopefully it's useful to readers.) – Dan Pritts Nov 13 '14 at 17:57