0

This is a multi-step question, so bear with me.

I am considering building a small (10 - 20 node) cluster. However, I want normal programs (not designed for clusters) to be able to take advantage of the extra processing speed. In the most ideal scenario, I would like to be able to run a single hypervisor over the whole cluster. As far as I can tell, there is not a good solution to this problem that can take a normal program and run it faster on a cluster.

Therefore, I am brainstorming how I would go about designing such a system, and if it is feasible to attempt. It seems that the inherent problem with clustering is that it takes more time to move data around than it does to process it. (i.e. it takes 2 seconds to transfer a problem from one node to another, but only 1 second to solve it on the first node.) However, I have thought of a possible solution for this.

Let's just say that it is theoretically possible for all the nodes in a cluster to boot off of the same disk. Therefore, they all have direct access to identical data and identical programs. Secondly, let's suppose that the Linux kernel could be modified to send each new command to a different slave node, looping infinitely through all the nodes. Given these two conditions, a user can log into the terminal of the master node and run commands in a normal (non-cluster oriented) format, but the load of the commands will be more or less evenly spread across the cluster.

So with that introduction, here are my two questions:

  1. Is it possible to create such an environment in which all the computers boot off of a single disk (probably a NAS)? (I am aware of PXE, but as far as I can tell it does not provide persistent storage, it only hosts the OS.) If it is currently possible, how can it be done?
  2. Is it possible to modify the kernel to delegate each new command to a separate node? (This may be able to be done by modifying the bash binary instead of the kernel itself - I am not sure.) If so, please expound on how.

That's the most complicated question I have ever tried to ask on Stack Exchange, so I expect people to have questions in the comments. However, if this solution could actually be implemented, it could potentially revolutionize virtualization.

B00TK1D
  • 685
  • 4
  • 18
  • By the way, I don't want to sound like I think I know better than all companies that have tried to do this. I just had an idea on how to solve it, and couldn't see a reason why it wouldn't work. I understand this won't be simple to implement though! – B00TK1D May 12 '17 at 19:21

2 Answers2

2

Is it possible to create such an environment in which all the computers boot off of a single disk (probably a NAS)? (I am aware of PXE, but as far as I can tell it does not provide persistent storage, it only hosts the OS.)

Sure, a combination of PXE with NFS-mounted share for persistent storage.

Is it possible to modify the kernel to delegate each new command to a separate node?

Yes.

Simple, right? No so much.

The question you should be asking is this: how much work would it be to do this?

There have been companies working on this problem for 30 years, with billions of USD invested, and the problem isn't yet solved. That's not to say it couldn't be solved, but it's a massively-complex problem.

EEAA
  • 108,414
  • 18
  • 172
  • 242
  • Thanks for the quick answer - could you expound on the combination of PXE and NFS? Also, do you have any idea whether the command delegation would be better implemented in the kernel or in bash? – B00TK1D May 12 '17 at 19:19
  • Boot the system off of PXE, then have it mount an NFS filesystem somewhere. This is a *very* common use pattern. As to your other question, that's entire off-topic here, as it's a system design and software development question. – EEAA May 12 '17 at 19:21
  • Okay, thanks. I can do my own research from there. But how is making changes to the kernel and binaries off topic on Server Fault? Where should that question be asked? – B00TK1D May 12 '17 at 19:23
  • 2
    Stack Overflow is for software development questions. However, beware that what you're undertaking is a *massive* effort, and if asked without detail and/or examples of what you've tried, it'll likely be closed there. I applaud your desire to solve this problem - if you do, you'll be an instant millionaire. Just know that it is a very, very difficult problem. You'll learn firsthand very shortly that things are much more complicated in this realm than it initially seems. – EEAA May 12 '17 at 19:25
0

Multiple node, independent OS image, shared storage systems exist. For example, VMScluster running on OpenVMS is inherently a multiple node system. However, that is not a Linux operating system, and applications would need to be designed for the different platform and multiple node awareness.

Usually Linux scales either bigger or to more machines. For bigger, add more CPUs and memory to one OS image. Think big database server. For more, add smaller machines with their own OS image, network them together, and use some kind of job management software or other load balancer. Think cloud application or high performance computing. Edit: Note that HPC applications in particular use MPI interfaces for direct memory access. This requires them to be developed for and built against those message libraries.

John Mahowald
  • 30,009
  • 1
  • 17
  • 32
  • While somewhat interesting, this doesn't answer my questions at all. My whole goal is to avoid customizing programs to a cluster-oriented operating system, and avoid a load balancer. – B00TK1D May 15 '17 at 16:39
  • Using unmodified programs, and not using load balancing, doesn't leave much. There exists single system image clustering software, like MOSIX, but it has its own limitations. Scale up server and load balances are vastly more common. – John Mahowald May 16 '17 at 03:40