Similar to Is there a Distributed SAN/Storage System out there? or Distributed redundant storage solution?, except with a couple key differences (and two years newer).
Context: I have a small Rocks cluster (a few dozen machines of varying capabilities), and would like to have all of my nodes pool their excess storage space to allow for a distributed filesystem. I maybe can spare a machine that would act as a server for it, but would prefer not to.
Unlike the above, all of these machines share user lists, and no user can gain root--security and encryption concerns are nearly negligible. (FS mounted by root; contains one directory per user with their perms)
I feel like it should be pretty easy to do, I just can't find any solutions that aren't WAY more complicated than what I'm looking for.
EDIT: My motivation is that every now and then a big IO load hits, taking down the NFS. I am well aware that "go buy a nice SAN with a bunch of SSDs" is a quite nice solution; it's just not available at the moment--I'd like to know what I can do with software and the hardware I have.