16

I've found myself needing redundant storage at a block level. File-level replication (Gluster, GFS, etc) does not work for my use case.

It looks like DRBD is the go-to solution for block replication. There don't seem to be too many other sane options. Have I failed in my research, or is DRBD the only game in town?

Charles
  • 1,194
  • 2
  • 12
  • 22
  • Hi Charles, why can't you use file level replication?\ – nsn Oct 04 '15 at 17:51
  • The use case was replication of VM disk devices across machines, where the devices were backed by LVM volumes and served via iSCSI - block devices, not files. The end goal, as noted in comments below, was basically DIY iSCSI failover. – Charles Oct 06 '15 at 21:56

7 Answers7

11

Yes, DRBD is the only replicated block device out there that can handle concurrent writes. If you plan to put a filesystem on top, it obviously needs to handle multiple writers as well, like GFS(2) and OCFS(2) do.

Please note that if you can afford higher levels of abstraction for redundancy, you'll likely be much, much happier with file-level semantics, so you should really think twice before going with block-level semantics. If you can't use higher levels of abstractions, but have money to throw at the problem, you can achieve significantly better performance with a good SAN.

But you probably know that already.

Pierre Carrier
  • 2,607
  • 17
  • 28
  • In this case, the thing I'm actually trying to achieve is some variety of warm failover for iSCSI targets to create a simplistic SAN. This is mostly a learning exercise. My distro of choice doesn't come with native DRBD support due to silly political decisions and running a slightly-too-old kernel. – Charles Jul 04 '12 at 20:06
  • DRBD will allow you to do active/standby and more recently active/active mirroring. If you chose active/active you'd have to ensure the filesystem supported it (hence GFS etc in the above). You'll probably want to use something like Heartbeat to trigger a failover (or just rely on an administrator doing 'drbdadm $resource up|down' as necessary). – David Goodwin Jan 02 '14 at 10:26
8

Well, there's also MARS (Light). According to the documentation this widely used at German provider 1&1

uli42
  • 195
  • 2
  • 9
  • Isn't it async only? "Synchronous or near-synchronous operating modes are planned for the future, but are expected to work reliably only over short distances (less than 50km), due to fundamental properties of distributed systems." <-- from MARS docs – BaronSamedi1958 Jun 13 '16 at 16:10
2

You could set up a RAID set using iSCSI devices but I'd be wary of doing this with asymmetric storage devices (which in the case of remote storage includes the network) - OTOH DRBD is explicitly designed to support such usage.

Is there a reason you don't like DRBD?

Have I failed in my research

If you think that GFS is a replicating cluster filesystem, then I'm afraid so.

symcbean
  • 19,931
  • 1
  • 29
  • 49
  • [It isn't](http://en.wikipedia.org/wiki/GFS2)? Okay, yeah, I suppose that *on it's own* it isn't. – Charles May 30 '12 at 17:02
1

I have heard about a variant of the network block device (NBD) which supports replication: ENBD. However, I don't know about the status of that project. The website doesn't look as if it was still supported, though.

Oliver
  • 5,883
  • 23
  • 32
  • Yikes, no updates since the 2.4 kernels? Still, good find. – Charles May 30 '12 at 08:20
  • 1
    Another solution would have been to export a file which is replicated using a cluster file system with NBD, but I don't think you would want to do that. No, DRBD is really the way to go! Using it for some years, never lost any data. – Oliver May 30 '12 at 09:33
0

That's an inaccurate statement. SIOS Technology www.us.sios.com has DataKeeper (https://us.sios.com/resources/product-brief-sios-datakeeper/) that does block level replication in Linux and Windows.

Edmond Melkomian
  • 1
  • 1
    This is why product recommendations are off-topic for ServerFault. You're answering an 8 year old question. Maybe the statement was accurate back then. But ServerFault isn't the right place to keep up with product developments. – Andrew Schulman Apr 16 '21 at 18:52
  • "Maybe the statement was accurate back then." -- It was, but it has been a very long time since I asked about this problem. While it's not the type of block level replication that I wanted, I've since settled on (Proxmox-provided) ZFS to solve my VM disk image replication needs. – Charles Apr 18 '21 at 16:05
0

There is one alternative: You could use SAN devices with native replication where the disk arrays's controllers do all the replication work themselves. It's quite expensive though, but doesn't need configuration on the hosts.

Sven
  • 97,248
  • 13
  • 177
  • 225
0

The question is wrong:

Is DRBD the only viable block replication solution for Linux? I've found myself needing redundant storage at a block level.

No, it is not. You have e.g. Linux MD (software RAID), LVM RAID. They provide redundancy for block devices.

So you probably wanted to ask:

Is DRBD the only viable NETWORK block replication solution for Linux?

And then again you have other options.

If the client is one node only you can build software RAID on the client which replicates over several network storages.

If you have multiple clients you can use clustered LVM block devices.

File-level replication (Gluster, GFS, etc) does not work for my use case.

GFS( Redhat's GFS2) is a shared device cluster file system. It does not provide redundancy. Other local filesystems like BTRFS and ZFS can do it though. As well as other distributed file systems.

Delian Krustev
  • 162
  • 3