5

I have Dell EMC SCv3020 Storage Array and I want to create shared storage for all 3 physical servers to use the same (LVM?) for creating KVM VMs in cluster so if 1 server dies, we can move VM to another one. We have solution for HA failover but we are not sure how to do this shared storage setup.

Should we setup simple NFS on virtual machines and share? If yes, which filesystem will be on NFS, and should we make LVM on it? Does filesystem need to be cluster aware so there is no data corruption because all servers are accessing same storage?

We also have VMware on this infrastructure using Dell storage array and production VMs on it so is it possible maybe to use VMFS Vmware datastore to create Virtual machines on which we will setup this shared storage on all 3 new physical servers for deploying KVM VMs all servers on same shared storage?

Mr green
  • 51
  • 2

3 Answers3

5

I have Dell EMC SCv3020 Storage Array and I want to create shared storage for all 3 physical servers to use the same (LVM?) for creating KVM VMs in cluster so if 1 server dies, we can move VM to another one. We have solution for HA failover but we are not sure how to do this shared storage setup.

What you have is a classic so-called dual-controller SAN, depending on your connectivity (Fibre Channel Vs iSCSI) you simply create a new virtual LUN from your SAN disk pool, and expose this virtual LUN to your storage consumers. If you'll use VMware it will format it with VMFS which is cluster-aware file system and will use this new datastore for VMs.

This is some great guide you should follow:

https://www.delltechnologies.com/asset/en-us/products/storage/industry-market/h16391-dellemc-unity-storage-vmware-vsphere.pdf

If you'll use just some servers w/out hypervisor you'll have to use some cluster-aware file system or you'll trash the metadata on your virtual LUN with some non-arbitrated writes. See:

https://forums.starwindsoftware.com/viewtopic.php?f=5&t=1392

Should we setup simple NFS on virtual machines and share?

You can use NFS for VMware datastore (so with KVM but not with Hyper-V), but you'll need some level of HA here as you don't want your VM datastore to become a single point of failure (SPOF).

If yes, which filesystem will be on NFS, and should we make LVM on it? Does filesystem need to be cluster aware so there is no data corruption because all servers are accessing same storage?

See above. In case you'll use VMware it will be VMFS on top of your virtual LUN and there will be VM datastore and you can expose any file system from inside of your production VMs.

We also have VMware on this infrastructure using Dell storage array and production VMs on it so is it possible maybe to use VMFS Vmware datastore to create Virtual machines on which we will setup this shared storage on all 3 new physical servers for deploying KVM VMs all servers on same shared storage?

This is a way to go actually.

BaronSamedi1958
  • 12,510
  • 1
  • 20
  • 46
4

Dell storage array can be connected to your hosts via iSCSI or FC. That's what you can start from. As mentioned, you will need cluster-aware filesystem to be able to use this storage simultaneously on three hosts. cLVM is one of the best options to use. However, it must be mentioned that having a single SAN creates a single point of failure for your infrastructure. Implementing software-defined storage, which is redundant ca solve this issue. Couple examples:

https://www.starwindsoftware.com/starwind-virtual-san

KVM+DRBD replicated between two active-passive servers with manual switching

Stuka
  • 5,181
  • 13
  • 11
0

The description is not comprehensive; this answer should serve a role of introduction into HA clustered storage, which is what the question essentially about.


Traditionally this problem is solved using dedicated SAN, which is external to the virtualisation cluster. The SAN is made reliable internally. It has two coordinated controllers, all storage media are dual ported (SAS), and their ports are connected one to first storage controller and other to the second controller. Disks are assembled into RAID arrays. To access those controllers, each host has two SAN NICs, one is connected to one controller or SAN switch, other is connected to other one, and if switches are used, each of them has a link to each controller, so in total each host has 2 or 4 different paths to each RAID array. In result, we get a storage system which has no single point of failure. The access to this SAN could be provided with several protocols, including SAS, Fibre Channel, FCoE, iSCSI; also some storage provides an access as a NAS via pNFS.

All major virtualisation cluster solutions support such SAN. In general, SAN volumes are seen as block devices by each host, and there is some arbitration with certain guarantees between nodes. Then those block devices are used to host cluster-aware file systems to store VM virtual disks. For example, in Linux cLVM (cluster-aware version of LVM) could be used; in Hyper-V clusters the NTFS is made cluster-aware.


The industry now trends to converge networks, i.e. the same NICs are used for the SAN and for the LAN, and distinction is made by introducing virtual subinterfaces, e.g. VLAN. This creates an appeal also to converge services further and spread all of the highly available SAN duty to the same hosts as those running virtual machines, effectively creating a virtual networked SAN. This is considered a hyperconvergent solution (a "software-defined storage"). I know there are vSAN solutions for VMWare, but I didn't tried those. The Linux vSAN solutions include CEPH and DRBD; both require at least three hosts (for quorum) to be truly redundant and also a plenty of network capacity (they require redundant 10G network). It is possible to only have three powerful servers with two 10G interfaces each and that's all you need to have a redundant HA virtualisation cluster; Proxmox VE (which is Linux KVM-based virtualisation cluster with built-in CEPH support) has videos on their YouTube channel explaining how to achieve this setup. However, nobody recommends to run such setup in production for various reasons and even if one chooses to employ hyperconvergent storage, it is usually recommended to dedicate some hosts to host a vSAN and others to run virtual machines.

Nikita Kipriyanov
  • 8,033
  • 1
  • 21
  • 39