1

I want to partition a CentOS 6.5 server that has 4 RAID VDs:

  • sdb - 102398 (100GB)
  • sdc - 2185727 (2TB)
  • sdd - 11440639 (11TB)
  • sde - 11440639 (11TB)

So that I can have the majority of the disk space as a single partition (/data). I want that large single partition to include /tmp, /var, /opt, and /usr/local. I want the / partition to include the rest of the operating system. The goal is to have the /data partition store volatile data (/tmp, /var, etc.) as well as user data and applications (/opt, /usr/local, /home, etc.) as either symbolic links (e.g., /tmp -> /data/tmp) or bind mounts (e.g., in /etc/fstab: /data/tmp /tmp none bind 0 0). I've considered partitioning such as:partitioning

The default CentOS partitioning during installation always includes /tmp in the / filesystem. Please note: I do not want /tmp, /var, etc. as separate filesystems. Given my RAID configuration and disk sizes, I'm assuming that the /data partition would need to be an LVM xfs partition, right?

Can anyone tell me how I can accomplish this? I've tried booting as single user and making bind mounts above, but rebooting failed with problems viewing /var. Perhaps the symbolic linking approach would work, but at this point I'm hoping for some expert advice!

Steve Amerige
  • 403
  • 2
  • 5
  • 11
  • Couldnt you just remount the directories? (http://www.cyberciti.biz/faq/how-to-mount-bind-partitions-filesystems-in-linux/) – Gekkie Jul 14 '14 at 12:11
  • 3
    Why do you want to do this thing? What **business** need does the change serve? – MadHatter Jul 14 '14 at 12:12
  • I don't think you _can_ relocate /tmp or /var in this way. I agree that you should talk about your real reasons for doing this, so that an alternative solution can be found. – Michael Hampton Jul 14 '14 at 12:25
  • @Gekkie I don't think remounting actually places the contents onto the other disk. It just makes it seem to be available there. – Steve Amerige Jul 14 '14 at 12:25
  • @MichaelHampton In our case the contents of /tmp and /var will be very sizeable (more than a few TB). We want to reduce any unnecessary partitioning as much as possible because that just wastes disk space for us. So, we want nearly all of our available disk space to be mounted as /data. We don't want to make the / filesystem have 6TB or so to give us data safety because that cuts into the 24TB that we have total. We really need to conserve available disk space for applications and volatile data that gets stored in /tmp and /var. – Steve Amerige Jul 14 '14 at 12:31
  • I've updated the partitioning image in the question. Note that the lv_root partition is a combination of sdb and sdc. In the RAID configuration, these are associated with 4 physical drives organized into a RAID 10 configuration to help ensure higher data safety for the operating system. The /data partition is just a RAID 0 configuration of physical drives. Despite the server's overall size, it is a compute server and the data is reloaded very frequently. If the server dies, that is okay. Of course, this note is just for background and is not relevant to the question. – Steve Amerige Jul 14 '14 at 12:39
  • But still you could (re)move /tmp /var etc and mount --bind them from /data/tmp -> /tmp etc? That way the `logical` /tmp is actually /data/tmp (and thus your big disk / raid array whatever) Right? – Gekkie Jul 14 '14 at 15:23

1 Answers1

2

Based on your comments I would suggest:

Use your hardware RAID controller to create a mirrored (RAID 1) volume for the operating system. This volume should be designated as your boot drive in your BIOS settings. With many raid controllers/drivers this volume will then appear as /dev/sda to your CentOS system.

If the remaining drives are also connected to your RAID controller you can use that create a JBOD volume (no redundancy and 1 drive dying will destroy all data) which will appear as /dev/sdb. Otherwise the other drives simply appear as individual drives in to CentOS. These will be /dev/sdb /dev/sdc etc.

Since you want to have a minimal of disk space assigned to the OS and everything else to your data volume I would partition /dev/sda as follows

/dev/sda 
    sda1 - 250 MB - ext3 - /boot  (I still find a separate boot partition useful)
    sda2 - 2-5 GB - ext4 - /      (this may also be a LVM physical volume for vg_root VG)
    sda3 - ???    - swap - (when using LVM merge with the vg_root Volume Group)     
    sda4 - all remaining space - physical volume for a vg_data Volume Group
/dev/sdb
    sdb1 - all space - physical volume for a vg_data Volume Group
/dev/sdc
    sdc1 - all space - physical volume for a vg_data Volume Group

vg_root 
   lv_root - 2-5 GB - ext4 - /
   lv_swap - ??? <swap>
vg_data
   lv_data - 20 TB? - ext4 - /data

Note that, as you specified, /data will die a merciful quick death when only one of the component disks dies.

The trick here is assigning the remaining space of the boot disk to your data volume group as well, giving you maximum space there.

Now if your applications and users do not honour $TEMPDIR settings simply move /tmp to /data/tmp and create a symbolic link instead ln -s /data/tmp /tmp. The same with anything else you want to relocate.

Best to do that in post install section of a kickstart script, or in single user mode after you have completed your installation.

N.B. I can see a point to moving /var/tmp to /data/tmp as well but not to moving the whole of /var there. By design /data is your most unreliable storage and you intend to allow users and applications to completely fill that space too...

HBruijn
  • 72,524
  • 21
  • 127
  • 192
  • In the end, I did the following configuration of the 24 physical drives: Disk Group 0, RAID 10 (4 drives): VD 0: BOOT 100GB; VD 1: ROOT 2134.5GB. Disk Group 1, RAID 0 (10 drives): VD 2: DATA1, 11172.5GB. Disk Group 2, RAID 0 (10 drives): VD 3: DATA2, 11172.5GB. It is possible that I might not have needed to have separate VD 0 BOOT and VD 1 ROOT Virtual Disks. I did it to ensure that the booting disk could do a standard (non-UEFI) boot. I used LVM later on so that I had / that exclusively used VD 0 and VD 1; and /data that used VD 2 and VD 3. I'm testing if symbolic links work now.... – Steve Amerige Jul 14 '14 at 19:38