4

My project is expanding 24-HDD ZFS box by adding 45-drive box with option to add another 45-drive box later (all are JBODs).

Host box (24 HDDs) is Supermicro with single port expander backplane and LSI RAID card (I think, MegaRAID SAS 9240-8i). Currently backplane occupies one SAS port on card, leaving 1 port open.

My current understanding is that on first level, RAID card has to support number of drives I want to have via expanders. So I need to change RAID card to something supporting 128-drives.

After that I should be able to expand by daisy-chaining: Host backplane expander -> host RAID card <- extension 1, backplane 1 (24 HDDs) <- extension 1, backplane 2 (21 HDDs) <- extension 2, backplane 1 (24 HDDs) <- extension 2, backplane 2 (21 HDDs)

My RAID setup is volumes of 4 or 5 vdevs, RAIDZ2 (RAID6), 4TB SAS drives

Questions: Will daisy-chaining of 2 or 4 backplanes preserve 6Gb/s speed? Is the only limitation is number of supported disks by RAID card?

Also, as I understand, use of dual-expander backplanes allows redundancy by connecting backplanes by daisy-chaining via additional paths. Is that true and is it worth extra 200$?

Should I throw away RAID card and use HBA instead since it will allow support for 128 drives cheaper and all I care is JBOD?

I used these sources to get understanding:

Community
  • 1
aaaaa says reinstate Monica
  • 500
  • 4
  • 16
  • 1
    Designing this sort of thing and doing it well requires some effort. And lots of tuning. I can give you some advice, but there's not enough information to tell you exactly how to accomplish al of this. What OS are you using? – ewwhite Dec 28 '15 at 02:24
  • 2
    @ewwhite I'll appreciate any advice! I am running FreeBSD 9.x. Original box has 24HDDs sitting on single-expander backplane, connected via 1 SAS2 cable – aaaaa says reinstate Monica Dec 28 '15 at 03:50

1 Answers1

2

Many questions here!

Will daisy-chaining of 2 or 4 backplanes preserve 6Gb/s speed? Is the only limitation is number of supported disks by RAID card?

The Supermicro 45 slots SC847Exx JBOD has one or two expanders on the front backplane, and one or two on the back. Cascading the front and back backplane is OK, however I'd strongly advise against cascading JBODs.

In your situation I'd use the first port to drive the internal backplane, the second port to drive the first JBOD, and I'd add another controller to drive the second JBOD and to avoid driving too many disks from the same port, which would severely limit performance.

Also, as I understand, use of dual-expander backplanes allows redundancy by connecting backplanes by daisy-chaining via additional paths. Is that true and is it worth extra 200$?

No, dual expander backplanes allow multipathing with SAS drives only. If you're using SATA drives, dual expanders are useless (the second expander simply won't detect the drives at all).

Should I throw away RAID card and use HBA instead since it will allow support for 128 drives cheaper and all I care is JBOD?

A RAID controller provides a battery-backed cache which tremendously enhance performance, a HBA doesn't. Better use a RAID controller and set up the individual disks as distinct volumes, to get the additional cache performance.

wazoox
  • 6,782
  • 4
  • 30
  • 62