13

I've been working with OpenSolaris and ZFS for 6 months, primarily on a Sun Fire x4540 and standard Dell and HP hardware. One downside to standard Perc and HP Smart Array controllers is that they do not have a true "passthrough" JBOD mode to present individual disks to ZFS.

One can configure multiple RAID 0 arrays and get them working in ZFS, but it impacts hotswap capabilities (thus requiring a reboot upon disk failure/replacement). I'm curious as to what SAS/SATA controllers are recommended for home-brewed ZFS storage solutions. In addition, what effect does battery-backed write cache (BBWC) have in ZFS storage?

ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • HP SmartArray controllers can quite easily work in 'pass-through' mode, in fact that's their default - just don't create any array sets. – Chopper3 Nov 12 '09 at 16:29
  • Really? So the drives would still be visible in Solaris if I don't configure any arrays/logical drives? – ewwhite Nov 12 '09 at 18:11
  • 1
    Battery-backed caches are tricky. Sort of controller dependent since ZFS forces a sync() after every write last I looked to ensure consistency, which can badly slow down some controllers which don't ignore the command. Of course, if you put your ZIL on a SSD or spare drive, this is not a problem. – phresus Dec 10 '09 at 14:24
  • To clarify, HP controllers are not capable of the SCSI passthrough needed for proper ZFS operation. – ewwhite Dec 24 '10 at 10:39

6 Answers6

10

I recommend a (almost) any SAS/RAID controller that uses the LSI 1068E SAS chip (PCI-E) and supports Initiator-Target mode (JBOD mode). This does not include MegaRAID controllers.

I use the a 3442E and Dell 6/iR.

  • LSI SAS 3442E (4 internal - SFF-8484, 4 external - SFF-8470)
  • LSI SAS 3081E (8 internal - SFF-8087)
  • LSI SAS 3041E (4 internal - SATA)
  • LSI SAS 3801E (8 external )
  • LSI SAS 9212-4i4e (4 internal - SATA, 4 external SFF-8088)
  • LSI SAS 9211-8i (8 internal - SFF-8087)

You get the idea.

Also I recommend Dell SAS Controllers which are cheap on ebay.

  • Dell SAS 5/iR - (4 internal ports)
  • Dell SAS 6/iR - (8 internal ports)
  • Dell H200

Not - Dell PERC 5/E - Dell PERC 5/I - Dell PERC 6/E - Dell PERC 6/I

Why not MegaRAID?

These controllers do not support a non-RAID, JBOD mode. They cannot present disks to the host directly. Each disk must first be included in a Logical disk. You can create a single logical volume for each disk you have and give this to ZFS. Performance should be equivalent. However, it means putting header information onto the physical disk. If your controller fails, you must use another MegaRAID, configured exactly the same. And there is not guarantee that another SAS or SATA controller will see the data correctly and present the volumes in a way that the zpool can be correctly constructed.

nicholasv
  • 116
  • 1
  • 2
  • 1
    I just converted to the LSI SAS 9211-4i and 8i to replace the P410 controllers in the HP ProLiant DL180 G6. It works wonderfully. – ewwhite Dec 24 '10 at 10:44
  • nicholasv, did you flash the Dell 6/iR with LSI firmware so you can present the disks as jbod to zfs? I am looking at buying a couple of these, but it seems the Dell 6/iR doesn't support jbod mode with the normal firmware. – Peter Valadez Apr 11 '12 at 19:44
  • @PeterValadez You don't actually have enough Rep on [SF] to post comments on just any Question. You need 50 Rep for that. Regardless, [SF] **is not** a forum; unless your directly answering the Question it's Off-Topic; and specific product recommendations are OT as well. – Chris S Apr 21 '12 at 02:01
7

The recommened controller is anything using the LSI SAS 1068E chipset. It's what Sun uses, so has the best support. As phresus mentions, the ASOC-USAS-L8i is a good cheap card that uses it. Battery-backed write cache helps performance, particularly for NFS. Another alternative is to use an SSD for the ZFS Intent Log, but only some have capacitors to ensure data is written after power-loss. For large read workloads, a MLC SSD can be used as L2ARC to supplement RAM caching.

TRS-80
  • 2,564
  • 17
  • 15
  • 1
    In Europe the easiest to get card is propably Intel sasuc8i, which is ~130€ in most German web stores. – Illotus Nov 20 '10 at 19:20
3

I would highly recommend you check out the Guide to Building a Media Storage Server thread at AVS Forum here. There is a wealth of knowledge in there comparing/contrasting various storage methods including ZFS. From the thread it seems like the Supermicro aoc-sat2-mv8 or newer aoc-saslp-mv8 are currently the most economical solutions for JBOD.

heydmj
  • 31
  • 1
  • 1
    I'm fairly sure the AOC-SASLP-MV8 isn't supported in Solaris. The AOC-USAS-L8i is recommended instead (albeit you don't get a mounting bracket, and it blocks the slot above it -- thanks, SuperMicro). – phresus Dec 10 '09 at 14:22
2

For 6Gb/s I have been testing the LSI 9211-4i and it works flawlessly.

If you need more ports or external JBOD connectivity, see the other 9xxx models.

Do not use a RAID controller for ZFS.

Giovanni Tirloni
  • 5,693
  • 3
  • 24
  • 49
2

Note that answer to this question changes with time (with new hardware and/or driver etc) - read the date of any post before making your order! :)

This blog post (Written in May 2010) is one of the most comprehensive listing I have seen which listed what works under Solaris, Linux and FreeBSD.

Lester Cheung
  • 659
  • 4
  • 11
0

You can use this Supermicro SATA JBOD card AOC-SAT2-MV8 to use ZFS on Solaris.

affanzbasalamah
  • 146
  • 2