The title says it all. When setting up free NAS, do I need 1GB per TB of usable storage, or 1GB of memory per TB of physical disc?
Asked
Active
Viewed 5.5k times
2 Answers
12
This topic is controversal, and still ongoing debate. I think the best answer is: "It depends". If you are going to need deduplication, you probably want to have huge amount of rams. There the 1 GB per TB rule seems to underestimate the empirical observation, that suggest more something like 5 GB RAM per TB storage (ZFS Tuning Guide).
Some ZFS developers seem to have a different view: Reddit-Comment 1 Reddit-Comment 2.
Quote from Comment 2:
That is wrong. 1GB is fine for ZFS. You can use the same amount for data deduplication, although writes will slow down from 3 random seeks being done on DDT misses after a certain amount of unique records have been stored. Each unique record takes 320 bytes of space in the DDT, it counts as metadata and ZFS ARC on ZoL will only allow 1/8 of RAM to be used for metadata by default. You can do the math.
As for the amount of RAM, 1GB or more. Performance tends to be better with more RAM for more cache though. As I said elsewhere, the amount of storage does not determine how much RAM you need.
Addendum:
Apparently it's the deduplication feature that requires lots of RAM See Oracle documentation. ZFS only support block-level deduplication, and if dedup is turned on, you will need approximately 320 Bytes per block per core. This makes the resulting RAM consumption a bit tricky. Conclusively you can only tell, that if you are not gonna need deduplication, ZFS will not require the often-stated 1 GB per TB rule.
TL;DR
1 GB RAM is fine, if you are not going to use deduplication (default is off)
grisu48
- 121
- 1
- 3
8
With ZFS, it's 1 GB per TB of actual disk (since you lose some to parity). See this post about how ZFS works for details. For example, if you have 16 TB in physical disks, you need 16 GB of RAM.
Depending on usage requirements, you need 8 GB minimum for ZFS. UFS requires much less.
-
One more question for you, if I build a FreeNAS system with 30-40TB of usable storage, and 64GB of memory, how much processor should I have? And would more cores or more single threaded performance make a difference? – Litzner Jan 23 '14 at 14:31
-
1+1 The normal dead minimum for ZFS (with any performance) is 4GB. Don't forget that the 1GB/1TB is over and above anything else the server might be doing (applications, services, whatever). – Chris S Jan 23 '14 at 14:38
-
@Litzner Less Faster or More Slower cores depends on if you'll be using Dedupe or Compression, and the speed of your drives. Dedupe and Compression are multi-threaded, so get better performance with more cores. All pool operations (storing/retrieving data) are single threaded. Most storage boxes will benefit from having a faster CPU, however normally the hard drives are much slower than the CPU so it's a non-issue. It depends on you setup... – Chris S Jan 23 '14 at 14:50
-
1I am looking at one FreeNAS server with 16 4TB drives, 15 of which in a striped zpool, (like RAID50), and 1 saved as a hot spare, there will be no compression, or deduplication on these devices, as that will already be being preformed by the BackupPC, and VEEAM servers backing up to them. I am looking at a second FreeNAS server with the same setup to duplicate the first, and in addition to that, have and additional 3SSDs, and 5HDDs, in a RAID10 with hot spares, that array will be used for a small VM backup datastore, incase I need to move somethings off the SAN in a emergency. – Litzner Jan 23 '14 at 14:52
-
@Litzner In that case I would go for a higher clock speed. Cores don't matter so much. – Nathan C Jan 23 '14 at 15:28
-
3is this relevant anymore? 16gb is good for 24TB and 32GB is good for 100TB, asuming RaidZ2. 12tb hard drives and RaidZ3 have really made this difficult to understand. I wish someone would explain why the 32gb for 100tb recommendation makes any sense. – Ray Foss Jan 31 '18 at 21:08