Are there no downsides to NetApp SAN solutions other than price?

Question

We have pretty much decided on a NetApp solution for our first SAN. Given that, I've been tasked with finding as much reason not to go with NetApp as I can. We like to do this A) so we know what we're getting into and B) so we aren't clouded by the inevitable post vendor demo euphoria. I have scoured the internet for cons and can only find one: price.

Have you had a nightmare experience with NetApp that you just want to get off your chest? Please, only people with NetApp experience.

Thank you!

Answer 1

We have had some ups and downs. Including finding our first ONTAP bug before we'd deployed and producing some certain benchmark conditions that weren't quite what we expected.

But that probably makes it sound worse than it really is. In service it has functioned very well and I wouldn't hesitate recommending them to someone else.

Just a few pointers:

• Assume circa 60% capacity from your disks after right sizing and reserves.
• Simulate real-use benchmarking on a demo filer of the same specs that you intend to purchase.
• Check the max aggregate size for your chosen filer. They vary depending on H/W specs.
• Check the max volume size for ASIS if you intend to use de-dupe. Again it varies.
• Be prepared to use a CLI. Unless perhaps the new Windows NSM is your thing.
• Ensure that you've specced all of the licensed features that you require.
• Bargain hard ;)

Answer 2

My thoughts:

• they are expensive. Find a reseller you trust, then work to buy at quarter- (or even better, year-) end.
• licenses will nickel-and-dime you (or rather, kick you in the junk for multiple-thousand-$) each time. Much of the nifty functionality is an extra-cost license.
• support is expensive, especially for EOL gear. However if you set it up right, most of the time the first indication of a problem with the unit will be when a replacement part shows up at the front desk.
• when you buy more capacity, buy it by the fully-populated shelf. (See above for quarter- and year-end.) Buying disks for a partially-populated shelf is UNBELIEVABLY expensive -- and NetApp will be very unlikely to drop price on them when you are buying one or two at a time.
• between disk right-sizing, spares, and parity, your effective space yield is disapointing compared to your "raw". Example: we have a 2020 with 12x500GB disk, but with right-sizing (a 500GB disk is right-sized down to 420GB), losing 1 disk as a spare and two disks as parity, 6TB raw turns into a fraction over 3TB yield. Make sure whomever is signing the PO understands this.
• NetApp does say that they don't have the ability to guarantee read throughput (this is because once enough pending writes are stored up, the filer interrupts all read processes to go do the writes right now), so processes which are sensitive to smooth data read flow (ie realtime video) might hiccup; for most purposes they are just fine.
• Backup of NAS (nfs/cifs) files is done through ndmp, which is usually an extra-cost option for your backup system. (You are planning to back this sucker up, right?)
• Mixed-mode filesystems (ie those which are shared through both NFS and CIFS) suck. Even though it is really really tempting to use mixed-mode, don't. Pick one or the other.
• Even though it is really tempting to over-commit volume space, don't. Over-committing means that when one volume fills, they all do. Between that and the usually unexpected behavior of snapshots (meaning that deleted files are not actually freeing up space) it can lead to all kinds of hilarity when the filer is full.
• Educate your user community about snapshots, that A) they can go back in time and dig things out themselves, and B) space-wise snapshots can be considered "free" so they shouldn't try to go into the .snapshot directories to "clean things out" themselves. This will take multiple tries because it's a tricky concept that some users just don't get.

All that said, we really like them. We pitch NetApps to all of the customers we think can afford them, and encourage those which can't to strongly consider it as a goal. For the most part they just work. Support is almost totally hands off, especially with the Autosupport turned on -- disks just show up when needed. The guys on the phone really know their stuff.

Answer 3

Your comments show that you know about the post-sales blues can go along with any sizable purchase so I think your approach of trying to find the warts now is a good one.

• Dan C made good points indeed. Right-sizing and the various reserves can feel like a slap in the face at first. It can be hard for folks to over the feeling that "that's not as many juicy terabytes as the salesrep said!" Here's a Capacity Calculator specific to Netapp that might be useful: http://nicholasbernstein.com/calc/ I believe it takes the right-size into account. A good sales-rep should be willing to be straight-up about all of this and take the time to make sure there's no huge after-sale shock.

• Also agree++ about bargain hard...times are tough times for buyers and sellers alike and new customers are hard to come by. This goes along with Dan C's comment about making sure you've got the licenses (and right-sized, usable capacity) spec'd that you anticipate needing before you are scheduled to grow the system. It is usually much more cost-effective to purchase it all at once rather than add a license a few months later and a shelf a couple months after that.

• Be aware of the limits of the unit you purchase and match it against your anticipated growth and what you expect in terms of usable lifespan. In addition to aggregate size, be aware of the max number of disks and the shelf limit so you can plan expansions wisely when the time comes. (These factors are key in storage planning. Apologies if they come off like "duh!" but if this is the first SAN in the environment and everybody is used to dealing with lots of servers and direct-attached storage, it's possible nobody has strategically pulled this kind of info together.)

• If you plan NAS usage, a big plus is having one pool of centrally & flexibly managed resource for shares and SAN as opposed to a SAN and sep. fileservers or a SAN dishing out LUNs mounted by fileservers.

My experience:

No nightmares but some learning curves with sizing and capacity and finding our comfort spot with "thin provisioning."

Overall, we've been pleased with flexibility and performance. Even more so with 7.3.x ONTAP that we went to a few months ago.

We did have a noticeable performance issue related to a deduplication-related bug that has been corrected.

Deduplication savings is a great value in our environment. Depending on your environment the savings can go a long way towards quieting the heartburn over right-sizing and usable capacity.

For NAS, filer-native quota management may not be quite what folks are expecting for "slick" management of shares. There are some feature-rich products with full Netapp integration available for purchase, however.

Answer 4

Operationally our little FAS270c has been pretty good for us, running iSCSI. The point-and-click software for taking snapshot backup's from SQL Server, Exchange, Sharepoint and VMWare and being able to restore back GB's of data in under 20 seconds is great stuff and probably one of NetApp's strongest points.

But the point where we've had to shelve it is when it's 3 year warranty ran out and NetApp wanted around £20k to support it for another 3 years (partially because the FAS270c has gone end-of-life). It was cheaper to bin it and buy a newer model (FAS2020) then to renew the warranty on our existing box.

So we had a look round at upgrading to a FAS3100 series or a FAS2050 and were offered heavily discounted prices, but in the end we got sick of the way that NetApp seemingly plucked prices out of the air every time we talked to them. If we ever wanted to buy another unit for our DR site, or extra licenses or disk shelves we expected to be hit very hard in the wallet.

So we've swapped over now to an HP LeftHand iSCSI box; we'd be able to put a Lefthand in our datacenter AND in our Disaster Recovery site replicating between the two for the cost of a single NetApp FAS3140. Hmmmm... Course if we kept talking to NetApp long enough they probably could have matched the price eventually but some of us have work to do.

Answer 5

(I do have NetApp experience, but it doesn't influence my answer here.)

You may choose to consider their patent threat against the open source ZFS file system as a potential reason to choose not to support the company.

(Or, if you're on their side, you could use it as a reason to support them!)

Answer 6

Some things to keep in mind about NetApp SAN. It's not true SAN first of all. It's SAN, on top of the waffel (sp?) file system which is awesome for NAS, but isn't go great for SAN, especially with another layer between them.

Because waffel is between the platform and the fibre ports if you blast the WAFL with data the FC can then slow down while it waits for the waffel to catch back up.

You also don't get any control over RAID level (unless this has changed recently). So if you need RAID 5 for some data, because it's all read, and some data needs to be RAID 10 because it's all write and very little read you can't control this.

Now don't get me wrong, NetApp makes an amazing NAS unit. But you can't take a NAS, and slap FC ports in the back of it and called it a SAN.

Now, I know that the units can be made redundant, but I believe that requires the purchase of an additional filer head (dual heads are standard is most all SAN setups in case of reboot, etc) plus the additional storage for that second filer (as I don't think that the two heads can talk to the same disks).

Take a look at this blog from Chuck Hollis and the Prove It Kit he published

I know that there used to be an issue with battery backups on the NetApp SANs. If you fired up SQL Server and created a table and started inserting data into it, then pull the plug on the NetApp (simulate power failure) then query the cache to get the last value inserted you'll get a different number from the database after the NetApp comes back up because some transactions where lost (this is an old issue and is hopefully solved now).

The NetApps don't allow you to control the amount of Read Cache or Write Cache. It's 50/50. If you are going to be hosting databases on that read cache is basically worthless, and write cache is king. Normally you'll want to disable the read cache for the SQL Server's LUNs and up the write cache. Not an option here.

Answer 7

In an effort not to duplicate others :)

• Deduplication is awesome. Plan your FlexVols carefully, as the unit for dedup is the FlexVol.
• Make sure you understand the following terms (and the differences between them): snapshot reserve, fractional reserve, volume guarantee, file guarantee. If you don't, you will make mistakes in your initial implementation and/or allocate way too much storage for your purposes. trust me on this one :)

NetApp does a lot of magic with their storage, but it makes architecting a solution that goes on top of NetApp a fairly complex endeavor.

Oh yes, find out who your resources are for setup/configuration/best practices support. NetApp tech support is very reticent to work on best practices cases as they consider it out of break/fix scope. If you need to know best practices, getting them from netapp support is not the way to do it.

Answer 8

We just purchased one to replace some OnStor Bobcat filers. We have no complaints thus far, with two months in. Quite to the contrary, they're very robust. The NDMP backup speeds are amazing. The management thus far has been easy, and everything has worked the way we expected.

Things to remember about this sort of solution, your antivirus software has to be hosted on another box for scanning the files on the NetApp (this matters with public storage at a University, it might matter less for you). Quota managment is by user per host(once again this may not matter to you).

I don't know of anything we've found we didn't like yet.

Answer 9

Where to beggin. Support is very very poor quality. Talk to 3 techs, get 3 differnt answers. Usually all of them wrong. Very poor read performance (good luck getting within latency tolerances for database systems). Read performance declines drastically over time. If it's not bad enough for you when you first set it up, just wait awhile. Wafl (write ANYHERE file system) spews data all over the place, fragmenting your database files beyond any reasonable limit. Netapp actually expects customers to defrag at host level AND volume level frquently (they told us EVERY DAY). Unfortunately the process is intrusive and completely ineffective. We switched from HP to Netapp. Big mistake. It has been the absolute worst experience of my 16 year IT career. The only time we had to spend on the HP was when we updated firmware or added capacity. Everything worked as advertised. With Netapp nothing works as advertised, and we spend endless hours trying to get things to work and troubleshooting the endless string of problems. But we can use CIFS now. Oboy.

Netapp is a decent NAS, but if you need a SAN, buy a SAN.

Answer 10

No nightmares but be aware that all the software cleverness does mean that overall system performance does vary - if you need your SAN to perform consistently at all times of the day then you may be disappointed. Otherwise they're great, oh and they're dropping their trousers on discounts at the moment :)

Answer 11

I have never heard anyone complaining about performance on their NetApp. I've never been able to afford one to back up that claim, though ;-)

Answer 12

I know that you said only people with experience, buuut,

There is a hosting company called joyent and while i was looking into them as a hosting provider, for problems etc, the only thing that came up as an issue with them was some serious slow downs they had a little while ago, and from what i can gather it was from the Netapp filers. They've now switched to thumpers and zfs, because they are a Sun shop. And this apparently fixed their problem. And this was a few years ago from what i remember. But it might be worth to check up.

Hope this helps.