27

NFSv3 is widespread, but the default security model is... quaint. CIFS can use Kerberos authentication, but without POSIX semantics it's a non-starter. AFS never did encrypt traffic on the wire and is krb4 — and basically a dead project. Fancy new experimental filesystems either never materialize or are focused on speed (and if you're lucky, data reliability) — for example, Lustre uses the same client-trust model as NFSv3. For home use, sshfs is nifty, but that sure doesn't scale.

And then of course there's NFSv4, with sec=krb5p. Great in theory, but after ten years, it seems to be troublingly unused in the real world. The Linux client has just now had the experimental tag removed. And if you look at EMC Celerra, Isilon, etc., it's all NFSv3. (Celerra supports NFSv4, but it's really buried in the documentation. Isilon apparently worked at adding the RPCGSS support to FreeBSD, so maybe it's coming, but it's not there now. ) I can't even tag this post as "nfsv4" because I'm new here and that'd be a new tag.

So, really. What are you all doing?

mattdm
  • 6,550
  • 1
  • 25
  • 48
  • I'd like to say, "NFS3 over IPSEC", but I can't. – sysadmin1138 Oct 29 '10 at 19:32
  • 1
    "NFS3 over IPSEC" helps with the on-the-wire issue, but doesn't address another fundamental NFS problem: if a client box gets rooted, or if you're in a environment where users get root on their own systems, they can trivially impersonate any remote user. – mattdm Oct 29 '10 at 19:40
  • There you are, new tag ;) – Cry Havok Oct 29 '10 at 20:00
  • 1
    AFAIK, Kerberos was defined for NFS – Javier Oct 29 '10 at 20:15
  • 2
    I'm not so sure about the need for encrypting traffic on the wire in a LAN environment (authentication should be encrypted though). You should monitor for ARP poisoning anyway... – Hubert Kario Oct 29 '10 at 21:06
  • Hubert, we're in an academic environment, and university policy mandates that all traffic on the wire be encrypted. On more controlled server-room-only networks, I'm not so concerned. Perhaps what you're saying reflects common real-world use, though..... – mattdm Oct 30 '10 at 21:35
  • Celerra supports NFSv4.1 – Thiago Figueiro Oct 31 '10 at 00:10
  • Then change the policy ;). Seriously though, only server machines should be allowed to bind to NFSv3, for clients there's `sshfs`. – Hubert Kario Oct 31 '10 at 18:40
  • XCondE: is there documentation on that publicly at all? I'm not terribly surprised that they support it, but given that they don't seem to mention it ever at all, I'd be (pleasantly, really) surprised if it's much _used_. – mattdm Oct 31 '10 at 19:54
  • FTR: I've confirmed that Celerra supports krb5/krb5i/krb5p. However, it's very, very sparsely documented. (Giving support to the "no one is doing this stuff in the real world" theory.) – mattdm Nov 01 '10 at 20:39
  • Kerberos a dead project? i hardly think so. – The Unix Janitor Nov 05 '10 at 10:33
  • Kerberos is not a dead project -- I was talking about AFS. Ever since IBM cut it lose, OpenAFS is basically limping along. And even if it got developer support, it's hamstrung by the GPL-incompatible license (it can never be merged into the mainline kernel). – mattdm Nov 05 '10 at 13:54
  • you are right that openafs is limping along, but it *does* do krb5 authentication. It even does have the ability to do encryption over the wire but, sadly, that encryption is ancient DES-based. IIRC there is an unfunded project to add modern encryption support. Not easy, apparently, the code isn't as modular as you'd hope. – Dan Pritts Feb 27 '13 at 21:10

8 Answers8

14

You seem to be asking two questions here:

What are we actually using? and What does this?

What I'm actually using is CIFS, in my use-cases POSIX is less important so I haven't had any problems. NFS3 is used in areas where security isn't important, such as my SLES install server. And finally, sshfs/gvfs for simple user-land sharing. Wireline encryption is not deemed needed, so that isn't a meaningful factor for us.

As for the other question, there seems to be six main requirements for what you're looking for:

  1. Encrypts traffic on the wire.
  2. Encrypts authentication.
  3. Posix semantics.
  4. Strong enforcement of server-based ACLs.
  5. Is not userland.
  6. Is actually used.

I suspect points 5 and 6 will be the killers here, but here goes (also, this is the point where a table would be really handy, but markdown/StackExchange doesn't support it).

NFSv3 + IPSec

  1. Encrypted on the wire, pass
  2. No encrypted authentication, fail
  3. Posix semantics, pass
  4. No strong enforcement of server-based ACLs, fail
  5. Is not userland, pass
  6. Is actually used, pass

NFSv4 + Krb + IPSec

  1. Encrypted on the wire, pass
  2. Encrypted authentication, pass
  3. Posix semantics, pass
  4. Strong enforcement of server-based ACLs, pass
  5. Is not userland, pass
  6. Is not actually used, fail

CIFS

  1. Not encrypted on the wire, fail
  2. Encrypted authentication
  3. Posix semantics, pass (Samba & Kernel now, Windows has had a Posix layer since the NT days)
  4. Strong enforcement of server-based ACLs, pass
  5. Is not userland, pass
  6. Is actually used, pass

CIFS + IPSec

  1. Encrypted on the wire, pass
  2. Encrypted authentication
  3. Posix semantics, pass (Samba & Kernel now)
  4. Strong enforcement of server-based ACLs, pass
  5. Is not userland, pass
  6. Is not actually used, fail

SSHFS

  1. Encrypted on the wire, pass
  2. Encrypted authentication, pass
  3. Posix semantics, pass
  4. Strong enforcement of server-based ACLs, pass
  5. Is userland, fail
  6. Is actually used, pass

AFP/NetATalk

  1. Encrypted on the wire, fail
  2. Encrypted authentication, pass
  3. Posix semantics, pass
  4. Strong enforcement of server-based ACLs, pass
  5. Is not userland, pass
  6. Is actually used, fail

And I'm not touching the distributed file-systems out there. There simply isn't one single thing that does it all. Some come close (CIFS) and some are already there but no one uses them (NFS4 + IPSec, CIFS+IPSec). For some reason a secure network filesystem is something that has been subjected to a lot of compromises over the years.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • You could have mentioned "NFSv4 + Krb" and added "7. Is it reasonably fast (i. e. compared to the same protocol stack without encryption)?" as a question. Which would probably be a *fail* for NFSv4 + krb5p, but *pass* for questions 1-6. – al. Nov 02 '10 at 14:33
  • might be time for a new secure network filesystem SNFS? – The Unix Janitor Nov 05 '10 at 10:36
  • @user37899 The problem, as always, is convincing appliance vendors to support it, and organizations to deploy it. – sysadmin1138 Nov 05 '10 at 14:44
  • 1
    We've had really bad luck trying to use CIFS in POSIX mode. Maybe it's time to revisit that. – mattdm Nov 07 '10 at 13:20
  • FWIW I am using CIFS + IPsec, but not with POSIX semantics. server is emc celerra, clients win7. ipsec tunnels done in lan-to-lan mode between cisco ASA (next to celerra) and the win7 builtin ipsec. – Dan Pritts Feb 27 '13 at 21:13
8

Since it's a specific question (What are you all doing), let's answer it: nothing. Most administrators and users just don't worry about NFS security, so everybody uses NFSv3. It's typically a controlled environment (in the sense that only well-known machines can attach to the network in the first place). If somebody gets caught abusing the infrastructure, they get fired or to jail.

For data that you really don't want anybody to be able to read, you encrypt them explicitly, e.g. Firefox password databases, ssh keys, or pgp keys. You do that because you know the admin could read them on the file server, so network file system security would not be of any help, anyway.

Martin v. Löwis
  • 580
  • 4
  • 15
3

I've been using openafs in production for years, with both Linux and Windows clients. It works great, has an active development community, and has gotten much easier to install and administer over the last few years as the various linux distros have included packaging for it. It has its warts, but I've found that they are offset by more administrative flexibility, the ability to have clients and servers separated by slow links, the ease of offsite backups, and other positive AFSisms.

One thing I like in particular is running production internet-facing web servers on openafs, with the ACLs locked down. Without a kerberos ticket there is no process on the machine -- even one running as root -- that can write to the filesystem. I can't count how many times we've noticed attacks utterly fail because of that simple measure.

There are some pretty large openafs users -- the largest commercial user I know of is Morgan Stanley.

stevegt
  • 240
  • 1
  • 5
1

How about OpenAFS which is still alive and a VPN under it because its only encryption at the moment is DES.

Rob Olmos
  • 2,220
  • 1
  • 15
  • 25
  • 2
    I've used OpenAFS in production. Rumors of its aliveness are greatly exaggerated. – mattdm Jan 27 '11 at 03:32
  • There have been new release *this month* and before that there have been fairly regular updates to support new versions of Windows and new versions of Linux kernel (newest release supports 3.0). – Hubert Kario Sep 25 '11 at 13:16
1

I see that a lot of people in this thread are talking about data hiding, i.e. attacks not being able to snoop on your data. A equally important to think about data integrity and authenticity. Are those nfs packet really from your nfs server? Did a nfs packet get changed in transit?

The Unix Janitor
  • 2,388
  • 14
  • 13
1

Well, to me it sounds like one of those Distributed Filesystems would be for you. I wouldn't quite want to recommend OpenAFS, as it is old, doesn't support IPv6 yet,..

I'm quite happy with GlusterFS myself. Gluster is pretty mature, performs okay and has a good feature set. However, as recently discussed in IRC, Gluster does not support IPv6 in a stable manner either. This feature will be scheduled for 3.6 or 3.7.

There is also a project called HekaFS, which builds on Gluster, which adds more advanced Authentification features and SSL. It is imo extremely well documented and very well designed.

What may alos be of interest to you is XtreemFS, which is designed for global grid computing, so it comes with SSL and stuff by default. My choice for usage fell on Gluster though, since the community seems more active and it way better documented.

Both are posix compliant of course.

ramsch
  • 3
  • 2
juwi
  • 573
  • 5
  • 14
0

I use NFS. But server to server NFS is done over a dedicated network backbone so encryption is not needed and authentication is sort of pointless. Just set each export to only share a select directory to a server based on IP.

Porch
  • 680
  • 5
  • 12
-3

With all due respect, you are completely looking at this problem the wrong way and you should back away from the console for a few hours.

Pretty much all storage io is unencrypted because it doesn't matter at that layer of the abstraction stack. Doubt it? Put a tap on your brocade fibre switch and you will see that fibre channel,just like iscsi and nfs, is all an unencrypted mess - by design. Solving that is a medium problem, not a storage protocol issue. For instance, want secure and encrypted nfs? Created a lan that's encrypted point to point between the nfs client and server using ipsec/ssl/tls or a pure hardware solution.

Rafael Ferreira
  • 153
  • 2
  • I think you're missing a key point. As the question says, the problem is with the NFS security model. While encryption is nice, it is, as you say, a solvable problem. The big problem with NFS is that once a filesystem is mounted on a system, anyone with root access on that system can access any file on that filesystem, regardless of ownership or permissions. A system like AFS or, in theory, NFSv4 with sec=krbp5, requires strong credentials to access files, and so represents a substantial increase in security. A rooted NFS client does not equate a massive data exposure. – larsks Nov 01 '10 at 13:07
  • 1
    Unless you ask the user to enter credentials for each access, credentials will be stored. A root compromised client is likely to give up the stored key easily. Any networked file system will increase the file system exposure to compromise. – BillThor Nov 02 '10 at 14:18
  • @BillThor This is what I was thinking.. Is it still open to attack if the credentials are in kernel memory? I think a kernel module could be loaded up to read any piece of kernel memory. – Rob Olmos Nov 05 '10 at 17:01
  • As long as the request is used in the context of a user with access to the shared storage, it likely doesn't matter where the credentials are. Credentials are often held by a background process, so anyone who can communicate with it can likely gain access to the shared storage. I would rank the risk to secured network storage about the same as local storage. – BillThor Nov 05 '10 at 21:23
  • 3
    @BillThor: with kerberos, the risk is significantly mitigated, since the attacker would only have access to the filesystems of users who have forwarded their tickets, and only for the lifetime of those tickets. With system-based authentication (a la nfsv3), root can access and manipulate the files of *any* user, even if that user had nothing to do with the compromised system ever. – mattdm Jan 27 '11 at 03:30