Questions tagged [distributed-filesystems]
124 questions
37
votes
1 answer
Why doesn't SSHFS let me look into a mounted directory?
I use SSHFS to mount a directory on a remote server. There is a user xxx on client and server. UID and GID are identical on both boxes.
I use
sshfs -o kernel_cache -o auto_cache -o reconnect -o compression=no \
-o cache_timeout=600 -o…
Jan Deinhard
- 2,363
- 5
- 26
- 33
20
votes
4 answers
Are ZFS clustered filesystems possible?
Is it possible to create a ZFS cluster? Or would you need to go the seemingly ugly (to me at least) route of ZFS with UFS formatted pools governed by GlusterFS?
The idea is to see if it is possible to expand to Multi-Petabyte distributed store which…
SteveMustafa
- 419
- 1
- 5
- 11
18
votes
3 answers
GlusterFS vs Ceph, which is better for production use for the moment?
I am evaluating GlusterFS and Ceph, seems Gluster is FUSE based which means it may be not as fast as Ceph. But looks like Gluster got a very friendly control panel and is ease to use.
Ceph was merged into linux kernel a few days ago and this…
Mickey Shine
- 929
- 4
- 16
- 33
14
votes
1 answer
How does NFS read cache work on Debian?
I am planning to use NFS to serve out many small files. They will be read very often so client side caching is crucial. Does NFS handle this? Is there a way to increase the client side caching in some way?
...or should I look at another solution?…
Ztyx
- 1,365
- 3
- 13
- 27
11
votes
9 answers
Geographically distributed file system with preferred locality
I'm building a application that needs to distribute a standard file server across a few sites over a WAN. Basically, each site needs to write a lot of misc files of varying size (some in the 100s MB range, but most small), and the application is…
dpb
- 445
- 5
- 16
10
votes
5 answers
Is there a Distributed SAN/Storage System out there?
Like many other places, we ask our users not to save files to their local machines. Instead, we encourage that they be put on a file server so that others (with appropriate permissions) can use them and that the files are backed up properly.
The…
Joel Coel
- 12,910
- 13
- 61
- 99
7
votes
5 answers
Experience with MooseFS?
Anyone have any experience using MooseFS? I want an easy distributed storage platform to store static data archive of about 10 TB and serve it to 20-40 nodes. Also I want to be able to add storage as the archive grows without having to rebuild the…
brown.2179
- 171
- 1
- 4
7
votes
4 answers
Distributed file systems
I need to implement a distributed storage system for a set of nodes(devices) connected in a mesh network.
So what basically my design goals are:
The storage system should be capable of handling dynamic entry and exit of nodes.
Replication (for…
sud03r
- 191
- 3
6
votes
1 answer
GlusterFS Transport endpoint not connected from time to time
I'm using GlusterFS 3.7.9, currently on a single server with 4 bricks.
Each brick has 4TB and the volume is set up as distribute only.
The volume is mounted on a secondary server and I use it for backup purposes.
The problem is that once every…
Alex Dumitru
- 315
- 2
- 3
- 8
6
votes
2 answers
What is reasonable storage failover time that most OS (VM) can tolerate?
I have a GlusterFS 2 node 2 replica setup. I am planning to use it as OpenStack instance store, in which the VM disk image is stored.
From my tests, if the GlusterFS node which the hypervisor currently mounts on fails, (using default GlusterFS…
Pellaeon
- 953
- 1
- 7
- 7
6
votes
0 answers
Fast distributed filesystem for a large amounts of data with metadata in database
My project uses several processing machines and one storage machine. Currently storage organized with a MSSQL filetable shared folder. Every file in storage have some metadata in database.
Processing machines executes tasks for which they needed…
Vasilly.Prokopyev
- 173
- 6
6
votes
9 answers
Distributed file system with local disk cache
My server infrastructure is growing fast and I decided to create a distributed storage cluster. I've been looking for a proper filesystem for this task which meet my requirement, but none of them supports a local disk cache functionality. Each of my…
Galmi
- 111
- 1
- 7
5
votes
0 answers
How does S3FS (or any other S3 FUSE filsystem) compare to AWS FSx for Lustre + S3
I remember trying s3fs a year back, trying to use some S3 bucket as a FUSE filesystem. I remember it being quite laggy, especially when coupled with git operations on it (an oblivious system architect had a weird requirement like this).
Fast forward…
dimisjim
- 215
- 2
- 10
5
votes
2 answers
Is there a way to connect a remote folder with local cache on OSX or Windows workstation?
A company with 50 graphic designers are working on the same file server. They are using mainly Indesign.
A typical project is a 60 pages indesign document with 1.5GB of linked files (something like 100 PSD, JPG, .ai). They continuously edit linked…
bokan
- 225
- 3
- 13
5
votes
1 answer
Why is GlusterFS so slow here?
We've set up a mirroring pair of GlusterFS servers. No special tuning, whatever came "out of the box" with GlusterFS-3.5.1 in the official RHEL6 RPM, that's what we have.
The cluster works, but the performance is pretty awful. For example,…
Mikhail T.
- 2,272
- 1
- 22
- 49