Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-filesystems]

124 questions
5
votes
2 answers

what distributed file system for a two-node failover setup?

I'm trying to set up a redundant setup consisting of two servers that have everything redundant: the database (MySQL master-master in active/passive mode) the file system (distributed/replicated) our application software (kept in sync using the…
Udo G
  • 423
  • 4
  • 9
  • 19
5
votes
1 answer

deduplicating and indexing directories of images across 150 linux machines

I have a client with 150 Linux servers spread about various cloud services and physical data-centres. Much of this infrastructure is acquired projects/teams and pre-existing servers/installs. The client is largely about image processing, and many of…
Tom
  • 10,886
  • 5
  • 39
  • 62
5
votes
2 answers

HW/SW Design: 2 Petabyte of storage

Disclaimer Yes I'm asking you to design a system for me :) I'm tasked to design a system to store about 10 TB / day with a retention time of 180 days. My first approach would be to go with GlusterFS and use a HW setup like this: Single Node in the…
Martin M.
  • 6,428
  • 2
  • 24
  • 42
5
votes
1 answer

Ceph: Why is a greater number of "placement groups" a "bad thing"?

I have been researching distributed databases and file systems, and while I was originally mostly interested in Hadoop/HBase because I'm a Java programmer, I found this very interesting document about Ceph, which as a major plus point, is now…
monster
  • 608
  • 2
  • 10
  • 17
5
votes
3 answers

Are we using DFS "wrong"?

We are a company that has branches across the country. We have a minimum of 1 T1 to each branch and a maxiumum of 2 T1s. We have a DFS server at each branch and at our main office. In the past week one particularly troublesome share that has some of…
Jason
  • 251
  • 4
  • 7
5
votes
1 answer

Can you create a glusterfs with existing data in a directory?

I'm looking into convert a single server/comp into the start of a glusterfs distributed system. I already have a directory mounted on this server of 24TB RAID. I want to use this initial computer to setup glusterfs with this volume/directory and…
Joey
  • 151
  • 1
  • 4
5
votes
3 answers

What differences are between a NAS, a shared disk file system on a SAN, and a distributed filesystem?

https://en.wikipedia.org/wiki/Clustered_file_system#Network-attached_storage says Network-attached storage (NAS) provides both storage and a file system, like a shared disk file system on top of a storage area network (SAN). NAS typically uses…
Tim
  • 1,467
  • 3
  • 25
  • 38
4
votes
5 answers

Setting up distributed fault tolerant storage at home

I am tired of worrying about data loss at home. My wife is a semi pro photographer, and essentially all of our family memories are digital (and we ought to convert the ones that are not). I am planning on setting up two systems to host the disks,…
Ronald Pottol
  • 1,683
  • 1
  • 11
  • 19
4
votes
2 answers

GlusterFS alternative for file upload website

I have a few file upload websites, with files ranging from hundreds of kilobytes to a few gigabytes. Currently I have all files in a distribute-replicate Gluster volume on a few servers. My biggest problem with Gluster is speed. For example I…
Alex Dumitru
  • 315
  • 2
  • 3
  • 8
4
votes
1 answer

Distributed mirrored filesystem under FreeBSD

Can someone share their experience in building a distributed mirrored filesystem between multiple FreeBSD machines? I. e. we have two (three, four...) servers and special partition "part1" mounted on each of them. We make some changes on it on the…
Mikhail Efremov
  • 41
  • 3
4
votes
1 answer

In a distributed filesystem like MooseFS or XtreemFS, should individual nodes expose "raw-er" storage, or LVM'd storage?

When preparing an infrastructure to utilize a distributed storage system like MooseFS or XtreemFS, how should individual nodes present storage to the rest of the environment? Is it better for partitions to be presented close to the physical…
warren
  • 17,829
  • 23
  • 82
  • 134
4
votes
2 answers

Distributed storage

At my University Department we are about to upgrade the computers of our student lab (about 25-30 machines). The machines will be running Linux. One thing about the new machines is that they have huge (1TB) hard disks (we did not ask for them, but…
nplatis
  • 141
  • 1
4
votes
4 answers

What should be done with excess computing resources after converting to thin clients?

I work part time for a small private school. The 24 node computer lab kept having hardware failures (mostly drives and cooling fans) so I turned it into a Linux based thin client network. Although the workstations now boot from the network, most…
Kenneth Cochran
  • 195
  • 5
4
votes
2 answers

Distributed filesystem across a slow link

I have an image in my head where a link is too slow to realize the real-time transfer of files, but fast enough to catch up every day. What I'd like to see is a master <-> master setup where when I write a file to Server A, the metadata will…
Jeff Ferland
  • 20,239
  • 2
  • 61
  • 85
4
votes
2 answers

linux distributed file system over WAN advice

I have a fairly simple (not really) requirement but I've looked at a few solutions and can't find a good solution. I've got a Red Hat EL 6 server environment in my co-location and office, and some linux and windows xp and win7 workstations my…
dmansfield
  • 161
  • 1
  • 4
1
2
3
…
8 9