Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [lustre]

Lustre is an Open Source, high performance, parallel, highly scalable distributed file system.

Lustre is distributed file system generally used for large-scale cluster computing. It is often used in supercomputers because of its high performance capabilities.

The name Lustre is derived from Linux + Cluster.

OpenSFS(http://opensfs.org) supports the continued evolution and success of the Lustre file system by ensuring that Lustre remains vendor-neutral, open, and free.

Official site for Lustre http://lustre.org/

23 questions
5
votes
0 answers

How does S3FS (or any other S3 FUSE filsystem) compare to AWS FSx for Lustre + S3

I remember trying s3fs a year back, trying to use some S3 bucket as a FUSE filesystem. I remember it being quite laggy, especially when coupled with git operations on it (an oblivious system architect had a weird requirement like this). Fast forward…
dimisjim
  • 215
  • 2
  • 10
4
votes
1 answer

What makes Lustre faster and more scalable than NFS?

I have read in various places (e.g. here and here) that NFS' I/O performance does not scale, while Lustre's does, and that Lustre can deliver better I/O rates in general. There seem to be various architectural differences between the two, but I…
Amelio Vazquez-Reina
  • 175
  • 1
  • 8
3
votes
1 answer

load balancing in multiple hardware raid arrays - soft raid 0 acceptable?

We have a central storage server (PowerEdge R720) serving shared files to an HPC cluster, and it has two hardware RAID controllers attached (PERC H810, each driving 2 MD1200 enclosures filled with 7200rpm 4TB disks). As with typical HPC workload,…
Carl Lei
  • 234
  • 1
  • 4
3
votes
0 answers

Are snapshots and clones filesystem-wide in ZFS-backed Lustre clusters

My goal is to find a distributed filesystem on Linux that supports ZFS-like lightweight snapshots and snapshot clones. This StackOverflow question expresses what I'm looking for pretty well. I'm trying to figure out whether Lustre + ZFS is the…
Anand
  • 31
  • 2
3
votes
1 answer

Which is the fastest way to move 1Petabyte from one storage to a new one?

First of all, thanks for reading, and sorry for asking something related to my job. I understand that this is something that I should solve by myself but as you will see its something a bit difficult. A small description: Now Storage => 1PB using…
Marc Riera
  • 1,587
  • 4
  • 21
  • 38
2
votes
0 answers

Lustre alternative for 70-node scientific cluster

Our lab has a cluster with 70 compute nodes 4 IO nodes Infiniband QDR interconnection 12T disk array accessed using IB SRP protocol The major application is debugging and running MPI-based parallel scientific program. The clients/compute-nodes…
Francium
  • 21
  • 3
2
votes
0 answers

What are the various factors that limits disk read and write speeds and on a network?

How to choose a server which: has high I/O performance of storage is highly flexible in terms of adding more storage So, basically these servers should act as storage servers for OST in lustre architecture. So, in the process of knowing the above,…
GP92
  • 599
  • 2
  • 6
  • 25
2
votes
1 answer

Beowulf cluster storage setup

I'm a Computer Engineering student working on a project with a Verari blade cluster, a bit outdated for today standards. I had acquired some Unix experience but I'm not an expert at all. This Verari cluster has 30 working blade nodes, 20 with two…
archector
  • 29
  • 1
2
votes
0 answers

Lustre glitch: latency of minutes

Using a HPC lustre filesystem, we occasionally experience glitchiness where even simply opening a terminal and typing "ls" can take minutes to return. That is, any process that involves the filesystem has random massive latency (but generally…
benjimin
  • 121
  • 2
1
vote
0 answers

Is it practical to keep home directories on OpenAFS?

I am a sysadmin by trade, and I do what I do at work at home as well for fun. I have a Gentoo Linux laptop, Raspberry Pis running Raspian, a Gentoo server, ARM devices running Debian and have various Android devices. I am always wrestling and…
Jesse Adelman
  • 968
  • 5
  • 15
1
vote
1 answer

Parallel filesystem which schedules simultaneous file requests to mutually exclusive sets of OSSs

My environment is RHEL based, interconnect is infiniband. I have some experience with Lustre. What i want to know is: Is there a parallel file-system, where if simultaneous write request arrive, they are scheduled on mutually exclusive sets of…
hrs
  • 151
  • 6
1
vote
1 answer

Lustre OST physical volume failure

What happens when the physical volume of an OST is irrecoverably lost? What data is lost? Only the data stored on this OST, or everything? And, if the first is true, how can one control what data is stored on which OST? Assuming that clients see a…
Pim Schellart
  • 131
  • 1
1
vote
0 answers

Disable atime update for specific files or folders

I hit an scenario where the update of the atime field in a Lustre file system from several simultaneous nodes performing reads on a single file is causing a severe performance loss. Is there any way to ask an arbitrary command not to update the…
Poshi
  • 111
  • 2
0
votes
1 answer

AWS FSx for lustre with S3 vs EMR (with EMRFS) for spark jobs

We are currently using EMR for easy job submission for our spark jobs. Recently I came across the "FSx lustre + S3" solution that is being advertised as ideal for HPC situations. EMRFS however is also said to be optimized for this particular…
dimisjim
  • 215
  • 2
  • 10
0
votes
1 answer

Lustre MDT full, rm: cannot remove No space left on device

I have a production server with lustre file system in it. It has 2 OSTs and one MDT. Now i my MDT is full. I can't delete my old files. Here a snapshot of my lustre file system. $ lfs df -i UUID Inodes IUsed IFree…
luthfi.imanal
  • 1
  • 2
1
2