Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [ceph]

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster.

Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available.

156 questions
18
votes
2 answers

Cause of page fragmentation on "large" server with xfs, 20 disks and Ceph

Any insight from someone with a bit of experience in the linux IO system would be helpful. Here is my story: Recently brought up a cluster of six Dell PowerEdge rx720xds to serve files via Ceph. These machines have 24 cores over two sockets with two…
pingu
  • 181
  • 1
  • 6
14
votes
1 answer

What is the difference between Ceph and NFS when it comes to shared file storage?

Can someone explain to me what is Ceph compared to NFS? From a shared file storage perspective? What is the added value of using Ceph instead of NFS?
Basil A
  • 1,910
  • 2
  • 17
  • 18
9
votes
3 answers

Is Ceph possible to handle hardware RAID arrays (LUNs) as OSD drives?

I am pretty new to Ceph and try to find out if Ceph supports hardware level raid HBAs. Sadly could not find any information. What I found is, that it is recommended to use plain disks for OSD. But this pushes the requirements to the PCIe, the…
cilap
  • 277
  • 5
  • 14
8
votes
2 answers

CEPH's raw space usage

I can't understand ceph raw space usage. I have 14 HDD (14 OSD's) on 7 servers , 3TB each HDD ~ 42 TB raw space in total. ceph -s osdmap e4055: 14 osds: 14 up, 14 in pgmap v8073416: 1920 pgs, 6 pools, 16777 GB data, 4196 kobjects …
virgism
  • 83
  • 1
  • 7
7
votes
1 answer

Better performance when HDD write cache is disabled? (HGST Ultrastar 7K6000 and Media Cache behavior)

Attention, please. Long read. During initial performance tests of Hitachi Ultrastar 7K6000 drives that I'm planning to use in my Ceph setup I've noticed a strange thing: write performance is better when disk write cache is disabled. I use…
J''
  • 91
  • 1
  • 6
5
votes
2 answers

Is ceph replication based on nodes or disks?

Im currently evaluating storage systems for xenserver. Because data replication is important in the case of a failure i have a question regarding replication in ceph. As far as i know every disk in a node is an osd by itself (disks are not in any…
laubed
  • 66
  • 1
  • 4
5
votes
1 answer

Why is GlusterFS so slow here?

We've set up a mirroring pair of GlusterFS servers. No special tuning, whatever came "out of the box" with GlusterFS-3.5.1 in the official RHEL6 RPM, that's what we have. The cluster works, but the performance is pretty awful. For example,…
Mikhail T.
  • 2,272
  • 1
  • 22
  • 49
5
votes
1 answer

How can I measure complex I/O activity to develop realistic benchmarking?

I need to retrieve a few metric from my service server to benchmark another stroage solution. There are various options are provied by benchmark program(I choosed FIO,http://freecode.com/projects/fio). I could easily set up io pattern with it. But I…
jinhwan
  • 183
  • 5
4
votes
1 answer

High availability for ceph storage?

Do ceph have high availability, I config 2 node like this cluster: id: 07df97db-f315-4c78-9d2a-ab85007a1856 health: HEALTH_WARN Reduced data availability: 32 pgs inactive Degraded data redundancy: 374/590…
Lê Minh Quân
  • 191
  • 12
4
votes
1 answer

Which iscsi target on linux for serving vSphere?

All such questions are a bit dated; I'm hoping things have changed in the past year. What do you guys use to serve a DRS cluster with vSphere 5.1/5.5? SCSI-3 persistent reservation support would be definitely needed, VAAI is a nice-to-have. I've…
Zoltan
  • 155
  • 11
4
votes
3 answers

Does it work to run mysql with the data directory on a CephFS mount?

We have a database cluster that uses MySQL on DRBD with Pacemaker+Corosync, which is great. My question is whether it is possible (and whether anyone has been successful) to mount a CephFS drive to /var/lib/mysql and run MySQL from there. The…
Bryan Agee
  • 1,179
  • 2
  • 10
  • 27
3
votes
1 answer

Setup ceph for storage mailserver?

I have read some article but none of them tell how to use ceph for mail server will be like. The most close to it: https://www.7layer.org/?p=313, but it was glusterFS So is there any different when I'm using cephFS as a storage pool and make a…
Lê Minh Quân
  • 191
  • 12
3
votes
1 answer

What is wrong with my initial setup to install ceph?

I have downloaded ceph-ansible to install ceph for an evaluation and I ran on an error with package dependencies and I can't find anything about how to fix this using either ceph-ansible or yum. The error I got using ceph-ansible is below: Tuesday…
Salsa
  • 151
  • 5
3
votes
1 answer

Libvirt pool cannot see or create rbd clones

While trying to get a cloned disk running from my OS snapshot I run into the problem that Libvirt cannot see existing images cloned from a snapshot. Created via: $ rbd -p vmdisks clone vmdisks/coreos_2023@base vmdisks/coreos00.disk The base image…
lhw
  • 141
  • 6
3
votes
1 answer

Ceph or Gluster for implementing big NAS

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as a block storage for virtual machines, so the…
prema
  • 33
  • 1
  • 3
1
2 3
…
10 11