Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [nvme]

98 questions
4
votes
1 answer

Encrypt disks using SED and store keys in TPM?

I'm buying servers lately and all of them have disks that support TCG Opal full-disk encryption (aka SED). What I'd like to do is: Store data encrypted-at-rest on the disks (NVMe & SAS). Not be required to enter a password/passphrase at server…
Evan
  • 307
  • 1
  • 4
  • 12
4
votes
1 answer

Is it worse to go with Samsung 960 EVO instead of Pro version in terms of price to perforamce measure?

We are about to deploy shared storage researching for ultra-fast storage to implement Microsoft SQL Server Failover Cluster (FCI). So far the project goes, we would to start with 500K IOPS and have ability of grows up to 2M IOPS in a year or so, due…
Joshua Turnwell
  • 530
  • 3
  • 12
4
votes
1 answer

NVMe latency monitoring

Is there a way to get read/write queue latencies from NVMe devices? If it's not split for read/write that's better than nothing but ideally I could see these separately. I'd like to go below the usual millisecond level and haven't found anything so…
Zoltan
  • 155
  • 11
4
votes
3 answers

Get exact SMART error from Intel RST (Rapid Storage Technology)

On one system I use dual NVMe m.2 drives on an Intel Z170 board in RAID0 with Intel RST. After about 6 months of light use the Intel RST Tool gives me a SMART error. The only option I have is to suppress the error. Instead of suppressing I would…
user643011
  • 129
  • 1
  • 10
4
votes
2 answers

nvme device dropouts - I/O 0 QID 0 timeout, controller disabled

We have 6 Supermicro servers all of the same (or very similar spec), Over the last two weeks one of them has been dropped an NVMe disk at random times due to: [ 66.856719] nvme 0000:03:00.0: I/O 0 QID 0 timeout, disable controller [ 66.957911]…
s_mcleod
  • 41
  • 1
  • 3
4
votes
1 answer

NVMe drives write performance in software RAID 1

We have just received two brand new Supermicro servers 1028U-TN10RT+ with 10 NVMe slots, two of them with Intel DC P3600 800gb hard drives. We were eager to test the performance of the drives as the specification is promising very good read (up to…
markz
  • 173
  • 1
  • 4
4
votes
2 answers

Map PCIe slot number (or BDF) to device number or vice versa

Is there any way to know the drive number from BDF or slot number of a PCIe or vice versa? Suppose I have four PCIe drives attached to my system. And I know their BDF's, I want to know each of the drives corresponding drive number in OS.…
kate
  • 41
  • 2
4
votes
3 answers

How can I override IRQ affinity for NVME devices

I am trying to move all interrupts over to cores 0-3 to keep the rest of my cores free for high speed, low latency virtualization. I wrote a quick script to set IRQ affinity to 0-3: #!/bin/bash while IFS= read -r LINE; do echo "0-3 ->…
Hubro
  • 1,098
  • 3
  • 16
  • 35
3
votes
1 answer

Intel NVME performance on Linux

I have an Intel 760P NVME drive hooked up to a Supermicro X11SRM-F with a Xeon W-2155 and 64GB of ddr4-2400 RAM. The specs for this drive claim 205K-265K IOPS (whatever 8GB span means) with about 3G/s random-read and 1.3G/s random write. I've…
Server Fault
  • 3,454
  • 7
  • 48
  • 88
3
votes
0 answers

How to setup nvmetcli for FC loopback mode?

I have a NVMe SSD, KC1000, from Kingston and a HBA, QLE2672 2-port 16Gb Fibre Channel Adapter, from QLogic and are installed on Linux kernel 4.16.6. Then I would like to set up a scenario which has a fc loop mode which is to use the sole FC HBA as…
codexplorer
  • 181
  • 5
3
votes
4 answers

ZFS: Configuration advice 1x NVMe as ARC and ZIL and 4x SSDs for zvols for virtualization

So recently upon testing a ZoL system we found out poor performance of random and sequential reads and poor performance of random writes on our SSDs. Our system is a stripe of 2x Samsung 1TB 850Evo SSDs to test ZFS performance and it was abysmal…
Ajay
  • 61
  • 1
  • 6
3
votes
1 answer

NVMe speed degradation after kernel upgrade

We are testing the speed of two Intel DC P3600 NVMe drives on a CentOS 7.1 system. They are configured in a software RAID1 using mdadm. After upgrading the kernel from the (very old) 3.10 to the latest 4.3 (or 4.2.5) we see can observe a very…
markz
  • 173
  • 1
  • 4
3
votes
1 answer

Dell R630 supported PCIe to NVMe (m.2) adapter and drive?

I can't seem to find any information anywhere about supported PCIe to NVMe (m.2) adapter and a brand of drive that will also work. I am aware third party adapters and drives tend to work with various OSs (ESXi) but are not compatible and cause the…
ehammer
  • 75
  • 1
  • 10
3
votes
5 answers

Linus/ext4/nvme crashes during high io

During mvn compilation, I have random crashes. The problem seems related to high IO and in kern.log, I can see things like: kernel: [158430.895045] nvme nvme1: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0x10 kernel: [158430.951331]…
Brimstedt
  • 151
  • 1
  • 12
2
votes
2 answers

Supermicro NVME ports - how to use?

So, my supermicro X11 Dai-N has 8 pci-e lanes routed to two dedicated nvme ports. I just can’t figure out how to connect my nvme harddisks to them. What’s the name of this port specification? Is there some special cable required? If so, what’s the…
matthias_buehlmann
  • 448
  • 1
  • 4
  • 17
1
2
3 4 5 6 7