We are facing one question about monitoring volume storage.
We use metrics "kubelet_volume_stats_used_bytes" and "kubelet_volume_stats_capacity_bytes". On our lab we have > 100 volumes mounted to pods, and among these volumes there are 14 volumes created by Glusterfs storage, the rest volumes are provided by Ceph.
$ kubectl get sc
NAME PROVISIONER AGE
default ceph.rook.io/block 166d
glusterfs-storageclass kubernetes.io/glusterfs 166d
local-storage kubernetes.io/no-provisioner 166d
non-repl ceph.rook.io/block 166d
rook-ceph-block-rep (default) ceph.rook.io/block 166d
We try to curl metrics from Kubernetes API
# curl -k -s -H "Authorization: Bearer <token>" https://kubernetes.default.svc:443/api/v1/nodes/<hostname>/proxy/metrics 2>&1 |grep kubelet_volume_stats_used_bytes
And unfortunately among all nodes we only get info from those 14 Glusterfs volumes - no metrics from the rest (created by Ceph)
So, is it possible to get metrics from those Ceph created volumes? If possible, how can we do to make Kubernetes provide them?
Thanks in advance.
BTW - we are not sure about Ceph because we also noticed on Kubernetes doc there is a chapter storage provisioner and this is what it says,
Volume Plugin Internal Provisioner Config Example
CephFS - -
GlusterFS ✓ Glusterfs
Don't know if this is the reason. Kubernetes version 1.12.3
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.3",
GitCommit:"435f92c719f279a3a67808c80521ea17d5715c66", GitTreeState:"clean",
BuildDate:"2018-11-26T12:57:14Z", GoVersion:"go1.10.4", Compiler:"gc",
Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.3",
GitCommit:"435f92c719f279a3a67808c80521ea17d5715c66", GitTreeState:"clean",
BuildDate:"2018-11-26T12:46:57Z", GoVersion:"go1.10.4", Compiler:"gc",
Platform:"linux/amd64"}
