Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [prometheus]

The Prometheus monitoring system.

117 questions
22
votes
6 answers

How to find out the number of time series stored in Prometheus LevelDB

i'm responsible for maintaining the Prometheus servers in our company. The metrics however are provided by the teams. Is there a way to find out the number of time series stored in the Prometheus datadase? We are using the default LevelDB data…
Tobias Wiesenthal
  • 363
  • 1
  • 2
  • 7
13
votes
1 answer

How to split out Prometheus config files?

Right now we're using Prometheus for our monitoring and we have a lot of config (our prometheus.yml main config file is 1400+ lines long). I would like to split this out into logical groupings (maybe DEV/TEST/PROD?) but I can't seem to find any…
srkiNZ84
  • 531
  • 1
  • 6
  • 10
8
votes
2 answers

Prometheus Alert Rule for Absent Discovered Target

I'm trying to write a general rule to fire alert when a discovered target goes missing. In particular kubernetes pods annotated for scraping and auto-discovered using kubernetes_sd_configs. Expressions of the form:…
Budric
  • 203
  • 1
  • 3
  • 6
8
votes
1 answer

Prometheus: no metric node_cpu

I've setup Prometheus and node_exporter. All example graphs I've found utilize node_cpu but I don't have this metric present. The closest I've come is node_cpu_seconds_total. Am I missing something or is my version of Prometheus incompatible with…
Chris
  • 85
  • 1
  • 5
5
votes
2 answers

Smoothing Grafana graph

I'm setting up montinoring on my company infrastructure. I installed Grafana with the Prometheus Datasource. I built my dashboards and have rather "Spiky" graphs. I was wondering if there was any way to smooth it up a bit so it would be more…
Ror
  • 111
  • 2
  • 12
4
votes
1 answer

Prometheus - Use case of service discovery by role endpoints and role pod in Kubernetes

While reading Prometheus Configuration documentation and some sample scrape configurations, I found some kubernetes_sd_configs with role service & role endpoints & role pod - job_name: kube-state-metrics kubernetes_sd_configs: - role: endpoints …
jiangwei
  • 41
  • 1
  • 3
4
votes
1 answer

Prometheus node_exporter metrics - huge spikes in Grafana

Our setup includes: a few Debian 9.12 nodes with Prometheus node_exporter v. 0.18.1 installed as service Prometheus server v. 2.14.0 (on Windows Server 2016) scraping the metrics from nodes Grafana visualizing metrics Basically, our load could be…
Arseny V.
  • 85
  • 2
  • 10
4
votes
1 answer

Do prometheus metrics have some sort of "freshness" lifetime?

I'm scraping some metrics (openstack cinder volume sizes) every 15 minutes, and the results produce a discontinuous graph, like this: (That's the result of the simple query cinder_volume_size_gb). The metrics "exist" for about five minutes, but…
larsks
  • 41,276
  • 13
  • 117
  • 170
3
votes
1 answer

Monitor Gitea with Prometheus

I have deployed Gitea. How can I want to monitor it with Prometheus ? I have tried to scape the /metrics but I get no metrics (HTTP/404)
Franklin Piat
  • 736
  • 6
  • 22
3
votes
2 answers

Prometheus: scrape interval is 1m, but resolution is still 15s

tl;dr: My scrape interval is 1m, yet I have a 15s resolution. Why? My prometheus configuration includes a job to scrape kong metrics: - job_name: kong_blue honor_timestamps: true scrape_interval: 1m scrape_timeout: 10s metrics_path:…
aspyct
  • 340
  • 6
  • 19
3
votes
0 answers

Prometheus HA setup with InfluxDB backend

I'm running Prometheus HA setup according to official FAQ (two prometheus instances scraping the same data) and would like to use InfluxDB as a remote storage backend. Should I be worried about having duplicate data points in the database? Is there…
Beli
  • 258
  • 1
  • 2
  • 5
3
votes
3 answers

Set Grafana dashboard variable from Prometheus scalar

I need to use the number of CPU cores in multiple calculations on a Grafana dashboard and was hoping to perform the query once and store in a variable. I can't find a way to return the data in a format that Grafana will accept. Grafana…
virullius
  • 988
  • 8
  • 22
3
votes
1 answer

Stackdriver vs Prometheus - what is the main differences?

Moving from on-premise monlotic architechture style to microservices running in GCP on Kubernetes Cluster. I would like to understand Stackdriver compared to Prometheus. I can not get what the difference is. What can Stackdriver do what Prometheus…
CodeR
  • 39
  • 2
3
votes
1 answer

How to monitor REST API latency with Prometheus and Grafana

I would like to monitor request processing latency on my Tomcat server. It would be nice to perform some alerting based on this later. I configured Tomcat valve with %D pattern so it will log time taken to process each request in access log file.…
Ninja
  • 31
  • 1
  • 4
3
votes
1 answer

Kubernetes Nginx Ingress Controller Metrics

I've tried to find a documentation about the metrics exposed by the NGINX ingress controller in Kubernetes but so far I haven't found any reliable source about the metrics and what they mean. For example, there are three different request_size…
MysteriousPerson
  • 188
  • 7
1
2 3 4 5 6 7 8